climateprediction.net home page
Task 22447584

Task 22447584

Name oifs_43r3_bl_a4co_2016092300_20_1018_12293967_0
Workunit 12293967
Created 13 Jun 2024, 4:10:45 UTC
Sent 13 Jun 2024, 16:56:02 UTC
Report deadline 12 Aug 2024, 16:56:02 UTC
Received 14 Jun 2024, 20:19:33 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1537687
Run time 16 hours 29 min 13 sec
CPU time 15 hours 16 min 43 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 7.20 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,021.90 MB
Peak swap size 5,981.07 MB
Peak disk usage 1,375.59 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<stderr_txt>
ook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:854692:854692] [20240614:151646:1718378206:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:854692:854692] [20240614:151646:1718378206:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:854692:854692] [20240614:151646:1718378206:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:box.zenlime.org:1:1:854692:854692] [20240614:151646:1718378206:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:854692:854692] [20240614:151646:1718378206:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:854692:854692] [20240614:151646:1718378206:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:854692:854692] [20240614:151646:1718378206:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:854692:854692] [20240614:151646:1718378206:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:854692:854692] [20240614:151646:1718378206:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:854692:854692] [20240614:151646:1718378206:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:854692:854692] [20240614:151646:1718378206:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:854692:854692] [20240614:151646:1718378206:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:854692:854692] [20240614:151646:1718378206:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  15:17:17 STEP 1488 H= 372:00 +CPU= 26.441
  15:17:41 STEP 1489 H= 372:15 +CPU= 23.614
  15:18:05 STEP 1490 H= 372:30 +CPU= 23.990
  15:18:29 STEP 1491 H= 372:45 +CPU= 24.022
  15:18:55 STEP 1492 H= 373:00 +CPU= 25.924
  15:19:24 STEP 1493 H= 373:15 +CPU= 28.823
  15:19:53 STEP 1494 H= 373:30 +CPU= 28.994
  15:20:23 STEP 1495 H= 373:45 +CPU= 28.792
  15:20:53 STEP 1496 H= 374:00 +CPU= 29.553
  15:21:22 STEP 1497 H= 374:15 +CPU= 29.276
  15:21:52 STEP 1498 H= 374:30 +CPU= 29.467
  15:22:22 STEP 1499 H= 374:45 +CPU= 28.930
  15:22:59 STEP 1500 H= 375:00 +CPU= 36.174
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001500
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001500
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001500
  15:23:28 STEP 1501 H= 375:15 +CPU= 29.149
  15:23:58 STEP 1502 H= 375:30 +CPU= 28.962
  15:24:28 STEP 1503 H= 375:45 +CPU= 29.534
  15:24:58 STEP 1504 H= 376:00 +CPU= 30.025
  15:25:28 STEP 1505 H= 376:15 +CPU= 29.443
  15:25:56 STEP 1506 H= 376:30 +CPU= 28.567
  15:26:26 STEP 1507 H= 376:45 +CPU= 29.334
  15:26:56 STEP 1508 H= 377:00 +CPU= 30.211
  15:27:27 STEP 1509 H= 377:15 +CPU= 29.340
  15:27:56 STEP 1510 H= 377:30 +CPU= 29.210
  15:28:26 STEP 1511 H= 377:45 +CPU= 29.077
  15:29:01 STEP 1512 H= 378:00 +CPU= 35.608
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001512
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001512
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001512
  15:29:31 STEP 1513 H= 378:15 +CPU= 28.817
  15:30:00 STEP 1514 H= 378:30 +CPU= 28.575
  15:30:29 STEP 1515 H= 378:45 +CPU= 29.141
  15:30:59 STEP 1516 H= 379:00 +CPU= 29.793
  15:31:29 STEP 1517 H= 379:15 +CPU= 28.802
  15:31:58 STEP 1518 H= 379:30 +CPU= 29.396
  15:32:28 STEP 1519 H= 379:45 +CPU= 29.107
  15:32:57 STEP 1520 H= 380:00 +CPU= 29.259
  15:33:26 STEP 1521 H= 380:15 +CPU= 29.076
  15:33:56 STEP 1522 H= 380:30 +CPU= 28.692
  15:34:25 STEP 1523 H= 380:45 +CPU= 28.908
  15:35:00 STEP 1524 H= 381:00 +CPU= 34.830
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001524
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001524
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001524
  15:35:30 STEP 1525 H= 381:15 +CPU= 28.955
  15:35:59 STEP 1526 H= 381:30 +CPU= 29.424
  15:36:29 STEP 1527 H= 381:45 +CPU= 29.348
  15:36:59 STEP 1528 H= 382:00 +CPU= 29.320
  15:37:28 STEP 1529 H= 382:15 +CPU= 29.095
  15:37:57 STEP 1530 H= 382:30 +CPU= 28.764
  15:38:26 STEP 1531 H= 382:45 +CPU= 29.146
  15:38:56 STEP 1532 H= 383:00 +CPU= 29.354
  15:39:26 STEP 1533 H= 383:15 +CPU= 29.523
  15:39:55 STEP 1534 H= 383:30 +CPU= 29.187
  15:40:25 STEP 1535 H= 383:45 +CPU= 29.245
  15:41:05 STEP 1536 H= 384:00 +CPU= 38.478
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001536
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001536
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001536
  15:41:35 STEP 1537 H= 384:15 +CPU= 28.451
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001536
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4co_2016092300_20_1018_12293967_0_r611390959_15.zip
Uploading the intermediate file: upload_file_15.zip
  15:42:04 STEP 1538 H= 384:30 +CPU= 28.889
  15:42:34 STEP 1539 H= 384:45 +CPU= 28.801
  15:43:03 STEP 1540 H= 385:00 +CPU= 29.312
  15:43:33 STEP 1541 H= 385:15 +CPU= 29.093
  15:44:02 STEP 1542 H= 385:30 +CPU= 28.658
  15:44:31 STEP 1543 H= 385:45 +CPU= 29.342
  15:45:02 STEP 1544 H= 386:00 +CPU= 29.296
  15:45:31 STEP 1545 H= 386:15 +CPU= 29.082
  15:46:01 STEP 1546 H= 386:30 +CPU= 29.104
  15:46:31 STEP 1547 H= 386:45 +CPU= 29.219
  15:47:08 STEP 1548 H= 387:00 +CPU= 36.542
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001548
  15:47:39 STEP 1549 H= 387:15 +CPU= 29.296
  15:48:10 STEP 1550 H= 387:30 +CPU= 29.555
  15:48:39 STEP 1551 H= 387:45 +CPU= 28.702
  15:49:10 STEP 1552 H= 388:00 +CPU= 30.265
  15:49:40 STEP 1553 H= 388:15 +CPU= 29.647
  15:50:10 STEP 1554 H= 388:30 +CPU= 29.687
  15:50:40 STEP 1555 H= 388:45 +CPU= 29.050
  15:51:10 STEP 1556 H= 389:00 +CPU= 29.890
  15:51:40 STEP 1557 H= 389:15 +CPU= 29.283
  15:52:10 STEP 1558 H= 389:30 +CPU= 29.769
  15:52:38 STEP 1559 H= 389:45 +CPU= 28.176
  15:53:15 STEP 1560 H= 390:00 +CPU= 36.487
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001560
  15:53:45 STEP 1561 H= 390:15 +CPU= 29.098
  15:54:15 STEP 1562 H= 390:30 +CPU= 29.372
  15:54:44 STEP 1563 H= 390:45 +CPU= 28.424
  15:55:15 STEP 1564 H= 391:00 +CPU= 30.229
  15:55:45 STEP 1565 H= 391:15 +CPU= 29.935
  15:56:16 STEP 1566 H= 391:30 +CPU= 29.826
  15:56:45 STEP 1567 H= 391:45 +CPU= 28.930
  15:57:16 STEP 1568 H= 392:00 +CPU= 30.059
  15:57:45 STEP 1569 H= 392:15 +CPU= 28.444
  15:58:14 STEP 1570 H= 392:30 +CPU= 28.772
  15:58:44 STEP 1571 H= 392:45 +CPU= 29.191
  15:59:20 STEP 1572 H= 393:00 +CPU= 35.785
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001572
  15:59:50 STEP 1573 H= 393:15 +CPU= 28.957
  16:00:18 STEP 1574 H= 393:30 +CPU= 28.471
  16:00:47 STEP 1575 H= 393:45 +CPU= 28.997
  16:01:17 STEP 1576 H= 394:00 +CPU= 29.486
  16:01:46 STEP 1577 H= 394:15 +CPU= 29.131
  16:02:16 STEP 1578 H= 394:30 +CPU= 29.221
  16:02:45 STEP 1579 H= 394:45 +CPU= 28.959
  16:03:15 STEP 1580 H= 395:00 +CPU= 29.919
  16:03:44 STEP 1581 H= 395:15 +CPU= 28.860
  16:04:13 STEP 1582 H= 395:30 +CPU= 28.928
  16:04:41 STEP 1583 H= 395:45 +CPU= 28.279
  16:05:22 STEP 1584 H= 396:00 +CPU= 38.107
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001584
  16:05:52 STEP 1585 H= 396:15 +CPU= 28.801
  16:06:21 STEP 1586 H= 396:30 +CPU= 29.396
  16:06:51 STEP 1587 H= 396:45 +CPU= 28.798
  16:07:22 STEP 1588 H= 397:00 +CPU= 29.335
  16:07:52 STEP 1589 H= 397:15 +CPU= 28.784
  16:08:22 STEP 1590 H= 397:30 +CPU= 29.264
  16:08:52 STEP 1591 H= 397:45 +CPU= 28.717
  16:09:24 STEP 1592 H= 398:00 +CPU= 29.994
  16:09:53 STEP 1593 H= 398:15 +CPU= 29.004
  16:10:23 STEP 1594 H= 398:30 +CPU= 29.436
  16:10:52 STEP 1595 H= 398:45 +CPU= 28.804
  16:11:27 STEP 1596 H= 399:00 +CPU= 35.190
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001596
  16:11:57 STEP 1597 H= 399:15 +CPU= 29.757
  16:12:26 STEP 1598 H= 399:30 +CPU= 28.985
  16:12:56 STEP 1599 H= 399:45 +CPU= 29.017
  16:13:26 STEP 1600 H= 400:00 +CPU= 29.561
  16:13:55 STEP 1601 H= 400:15 +CPU= 29.261
  16:14:24 STEP 1602 H= 400:30 +CPU= 29.096
  16:14:54 STEP 1603 H= 400:45 +CPU= 29.275
  16:15:24 STEP 1604 H= 401:00 +CPU= 29.760
  16:15:54 STEP 1605 H= 401:15 +CPU= 29.581
  16:16:23 STEP 1606 H= 401:30 +CPU= 29.003
  16:16:53 STEP 1607 H= 401:45 +CPU= 29.440
  16:17:29 STEP 1608 H= 402:00 +CPU= 36.460
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001608
  16:17:58 STEP 1609 H= 402:15 +CPU= 29.073
  16:18:28 STEP 1610 H= 402:30 +CPU= 29.336
  16:18:57 STEP 1611 H= 402:45 +CPU= 29.261
  16:19:28 STEP 1612 H= 403:00 +CPU= 30.424
  16:19:58 STEP 1613 H= 403:15 +CPU= 29.990
  16:20:28 STEP 1614 H= 403:30 +CPU= 29.718
  16:20:58 STEP 1615 H= 403:45 +CPU= 29.767
  16:21:28 STEP 1616 H= 404:00 +CPU= 30.087
  16:21:58 STEP 1617 H= 404:15 +CPU= 29.518
  16:22:27 STEP 1618 H= 404:30 +CPU= 28.745
  16:22:56 STEP 1619 H= 404:45 +CPU= 28.926
  16:23:34 STEP 1620 H= 405:00 +CPU= 36.620
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001620
  16:24:03 STEP 1621 H= 405:15 +CPU= 29.230
  16:24:32 STEP 1622 H= 405:30 +CPU= 28.909
  16:25:02 STEP 1623 H= 405:45 +CPU= 29.393
  16:25:32 STEP 1624 H= 406:00 +CPU= 30.380
  16:26:02 STEP 1625 H= 406:15 +CPU= 29.530
  16:26:31 STEP 1626 H= 406:30 +CPU= 28.885
  16:27:00 STEP 1627 H= 406:45 +CPU= 29.610
  16:27:30 STEP 1628 H= 407:00 +CPU= 29.860
  16:28:00 STEP 1629 H= 407:15 +CPU= 29.054
  16:28:30 STEP 1630 H= 407:30 +CPU= 28.799
  16:28:58 STEP 1631 H= 407:45 +CPU= 28.511
  16:29:39 STEP 1632 H= 408:00 +CPU= 39.021
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001632
  16:30:08 STEP 1633 H= 408:15 +CPU= 28.493
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001632
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4co_2016092300_20_1018_12293967_0_r611390959_16.zip
Uploading the intermediate file: upload_file_16.zip
  16:30:38 STEP 1634 H= 408:30 +CPU= 28.194
  16:31:07 STEP 1635 H= 408:45 +CPU= 28.544
  16:31:36 STEP 1636 H= 409:00 +CPU= 29.528
  16:32:06 STEP 1637 H= 409:15 +CPU= 29.131
  16:32:36 STEP 1638 H= 409:30 +CPU= 29.633
  16:33:06 STEP 1639 H= 409:45 +CPU= 29.565
  16:33:36 STEP 1640 H= 410:00 +CPU= 29.711
  16:34:06 STEP 1641 H= 410:15 +CPU= 29.010
  16:34:35 STEP 1642 H= 410:30 +CPU= 28.803
  16:35:04 STEP 1643 H= 410:45 +CPU= 28.700
  16:35:40 STEP 1644 H= 411:00 +CPU= 35.156
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001644
  16:36:10 STEP 1645 H= 411:15 +CPU= 29.249
  16:36:40 STEP 1646 H= 411:30 +CPU= 29.118
  16:37:09 STEP 1647 H= 411:45 +CPU= 28.982
  16:37:39 STEP 1648 H= 412:00 +CPU= 29.656
  16:38:08 STEP 1649 H= 412:15 +CPU= 29.342
  16:38:38 STEP 1650 H= 412:30 +CPU= 29.500
  16:39:07 STEP 1651 H= 412:45 +CPU= 29.324
  16:39:37 STEP 1652 H= 413:00 +CPU= 29.653
  16:40:06 STEP 1653 H= 413:15 +CPU= 29.213
  16:40:36 STEP 1654 H= 413:30 +CPU= 29.189
  16:41:06 STEP 1655 H= 413:45 +CPU= 29.574
  16:41:42 STEP 1656 H= 414:00 +CPU= 35.588
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001656
  16:42:10 STEP 1657 H= 414:15 +CPU= 28.491
  16:42:40 STEP 1658 H= 414:30 +CPU= 29.185
  16:43:09 STEP 1659 H= 414:45 +CPU= 29.164
  16:43:39 STEP 1660 H= 415:00 +CPU= 29.981
  16:44:09 STEP 1661 H= 415:15 +CPU= 29.577
  16:44:38 STEP 1662 H= 415:30 +CPU= 29.132
  16:45:07 STEP 1663 H= 415:45 +CPU= 29.163
  16:45:38 STEP 1664 H= 416:00 +CPU= 30.209
  16:46:07 STEP 1665 H= 416:15 +CPU= 28.921
  16:46:36 STEP 1666 H= 416:30 +CPU= 28.884
  16:47:05 STEP 1667 H= 416:45 +CPU= 29.351
  16:47:41 STEP 1668 H= 417:00 +CPU= 35.945
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001668
  16:48:11 STEP 1669 H= 417:15 +CPU= 28.869
  16:48:40 STEP 1670 H= 417:30 +CPU= 29.228
  16:49:09 STEP 1671 H= 417:45 +CPU= 28.691
  16:49:39 STEP 1672 H= 418:00 +CPU= 29.644
  16:50:08 STEP 1673 H= 418:15 +CPU= 29.054
  16:50:37 STEP 1674 H= 418:30 +CPU= 28.913
  16:51:07 STEP 1675 H= 418:45 +CPU= 29.343
  16:51:37 STEP 1676 H= 419:00 +CPU= 29.927
  16:52:06 STEP 1677 H= 419:15 +CPU= 28.734
  16:52:36 STEP 1678 H= 419:30 +CPU= 27.995
  16:53:07 STEP 1679 H= 419:45 +CPU= 29.255
  16:53:47 STEP 1680 H= 420:00 +CPU= 39.589
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001680
  16:54:16 STEP 1681 H= 420:15 +CPU= 28.568
  16:54:45 STEP 1682 H= 420:30 +CPU= 28.924
  16:55:14 STEP 1683 H= 420:45 +CPU= 27.841
  16:55:44 STEP 1684 H= 421:00 +CPU= 29.418
  16:56:13 STEP 1685 H= 421:15 +CPU= 29.419
  16:56:42 STEP 1686 H= 421:30 +CPU= 27.949
  16:57:11 STEP 1687 H= 421:45 +CPU= 29.118
  16:57:41 STEP 1688 H= 422:00 +CPU= 29.848
  16:58:10 STEP 1689 H= 422:15 +CPU= 29.116
  16:58:39 STEP 1690 H= 422:30 +CPU= 28.940
  16:59:09 STEP 1691 H= 422:45 +CPU= 29.120
  16:59:44 STEP 1692 H= 423:00 +CPU= 35.438
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001692
  17:00:13 STEP 1693 H= 423:15 +CPU= 29.047
  17:00:42 STEP 1694 H= 423:30 +CPU= 29.239
  17:01:12 STEP 1695 H= 423:45 +CPU= 29.528
  17:01:42 STEP 1696 H= 424:00 +CPU= 29.621
  17:02:11 STEP 1697 H= 424:15 +CPU= 29.404
  17:02:40 STEP 1698 H= 424:30 +CPU= 29.037
  17:03:09 STEP 1699 H= 424:45 +CPU= 28.810
  17:03:39 STEP 1700 H= 425:00 +CPU= 29.576
  17:04:08 STEP 1701 H= 425:15 +CPU= 29.286
  17:04:37 STEP 1702 H= 425:30 +CPU= 29.041
  17:05:07 STEP 1703 H= 425:45 +CPU= 29.215
  17:05:42 STEP 1704 H= 426:00 +CPU= 35.357
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001704
  17:06:12 STEP 1705 H= 426:15 +CPU= 29.306
  17:06:41 STEP 1706 H= 426:30 +CPU= 28.960
  17:07:11 STEP 1707 H= 426:45 +CPU= 29.599
  17:07:41 STEP 1708 H= 427:00 +CPU= 30.077
  17:08:11 STEP 1709 H= 427:15 +CPU= 29.126
  17:08:40 STEP 1710 H= 427:30 +CPU= 29.395
  17:09:11 STEP 1711 H= 427:45 +CPU= 29.812
  17:09:41 STEP 1712 H= 428:00 +CPU= 30.059
  17:10:11 STEP 1713 H= 428:15 +CPU= 29.616
  17:10:41 STEP 1714 H= 428:30 +CPU= 29.241
  17:11:11 STEP 1715 H= 428:45 +CPU= 29.811
  17:11:48 STEP 1716 H= 429:00 +CPU= 36.363
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001716
  17:12:17 STEP 1717 H= 429:15 +CPU= 28.822
  17:12:47 STEP 1718 H= 429:30 +CPU= 29.769
  17:13:17 STEP 1719 H= 429:45 +CPU= 29.655
  17:13:47 STEP 1720 H= 430:00 +CPU= 30.115
  17:14:17 STEP 1721 H= 430:15 +CPU= 29.729
  17:14:47 STEP 1722 H= 430:30 +CPU= 29.517
  17:15:16 STEP 1723 H= 430:45 +CPU= 29.555
  17:15:47 STEP 1724 H= 431:00 +CPU= 30.581
  17:16:16 STEP 1725 H= 431:15 +CPU= 28.581
  17:16:43 STEP 1726 H= 431:30 +CPU= 26.819
  17:17:08 STEP 1727 H= 431:45 +CPU= 25.068
  17:17:44 STEP 1728 H= 432:00 +CPU= 36.437
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001728
  17:18:12 STEP 1729 H= 432:15 +CPU= 27.375
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001728
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4co_2016092300_20_1018_12293967_0_r611390959_17.zip
Uploading the intermediate file: upload_file_17.zip
  17:18:39 STEP 1730 H= 432:30 +CPU= 27.068
  17:19:05 STEP 1731 H= 432:45 +CPU= 26.403
  17:19:33 STEP 1732 H= 433:00 +CPU= 27.522
  17:20:02 STEP 1733 H= 433:15 +CPU= 28.919
  17:20:32 STEP 1734 H= 433:30 +CPU= 29.435
  17:21:01 STEP 1735 H= 433:45 +CPU= 28.928
  17:21:31 STEP 1736 H= 434:00 +CPU= 29.770
  17:22:01 STEP 1737 H= 434:15 +CPU= 28.569
  17:22:30 STEP 1738 H= 434:30 +CPU= 29.017
  17:23:01 STEP 1739 H= 434:45 +CPU= 29.453
  17:23:37 STEP 1740 H= 435:00 +CPU= 35.504
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001740
  17:24:06 STEP 1741 H= 435:15 +CPU= 28.425
  17:24:35 STEP 1742 H= 435:30 +CPU= 28.985
  17:25:04 STEP 1743 H= 435:45 +CPU= 28.778
  17:25:33 STEP 1744 H= 436:00 +CPU= 29.279
  17:26:02 STEP 1745 H= 436:15 +CPU= 28.584
  17:26:31 STEP 1746 H= 436:30 +CPU= 28.662
  17:27:00 STEP 1747 H= 436:45 +CPU= 29.023
  17:27:31 STEP 1748 H= 437:00 +CPU= 29.514
  17:28:01 STEP 1749 H= 437:15 +CPU= 27.807
  17:28:30 STEP 1750 H= 437:30 +CPU= 29.182
  17:29:00 STEP 1751 H= 437:45 +CPU= 28.980
  17:29:35 STEP 1752 H= 438:00 +CPU= 34.687
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001752
  17:30:05 STEP 1753 H= 438:15 +CPU= 28.819
  17:30:35 STEP 1754 H= 438:30 +CPU= 29.067
  17:31:04 STEP 1755 H= 438:45 +CPU= 28.958
  17:31:34 STEP 1756 H= 439:00 +CPU= 29.055
  17:32:03 STEP 1757 H= 439:15 +CPU= 28.928
  17:32:32 STEP 1758 H= 439:30 +CPU= 28.757
  17:33:02 STEP 1759 H= 439:45 +CPU= 29.204
  17:33:32 STEP 1760 H= 440:00 +CPU= 29.585
  17:34:01 STEP 1761 H= 440:15 +CPU= 28.714
  17:34:30 STEP 1762 H= 440:30 +CPU= 28.803
  17:34:59 STEP 1763 H= 440:45 +CPU= 28.968
  17:35:35 STEP 1764 H= 441:00 +CPU= 35.302
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001764
  17:36:05 STEP 1765 H= 441:15 +CPU= 29.237
  17:36:34 STEP 1766 H= 441:30 +CPU= 29.590
  17:37:03 STEP 1767 H= 441:45 +CPU= 28.965
  17:37:33 STEP 1768 H= 442:00 +CPU= 29.777
  17:38:03 STEP 1769 H= 442:15 +CPU= 29.253
  17:38:32 STEP 1770 H= 442:30 +CPU= 29.354
  17:39:02 STEP 1771 H= 442:45 +CPU= 29.133
  17:39:32 STEP 1772 H= 443:00 +CPU= 28.727
  17:40:01 STEP 1773 H= 443:15 +CPU= 28.239
  17:40:30 STEP 1774 H= 443:30 +CPU= 28.381
  17:40:59 STEP 1775 H= 443:45 +CPU= 28.848
  17:41:39 STEP 1776 H= 444:00 +CPU= 39.685
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001776
  17:42:09 STEP 1777 H= 444:15 +CPU= 27.435
  17:42:38 STEP 1778 H= 444:30 +CPU= 29.418
  17:43:08 STEP 1779 H= 444:45 +CPU= 28.978
  17:43:38 STEP 1780 H= 445:00 +CPU= 29.984
  17:44:07 STEP 1781 H= 445:15 +CPU= 28.675
  17:44:36 STEP 1782 H= 445:30 +CPU= 27.533
  17:45:05 STEP 1783 H= 445:45 +CPU= 29.235
  17:45:36 STEP 1784 H= 446:00 +CPU= 29.694
  17:46:06 STEP 1785 H= 446:15 +CPU= 29.277
  17:46:33 STEP 1786 H= 446:30 +CPU= 27.594
  17:46:59 STEP 1787 H= 446:45 +CPU= 25.629
  17:47:31 STEP 1788 H= 447:00 +CPU= 32.333
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001788
  17:47:59 STEP 1789 H= 447:15 +CPU= 27.734
  17:48:26 STEP 1790 H= 447:30 +CPU= 26.918
  17:48:54 STEP 1791 H= 447:45 +CPU= 27.564
  17:49:21 STEP 1792 H= 448:00 +CPU= 27.579
  17:49:50 STEP 1793 H= 448:15 +CPU= 28.401
  17:50:19 STEP 1794 H= 448:30 +CPU= 28.476
  17:50:49 STEP 1795 H= 448:45 +CPU= 29.296
  17:51:19 STEP 1796 H= 449:00 +CPU= 30.209
  17:51:49 STEP 1797 H= 449:15 +CPU= 29.340
  17:52:17 STEP 1798 H= 449:30 +CPU= 28.029
  17:52:46 STEP 1799 H= 449:45 +CPU= 28.513
  17:53:23 STEP 1800 H= 450:00 +CPU= 35.640
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001800
  17:53:52 STEP 1801 H= 450:15 +CPU= 28.922
  17:54:21 STEP 1802 H= 450:30 +CPU= 28.236
  17:54:50 STEP 1803 H= 450:45 +CPU= 28.980
  17:55:21 STEP 1804 H= 451:00 +CPU= 29.826
  17:55:50 STEP 1805 H= 451:15 +CPU= 29.126
  17:56:19 STEP 1806 H= 451:30 +CPU= 28.842
  17:56:49 STEP 1807 H= 451:45 +CPU= 28.829
  17:57:19 STEP 1808 H= 452:00 +CPU= 29.510
  17:57:48 STEP 1809 H= 452:15 +CPU= 28.660
  17:58:17 STEP 1810 H= 452:30 +CPU= 29.034
  17:58:46 STEP 1811 H= 452:45 +CPU= 28.198
  17:59:22 STEP 1812 H= 453:00 +CPU= 35.468
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001812
  17:59:52 STEP 1813 H= 453:15 +CPU= 29.393
  18:00:15 STEP 1814 H= 453:30 +CPU= 23.135
  18:00:35 STEP 1815 H= 453:45 +CPU= 20.647
  18:00:59 STEP 1816 H= 454:00 +CPU= 23.212
  18:01:26 STEP 1817 H= 454:15 +CPU= 27.057
  18:01:53 STEP 1818 H= 454:30 +CPU= 27.274
  18:02:18 STEP 1819 H= 454:45 +CPU= 24.422
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a4co
(argv4) batchid: 1018
(argv5) wuid: 12293967
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/7
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/7/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/7/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_591e8c761639125fb1988b62cc310403 to: /var/lib/boinc-client/slots/7/oifs_43r3_bl_a4co_2016092300_20_1018_12293967.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/7/oifs_43r3_bl_a4co_2016092300_20_1018_12293967.zip
ic_ancil_file: ic_ancil_12293967
ifsdata_file: ifsdata_12293967
climate_data_file: clim_data_12293967
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/7/ic_ancil_12293967.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/7/ic_ancil_12293967.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_6d010075bffaee92b7eaea95c35c5525 to: /var/lib/boinc-client/slots/7/ifsdata/ifsdata_12293967.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/7/ifsdata/ifsdata_12293967.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/7/159l_2/clim_data_12293967.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/7/159l_2/clim_data_12293967.zip
Checking for progress XML file: /var/lib/boinc-client/slots/7/progress_file_12293967.xml
Opened progress file ok : /var/lib/boinc-client/slots/7/progress_file_12293967.xml
-- Model is restarting --
Adjusting last_iter, 1820, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/7/progress_file_12293967.xml
last_cpu_time: 50808
upload_file_number: 18
last_iter: 1777
last_upload: 1556100
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a4co_2016092300_20_1018_12293967_0_r611390959
The child process has been launched with process id: 859877
Executing the command: /var/lib/boinc-client/slots/7/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:859877:859877] [20240614:185610:1718391370:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  18:56:46 STEP 1776 H= 444:00 +CPU= 30.594
  18:57:16 STEP 1777 H= 444:15 +CPU= 29.397
  18:57:46 STEP 1778 H= 444:30 +CPU= 29.672
  18:58:16 STEP 1779 H= 444:45 +CPU= 29.662
  18:58:46 STEP 1780 H= 445:00 +CPU= 29.923
  18:59:16 STEP 1781 H= 445:15 +CPU= 29.640
  18:59:45 STEP 1782 H= 445:30 +CPU= 28.847
  19:00:07 STEP 1783 H= 445:45 +CPU= 21.659
  19:00:32 STEP 1784 H= 446:00 +CPU= 24.889
  19:00:57 STEP 1785 H= 446:15 +CPU= 24.997
  19:01:21 STEP 1786 H= 446:30 +CPU= 24.125
  19:01:45 STEP 1787 H= 446:45 +CPU= 23.758
  19:02:16 STEP 1788 H= 447:00 +CPU= 30.780
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001788
  19:02:40 STEP 1789 H= 447:15 +CPU= 24.113
  19:03:06 STEP 1790 H= 447:30 +CPU= 25.642
  19:03:35 STEP 1791 H= 447:45 +CPU= 29.223
  19:04:05 STEP 1792 H= 448:00 +CPU= 29.882
  19:04:35 STEP 1793 H= 448:15 +CPU= 29.303
  19:05:04 STEP 1794 H= 448:30 +CPU= 29.251
  19:05:33 STEP 1795 H= 448:45 +CPU= 29.208
  19:05:55 STEP 1796 H= 449:00 +CPU= 21.227
  19:06:17 STEP 1797 H= 449:15 +CPU= 22.160
  19:06:39 STEP 1798 H= 449:30 +CPU= 22.060
  19:07:06 STEP 1799 H= 449:45 +CPU= 27.022
  19:07:38 STEP 1800 H= 450:00 +CPU= 32.067
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001800
  19:08:04 STEP 1801 H= 450:15 +CPU= 25.541
  19:08:30 STEP 1802 H= 450:30 +CPU= 25.955
  19:08:58 STEP 1803 H= 450:45 +CPU= 28.267
  19:09:29 STEP 1804 H= 451:00 +CPU= 29.733
  19:09:58 STEP 1805 H= 451:15 +CPU= 28.951
  19:10:28 STEP 1806 H= 451:30 +CPU= 29.729
  19:10:57 STEP 1807 H= 451:45 +CPU= 29.515
  19:11:28 STEP 1808 H= 452:00 +CPU= 30.118
  19:11:57 STEP 1809 H= 452:15 +CPU= 28.935
  19:12:28 STEP 1810 H= 452:30 +CPU= 29.662
  19:12:59 STEP 1811 H= 452:45 +CPU= 29.470
  19:13:36 STEP 1812 H= 453:00 +CPU= 35.448
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001812
  19:14:05 STEP 1813 H= 453:15 +CPU= 28.288
  19:14:36 STEP 1814 H= 453:30 +CPU= 29.537
  19:15:06 STEP 1815 H= 453:45 +CPU= 29.688
  19:15:36 STEP 1816 H= 454:00 +CPU= 30.189
  19:16:06 STEP 1817 H= 454:15 +CPU= 29.264
  19:16:36 STEP 1818 H= 454:30 +CPU= 29.619
  19:17:02 STEP 1819 H= 454:45 +CPU= 26.447
  19:17:23 STEP 1820 H= 455:00 +CPU= 21.248
  19:17:43 STEP 1821 H= 455:15 +CPU= 19.773
  19:18:07 STEP 1822 H= 455:30 +CPU= 23.475
  19:18:31 STEP 1823 H= 455:45 +CPU= 24.666
  19:19:05 STEP 1824 H= 456:00 +CPU= 34.016
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001824
  19:19:32 STEP 1825 H= 456:15 +CPU= 26.467
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001824
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4co_2016092300_20_1018_12293967_0_r611390959_18.zip
Uploading the intermediate file: upload_file_18.zip
  19:19:54 STEP 1826 H= 456:30 +CPU= 22.303
  19:20:23 STEP 1827 H= 456:45 +CPU= 27.960
  19:20:52 STEP 1828 H= 457:00 +CPU= 29.385
  19:21:21 STEP 1829 H= 457:15 +CPU= 28.843
  19:21:51 STEP 1830 H= 457:30 +CPU= 29.426
  19:22:20 STEP 1831 H= 457:45 +CPU= 28.894
  19:22:50 STEP 1832 H= 458:00 +CPU= 29.642
  19:23:20 STEP 1833 H= 458:15 +CPU= 28.951
  19:23:47 STEP 1834 H= 458:30 +CPU= 26.845
  19:24:13 STEP 1835 H= 458:45 +CPU= 26.619
  19:24:45 STEP 1836 H= 459:00 +CPU= 31.721
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001836
  19:25:12 STEP 1837 H= 459:15 +CPU= 26.570
  19:25:38 STEP 1838 H= 459:30 +CPU= 26.397
  19:26:04 STEP 1839 H= 459:45 +CPU= 25.773
  19:26:31 STEP 1840 H= 460:00 +CPU= 27.212
  19:26:59 STEP 1841 H= 460:15 +CPU= 27.799
  19:27:29 STEP 1842 H= 460:30 +CPU= 29.312
  19:27:58 STEP 1843 H= 460:45 +CPU= 29.631
  19:28:28 STEP 1844 H= 461:00 +CPU= 29.683
  19:28:58 STEP 1845 H= 461:15 +CPU= 29.394
  19:29:28 STEP 1846 H= 461:30 +CPU= 29.392
  19:29:57 STEP 1847 H= 461:45 +CPU= 29.035
  19:30:34 STEP 1848 H= 462:00 +CPU= 35.471
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001848
  19:31:04 STEP 1849 H= 462:15 +CPU= 29.349
  19:31:33 STEP 1850 H= 462:30 +CPU= 28.760
  19:32:02 STEP 1851 H= 462:45 +CPU= 28.730
  19:32:32 STEP 1852 H= 463:00 +CPU= 29.519
  19:33:02 STEP 1853 H= 463:15 +CPU= 29.046
  19:33:32 STEP 1854 H= 463:30 +CPU= 29.374
  19:34:00 STEP 1855 H= 463:45 +CPU= 28.533
  19:34:31 STEP 1856 H= 464:00 +CPU= 29.958
  19:35:00 STEP 1857 H= 464:15 +CPU= 28.840
  19:35:29 STEP 1858 H= 464:30 +CPU= 29.173
  19:35:59 STEP 1859 H= 464:45 +CPU= 29.244
  19:36:36 STEP 1860 H= 465:00 +CPU= 36.048
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001860
  19:37:06 STEP 1861 H= 465:15 +CPU= 29.909
  19:37:35 STEP 1862 H= 465:30 +CPU= 29.259
  19:38:05 STEP 1863 H= 465:45 +CPU= 29.279
  19:38:36 STEP 1864 H= 466:00 +CPU= 29.976
  19:39:06 STEP 1865 H= 466:15 +CPU= 29.394
  19:39:34 STEP 1866 H= 466:30 +CPU= 28.708
  19:40:04 STEP 1867 H= 466:45 +CPU= 29.475
  19:40:34 STEP 1868 H= 467:00 +CPU= 29.845
  19:41:04 STEP 1869 H= 467:15 +CPU= 29.870
  19:41:35 STEP 1870 H= 467:30 +CPU= 29.554
  19:42:05 STEP 1871 H= 467:45 +CPU= 28.712
  19:42:45 STEP 1872 H= 468:00 +CPU= 39.606
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001872
  19:43:15 STEP 1873 H= 468:15 +CPU= 29.052
  19:43:44 STEP 1874 H= 468:30 +CPU= 28.605
  19:44:13 STEP 1875 H= 468:45 +CPU= 29.312
  19:44:44 STEP 1876 H= 469:00 +CPU= 29.940
  19:45:14 STEP 1877 H= 469:15 +CPU= 29.183
  19:45:44 STEP 1878 H= 469:30 +CPU= 29.148
  19:46:14 STEP 1879 H= 469:45 +CPU= 29.530
  19:46:45 STEP 1880 H= 470:00 +CPU= 30.227
  19:47:16 STEP 1881 H= 470:15 +CPU= 30.344
  19:47:46 STEP 1882 H= 470:30 +CPU= 29.284
  19:48:15 STEP 1883 H= 470:45 +CPU= 29.241
  19:48:51 STEP 1884 H= 471:00 +CPU= 35.797
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001884
  19:49:22 STEP 1885 H= 471:15 +CPU= 29.497
  19:49:51 STEP 1886 H= 471:30 +CPU= 29.189
  19:50:21 STEP 1887 H= 471:45 +CPU= 29.945
  19:50:52 STEP 1888 H= 472:00 +CPU= 29.989
  19:51:21 STEP 1889 H= 472:15 +CPU= 29.191
  19:51:51 STEP 1890 H= 472:30 +CPU= 29.028
  19:52:21 STEP 1891 H= 472:45 +CPU= 29.673
  19:52:52 STEP 1892 H= 473:00 +CPU= 30.211
  19:53:23 STEP 1893 H= 473:15 +CPU= 29.213
  19:53:52 STEP 1894 H= 473:30 +CPU= 28.622
  19:54:22 STEP 1895 H= 473:45 +CPU= 28.895
  19:54:58 STEP 1896 H= 474:00 +CPU= 36.176
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001896
  19:55:27 STEP 1897 H= 474:15 +CPU= 29.062
  19:55:56 STEP 1898 H= 474:30 +CPU= 29.247
  19:56:26 STEP 1899 H= 474:45 +CPU= 29.325
  19:56:56 STEP 1900 H= 475:00 +CPU= 29.313
  19:57:25 STEP 1901 H= 475:15 +CPU= 29.263
  19:57:54 STEP 1902 H= 475:30 +CPU= 29.084
  19:58:24 STEP 1903 H= 475:45 +CPU= 29.368
  19:58:54 STEP 1904 H= 476:00 +CPU= 29.737
  19:59:24 STEP 1905 H= 476:15 +CPU= 29.368
  19:59:53 STEP 1906 H= 476:30 +CPU= 28.899
  20:00:23 STEP 1907 H= 476:45 +CPU= 28.705
  20:00:58 STEP 1908 H= 477:00 +CPU= 35.264
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001908
  20:01:28 STEP 1909 H= 477:15 +CPU= 29.193
  20:01:57 STEP 1910 H= 477:30 +CPU= 29.063
  20:02:26 STEP 1911 H= 477:45 +CPU= 28.835
  20:02:56 STEP 1912 H= 478:00 +CPU= 29.477
  20:03:25 STEP 1913 H= 478:15 +CPU= 28.937
  20:03:54 STEP 1914 H= 478:30 +CPU= 29.071
  20:04:24 STEP 1915 H= 478:45 +CPU= 29.500
  20:04:54 STEP 1916 H= 479:00 +CPU= 29.654
  20:05:24 STEP 1917 H= 479:15 +CPU= 29.584
  20:05:54 STEP 1918 H= 479:30 +CPU= 29.099
  20:06:22 STEP 1919 H= 479:45 +CPU= 27.574
  20:07:00 STEP 1920 H= 480:00 +CPU= 35.543
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  480.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I      0.1886584006E-21      I       0.2247977458E-06      I
ICloud fractionI      0.0000000000E+00      I       0.2318271474E-01      I
I  Relat. Hum. I      0.8490218657E-01      I       0.1851681144E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.2995718886E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    306002228 BYTES IN      7 RECORDS TRANSFERRED IN    0.0005 SECONDS612004.5 Mbytes/s, TOTAL TIME=    0.0108( 4.6%)
 20:07:00 STEP 1920 H= 480:00 +CPU= 35.543
 END CNT3
 NSTEP =  1920 CNT0   000000000  26.861  26.861  28.422   0.945               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN         4290/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1     78    0       46      1044068     4290 ************     8.6     2.0     6.6     2.9    14.2
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 4249577.8 4249577.8    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 4244890.0 4244890.0     99.89      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  157    5384.3    5384.3     19.89      0.00
   9 SPCM     - SPECTRAL COMP.                     144     942.0     942.0      3.19      0.00
  10 SCAN2M - PHYSICS                              145   12887.1   12887.1     43.97      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 12     810.6     810.6      0.23      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                  7      88.6      88.6      0.01      0.00
  17 GRIDFPOS IN CNT4                               12      32.4      32.4      0.01      0.00
  27 RESTART FILES - WRITING                         3    3237.1    3237.1      0.23      0.00
  28 RESTART FILES - READING                         1    4057.7    4057.7      0.10      0.00
  29 SU4FPOS IN CNT4                                12       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                12    5681.5    5681.5      1.60      0.00
  31 POSDDH IN STEPO                                 1       2.5       2.5      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    145    6053.5    6053.5     20.66      0.00
  39 SU0YOMB                                         1     397.1     397.1      0.01      0.00
  51 SCAN2M   - SL COMM. PART 1                    145      30.2      30.2      0.10      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              144     577.4     577.4      1.96      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              144      70.0      70.0      0.24      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                  7       0.3       0.3      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      316     397.1     397.1      2.95      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       300     233.7     233.7      1.65      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        300      77.1      77.1      0.54      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       316     111.2     111.2      0.83      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      38.3      38.3      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            316      63.0      63.0      0.47      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            300      36.1      36.1      0.25      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            316     134.6     134.6      1.00      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            300      61.4      61.4      0.43      0.00
 400 GSTATS                                      17763       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 16473       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME     4249.6 CPU TIME    4195.5 VECTOR TIME     4195.5


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      407.1
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/slots/7/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/7/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMGGh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMSHh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12293967/ICMUAh7zg+001848
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4co_2016092300_20_1018_12293967_0_r611390959_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
20:10:29 (859871): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
14 Jun 2024 20:20:50 1537687 22447584 oifs_43r3_bl_a4co_2016092300_20_1018_12293967_0 1,727,100 55,003 0.0318
14 Jun 2024 14:04:29 1537687 22447584 oifs_43r3_bl_a4co_2016092300_20_1018_12293967_0 1,296,900 39,611 0.0305


©2024 climateprediction.net