climateprediction.net home page
Task 22448520

Task 22448520

Name oifs_43r3_bl_a2lr_2016092300_20_1018_12291702_1
Workunit 12291702
Created 13 Jun 2024, 14:16:41 UTC
Sent 13 Jun 2024, 14:20:30 UTC
Report deadline 12 Aug 2024, 14:20:30 UTC
Received 14 Jun 2024, 6:31:32 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1541092
Run time 14 hours 34 min 58 sec
CPU time 14 hours 29 min 25 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 1.00 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,024.14 MB
Peak swap size 5,981.07 MB
Peak disk usage 1,507.41 MB
Stderr
<core_client_version>8.0.2</core_client_version>
<![CDATA[
<stderr_txt>

  01:42:59 STEP 1405 H= 351:15 +CPU= 28.169
  01:43:25 STEP 1406 H= 351:30 +CPU= 25.804
  01:43:53 STEP 1407 H= 351:45 +CPU= 27.292
  01:44:27 STEP 1408 H= 352:00 +CPU= 33.299
  01:44:59 STEP 1409 H= 352:15 +CPU= 31.314
  01:45:31 STEP 1410 H= 352:30 +CPU= 30.773
  01:46:03 STEP 1411 H= 352:45 +CPU= 30.713
  01:46:36 STEP 1412 H= 353:00 +CPU= 31.819
  01:47:06 STEP 1413 H= 353:15 +CPU= 28.670
  01:47:38 STEP 1414 H= 353:30 +CPU= 30.640
  01:48:10 STEP 1415 H= 353:45 +CPU= 30.574
  01:48:50 STEP 1416 H= 354:00 +CPU= 37.214
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001416
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001416
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001416
  01:49:23 STEP 1417 H= 354:15 +CPU= 31.968
  01:49:57 STEP 1418 H= 354:30 +CPU= 31.869
  01:50:31 STEP 1419 H= 354:45 +CPU= 32.130
  01:51:04 STEP 1420 H= 355:00 +CPU= 31.886
  01:51:35 STEP 1421 H= 355:15 +CPU= 29.570
  01:52:04 STEP 1422 H= 355:30 +CPU= 28.721
  01:52:35 STEP 1423 H= 355:45 +CPU= 29.683
  01:53:07 STEP 1424 H= 356:00 +CPU= 31.004
  01:53:39 STEP 1425 H= 356:15 +CPU= 30.218
  01:54:10 STEP 1426 H= 356:30 +CPU= 29.250
  01:54:40 STEP 1427 H= 356:45 +CPU= 28.541
  01:55:16 STEP 1428 H= 357:00 +CPU= 34.827
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001428
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001428
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001428
  01:55:44 STEP 1429 H= 357:15 +CPU= 27.194
  01:56:12 STEP 1430 H= 357:30 +CPU= 27.897
  01:56:41 STEP 1431 H= 357:45 +CPU= 27.481
  01:57:12 STEP 1432 H= 358:00 +CPU= 29.333
  01:57:41 STEP 1433 H= 358:15 +CPU= 28.108
  01:58:09 STEP 1434 H= 358:30 +CPU= 27.896
  01:58:38 STEP 1435 H= 358:45 +CPU= 27.876
  01:59:07 STEP 1436 H= 359:00 +CPU= 28.175
  01:59:35 STEP 1437 H= 359:15 +CPU= 27.944
  02:00:04 STEP 1438 H= 359:30 +CPU= 27.387
  02:00:33 STEP 1439 H= 359:45 +CPU= 28.899
  02:01:15 STEP 1440 H= 360:00 +CPU= 38.857
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001440
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001440
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001428
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a2lr_2016092300_20_1018_12291702_1_r476214054_14.zip
Uploading the intermediate file: upload_file_14.zip
  02:01:46 STEP 1441 H= 360:15 +CPU= 26.938
  02:02:16 STEP 1442 H= 360:30 +CPU= 26.918
  02:02:46 STEP 1443 H= 360:45 +CPU= 26.309
  02:03:12 STEP 1444 H= 361:00 +CPU= 25.296
  02:03:41 STEP 1445 H= 361:15 +CPU= 25.219
  02:04:11 STEP 1446 H= 361:30 +CPU= 25.497
  02:04:39 STEP 1447 H= 361:45 +CPU= 25.548
  02:05:09 STEP 1448 H= 362:00 +CPU= 26.014
  02:05:38 STEP 1449 H= 362:15 +CPU= 25.800
  02:06:07 STEP 1450 H= 362:30 +CPU= 25.813
  02:06:36 STEP 1451 H= 362:45 +CPU= 25.245
  02:07:11 STEP 1452 H= 363:00 +CPU= 31.529
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001452
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001452
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001452
  02:07:41 STEP 1453 H= 363:15 +CPU= 25.832
  02:08:10 STEP 1454 H= 363:30 +CPU= 25.735
  02:08:39 STEP 1455 H= 363:45 +CPU= 25.571
  02:09:07 STEP 1456 H= 364:00 +CPU= 26.076
  02:09:34 STEP 1457 H= 364:15 +CPU= 25.301
  02:10:03 STEP 1458 H= 364:30 +CPU= 25.787
  02:10:32 STEP 1459 H= 364:45 +CPU= 25.829
  02:11:01 STEP 1460 H= 365:00 +CPU= 25.876
  02:11:29 STEP 1461 H= 365:15 +CPU= 25.014
  02:11:56 STEP 1462 H= 365:30 +CPU= 25.298
  02:12:24 STEP 1463 H= 365:45 +CPU= 25.256
  02:12:59 STEP 1464 H= 366:00 +CPU= 31.406
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001464
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001464
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001464
  02:13:26 STEP 1465 H= 366:15 +CPU= 25.155
  02:13:56 STEP 1466 H= 366:30 +CPU= 25.614
  02:14:25 STEP 1467 H= 366:45 +CPU= 25.672
  02:14:53 STEP 1468 H= 367:00 +CPU= 26.158
  02:15:21 STEP 1469 H= 367:15 +CPU= 25.267
  02:15:50 STEP 1470 H= 367:30 +CPU= 25.554
  02:16:19 STEP 1471 H= 367:45 +CPU= 25.558
  02:16:47 STEP 1472 H= 368:00 +CPU= 25.740
  02:17:16 STEP 1473 H= 368:15 +CPU= 25.557
  02:17:44 STEP 1474 H= 368:30 +CPU= 25.346
  02:18:12 STEP 1475 H= 368:45 +CPU= 25.556
  02:18:48 STEP 1476 H= 369:00 +CPU= 32.004
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001476
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001476
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001476
  02:19:15 STEP 1477 H= 369:15 +CPU= 25.865
  02:19:43 STEP 1478 H= 369:30 +CPU= 26.535
  02:20:10 STEP 1479 H= 369:45 +CPU= 26.217
  02:20:37 STEP 1480 H= 370:00 +CPU= 26.715
  02:21:04 STEP 1481 H= 370:15 +CPU= 26.069
  02:21:31 STEP 1482 H= 370:30 +CPU= 26.429
  02:21:58 STEP 1483 H= 370:45 +CPU= 26.594
  02:22:26 STEP 1484 H= 371:00 +CPU= 27.031
  02:22:53 STEP 1485 H= 371:15 +CPU= 26.621
  02:23:20 STEP 1486 H= 371:30 +CPU= 26.318
  02:23:50 STEP 1487 H= 371:45 +CPU= 26.365
  02:24:30 STEP 1488 H= 372:00 +CPU= 36.389
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001488
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001488
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001488
  02:24:57 STEP 1489 H= 372:15 +CPU= 26.461
  02:25:24 STEP 1490 H= 372:30 +CPU= 26.441
  02:25:51 STEP 1491 H= 372:45 +CPU= 26.538
  02:26:20 STEP 1492 H= 373:00 +CPU= 27.323
  02:26:47 STEP 1493 H= 373:15 +CPU= 26.561
  02:27:14 STEP 1494 H= 373:30 +CPU= 27.053
  02:27:41 STEP 1495 H= 373:45 +CPU= 26.451
  02:28:09 STEP 1496 H= 374:00 +CPU= 27.076
  02:28:36 STEP 1497 H= 374:15 +CPU= 26.365
  02:29:03 STEP 1498 H= 374:30 +CPU= 26.188
  02:29:29 STEP 1499 H= 374:45 +CPU= 26.176
  02:30:03 STEP 1500 H= 375:00 +CPU= 32.431
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001500
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001500
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001500
  02:30:30 STEP 1501 H= 375:15 +CPU= 26.461
  02:30:57 STEP 1502 H= 375:30 +CPU= 26.612
  02:31:24 STEP 1503 H= 375:45 +CPU= 26.443
  02:31:52 STEP 1504 H= 376:00 +CPU= 27.065
  02:32:18 STEP 1505 H= 376:15 +CPU= 26.077
  02:32:45 STEP 1506 H= 376:30 +CPU= 26.502
  02:33:12 STEP 1507 H= 376:45 +CPU= 26.498
  02:33:40 STEP 1508 H= 377:00 +CPU= 26.992
  02:34:07 STEP 1509 H= 377:15 +CPU= 26.216
  02:34:34 STEP 1510 H= 377:30 +CPU= 26.497
  02:35:01 STEP 1511 H= 377:45 +CPU= 26.460
  02:35:35 STEP 1512 H= 378:00 +CPU= 32.257
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001512
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001512
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001512
  02:36:03 STEP 1513 H= 378:15 +CPU= 26.905
  02:36:30 STEP 1514 H= 378:30 +CPU= 26.704
  02:36:57 STEP 1515 H= 378:45 +CPU= 26.640
  02:37:25 STEP 1516 H= 379:00 +CPU= 27.454
  02:37:53 STEP 1517 H= 379:15 +CPU= 26.713
  02:38:21 STEP 1518 H= 379:30 +CPU= 26.923
  02:38:48 STEP 1519 H= 379:45 +CPU= 26.838
  02:39:16 STEP 1520 H= 380:00 +CPU= 27.435
  02:39:43 STEP 1521 H= 380:15 +CPU= 26.498
  02:40:11 STEP 1522 H= 380:30 +CPU= 27.024
  02:40:38 STEP 1523 H= 380:45 +CPU= 26.473
  02:41:13 STEP 1524 H= 381:00 +CPU= 32.951
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001524
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001524
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001524
  02:41:40 STEP 1525 H= 381:15 +CPU= 26.576
  02:42:07 STEP 1526 H= 381:30 +CPU= 26.762
  02:42:34 STEP 1527 H= 381:45 +CPU= 26.382
  02:43:02 STEP 1528 H= 382:00 +CPU= 27.178
  02:43:30 STEP 1529 H= 382:15 +CPU= 26.592
  02:43:57 STEP 1530 H= 382:30 +CPU= 27.075
  02:44:24 STEP 1531 H= 382:45 +CPU= 26.765
  02:44:53 STEP 1532 H= 383:00 +CPU= 27.431
  02:45:20 STEP 1533 H= 383:15 +CPU= 26.693
  02:45:48 STEP 1534 H= 383:30 +CPU= 26.858
  02:46:16 STEP 1535 H= 383:45 +CPU= 26.361
  02:46:54 STEP 1536 H= 384:00 +CPU= 37.033
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001536
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001536
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001536
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001452
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001452
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001452
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001464
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001464
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001464
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001476
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001476
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001476
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001488
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001488
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001488
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001500
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001500
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001500
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001512
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001512
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001512
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001524
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001524
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001524
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a2lr_2016092300_20_1018_12291702_1_r476214054_15.zip
Uploading the intermediate file: upload_file_15.zip
  02:47:21 STEP 1537 H= 384:15 +CPU= 26.116
  02:47:48 STEP 1538 H= 384:30 +CPU= 26.549
  02:48:15 STEP 1539 H= 384:45 +CPU= 26.363
  02:48:44 STEP 1540 H= 385:00 +CPU= 27.060
  02:49:11 STEP 1541 H= 385:15 +CPU= 26.363
  02:49:38 STEP 1542 H= 385:30 +CPU= 26.378
  02:50:06 STEP 1543 H= 385:45 +CPU= 26.615
  02:50:33 STEP 1544 H= 386:00 +CPU= 27.270
  02:51:01 STEP 1545 H= 386:15 +CPU= 26.510
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a2lr
(argv4) batchid: 1018
(argv5) wuid: 12291702
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/15
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/15/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/15/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_cb883442336eebc308391467ff859783 to: /var/lib/boinc/slots/15/oifs_43r3_bl_a2lr_2016092300_20_1018_12291702.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/15/oifs_43r3_bl_a2lr_2016092300_20_1018_12291702.zip
ic_ancil_file: ic_ancil_12291702
ifsdata_file: ifsdata_12291702
climate_data_file: clim_data_12291702
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc/slots/15/ic_ancil_12291702.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/15/ic_ancil_12291702.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_761cf37771e6db63f4a55c707e99a546 to: /var/lib/boinc/slots/15/ifsdata/ifsdata_12291702.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/15/ifsdata/ifsdata_12291702.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/15/159l_2/clim_data_12291702.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/15/159l_2/clim_data_12291702.zip
Checking for progress XML file: /var/lib/boinc/slots/15/progress_file_12291702.xml
Opened progress file ok : /var/lib/boinc/slots/15/progress_file_12291702.xml
-- Model is restarting --
Adjusting last_iter, 1546, to previous model restart step.
Creating progress file: /var/lib/boinc/slots/15/progress_file_12291702.xml
last_cpu_time: 43362
upload_file_number: 16
last_iter: 1537
last_upload: 1382400
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a2lr_2016092300_20_1018_12291702_1_r476214054
The child process has been launched with process id: 182
Executing the command: /var/lib/boinc/slots/15/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.001] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:1c3c15f9e3f2:1:1:182:182] [20240614:025556:1718333756:0.001] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  02:56:25 STEP 1536 H= 384:00 +CPU= 25.992
  02:56:54 STEP 1537 H= 384:15 +CPU= 28.413
  02:57:24 STEP 1538 H= 384:30 +CPU= 29.290
  02:57:54 STEP 1539 H= 384:45 +CPU= 28.504
  02:58:25 STEP 1540 H= 385:00 +CPU= 30.020
  02:58:55 STEP 1541 H= 385:15 +CPU= 29.332
  02:59:25 STEP 1542 H= 385:30 +CPU= 29.462
  02:59:55 STEP 1543 H= 385:45 +CPU= 28.243
  03:00:25 STEP 1544 H= 386:00 +CPU= 28.932
  03:00:54 STEP 1545 H= 386:15 +CPU= 28.692
  03:01:24 STEP 1546 H= 386:30 +CPU= 28.592
  03:01:52 STEP 1547 H= 386:45 +CPU= 27.864
  03:02:28 STEP 1548 H= 387:00 +CPU= 34.775
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001548
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001548
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001548
  03:02:57 STEP 1549 H= 387:15 +CPU= 27.995
  03:03:25 STEP 1550 H= 387:30 +CPU= 27.468
  03:03:55 STEP 1551 H= 387:45 +CPU= 29.294
  03:04:25 STEP 1552 H= 388:00 +CPU= 28.876
  03:04:54 STEP 1553 H= 388:15 +CPU= 29.094
  03:05:25 STEP 1554 H= 388:30 +CPU= 29.868
  03:05:54 STEP 1555 H= 388:45 +CPU= 28.641
  03:06:25 STEP 1556 H= 389:00 +CPU= 29.541
  03:06:53 STEP 1557 H= 389:15 +CPU= 28.021
  03:07:22 STEP 1558 H= 389:30 +CPU= 27.216
  03:07:49 STEP 1559 H= 389:45 +CPU= 27.184
  03:08:23 STEP 1560 H= 390:00 +CPU= 33.148
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001560
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001560
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001560
  03:08:51 STEP 1561 H= 390:15 +CPU= 27.316
  03:09:21 STEP 1562 H= 390:30 +CPU= 28.735
  03:09:49 STEP 1563 H= 390:45 +CPU= 27.426
  03:10:18 STEP 1564 H= 391:00 +CPU= 27.887
  03:10:44 STEP 1565 H= 391:15 +CPU= 25.771
  03:11:12 STEP 1566 H= 391:30 +CPU= 26.963
  03:11:43 STEP 1567 H= 391:45 +CPU= 30.072
  03:12:15 STEP 1568 H= 392:00 +CPU= 31.260
  03:12:47 STEP 1569 H= 392:15 +CPU= 29.506
  03:13:19 STEP 1570 H= 392:30 +CPU= 29.729
  03:13:50 STEP 1571 H= 392:45 +CPU= 29.346
  03:14:27 STEP 1572 H= 393:00 +CPU= 34.479
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001572
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001572
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001572
  03:14:59 STEP 1573 H= 393:15 +CPU= 29.191
  03:15:29 STEP 1574 H= 393:30 +CPU= 28.584
  03:16:01 STEP 1575 H= 393:45 +CPU= 29.945
  03:16:34 STEP 1576 H= 394:00 +CPU= 30.728
  03:17:06 STEP 1577 H= 394:15 +CPU= 30.242
  03:17:37 STEP 1578 H= 394:30 +CPU= 29.181
  03:18:10 STEP 1579 H= 394:45 +CPU= 30.079
  03:18:40 STEP 1580 H= 395:00 +CPU= 28.025
  03:19:08 STEP 1581 H= 395:15 +CPU= 26.555
  03:19:37 STEP 1582 H= 395:30 +CPU= 28.306
  03:20:08 STEP 1583 H= 395:45 +CPU= 29.562
  03:20:48 STEP 1584 H= 396:00 +CPU= 39.440
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001584
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001584
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001584
  03:21:16 STEP 1585 H= 396:15 +CPU= 27.191
  03:21:44 STEP 1586 H= 396:30 +CPU= 27.155
  03:22:11 STEP 1587 H= 396:45 +CPU= 26.291
  03:22:39 STEP 1588 H= 397:00 +CPU= 26.681
  03:23:07 STEP 1589 H= 397:15 +CPU= 26.355
  03:23:34 STEP 1590 H= 397:30 +CPU= 25.949
  03:24:02 STEP 1591 H= 397:45 +CPU= 26.725
  03:24:31 STEP 1592 H= 398:00 +CPU= 26.919
  03:24:58 STEP 1593 H= 398:15 +CPU= 26.266
  03:25:27 STEP 1594 H= 398:30 +CPU= 27.164
  03:25:55 STEP 1595 H= 398:45 +CPU= 26.003
  03:26:29 STEP 1596 H= 399:00 +CPU= 32.016
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001596
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001596
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001596
  03:26:57 STEP 1597 H= 399:15 +CPU= 26.419
  03:27:26 STEP 1598 H= 399:30 +CPU= 27.479
  03:27:55 STEP 1599 H= 399:45 +CPU= 26.961
  03:28:24 STEP 1600 H= 400:00 +CPU= 26.954
  03:28:52 STEP 1601 H= 400:15 +CPU= 26.135
  03:29:20 STEP 1602 H= 400:30 +CPU= 26.860
  03:29:48 STEP 1603 H= 400:45 +CPU= 25.572
  03:30:16 STEP 1604 H= 401:00 +CPU= 25.980
  03:30:43 STEP 1605 H= 401:15 +CPU= 25.229
  03:31:10 STEP 1606 H= 401:30 +CPU= 24.775
  03:31:37 STEP 1607 H= 401:45 +CPU= 24.804
  03:32:10 STEP 1608 H= 402:00 +CPU= 31.279
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001608
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001608
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001608
  03:32:36 STEP 1609 H= 402:15 +CPU= 24.843
  03:33:03 STEP 1610 H= 402:30 +CPU= 25.043
  03:33:30 STEP 1611 H= 402:45 +CPU= 25.010
  03:33:57 STEP 1612 H= 403:00 +CPU= 25.474
  03:34:24 STEP 1613 H= 403:15 +CPU= 24.823
  03:34:51 STEP 1614 H= 403:30 +CPU= 25.122
  03:35:18 STEP 1615 H= 403:45 +CPU= 25.259
  03:35:46 STEP 1616 H= 404:00 +CPU= 25.720
  03:36:13 STEP 1617 H= 404:15 +CPU= 24.945
  03:36:40 STEP 1618 H= 404:30 +CPU= 25.036
  03:37:07 STEP 1619 H= 404:45 +CPU= 24.827
  03:37:40 STEP 1620 H= 405:00 +CPU= 31.271
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001620
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001620
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001620
  03:38:07 STEP 1621 H= 405:15 +CPU= 25.144
  03:38:34 STEP 1622 H= 405:30 +CPU= 25.088
  03:39:00 STEP 1623 H= 405:45 +CPU= 24.851
  03:39:28 STEP 1624 H= 406:00 +CPU= 25.380
  03:39:54 STEP 1625 H= 406:15 +CPU= 24.799
  03:40:22 STEP 1626 H= 406:30 +CPU= 25.007
  03:40:49 STEP 1627 H= 406:45 +CPU= 25.041
  03:41:17 STEP 1628 H= 407:00 +CPU= 25.736
  03:41:45 STEP 1629 H= 407:15 +CPU= 25.034
  03:42:12 STEP 1630 H= 407:30 +CPU= 25.080
  03:42:40 STEP 1631 H= 407:45 +CPU= 25.097
  03:43:18 STEP 1632 H= 408:00 +CPU= 34.830
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001632
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001632
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001536
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001536
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001536
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001548
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001548
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001548
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001560
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001560
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001560
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001572
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001572
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001572
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001584
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001584
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001584
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001596
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001596
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001596
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001608
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001608
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001608
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001620
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001620
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001620
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a2lr_2016092300_20_1018_12291702_1_r476214054_16.zip
Uploading the intermediate file: upload_file_16.zip
  03:43:45 STEP 1633 H= 408:15 +CPU= 25.291
  03:44:12 STEP 1634 H= 408:30 +CPU= 24.903
  03:44:39 STEP 1635 H= 408:45 +CPU= 24.961
  03:45:07 STEP 1636 H= 409:00 +CPU= 25.594
  03:45:33 STEP 1637 H= 409:15 +CPU= 24.691
  03:46:00 STEP 1638 H= 409:30 +CPU= 25.311
  03:46:26 STEP 1639 H= 409:45 +CPU= 25.163
  03:46:54 STEP 1640 H= 410:00 +CPU= 25.776
  03:47:21 STEP 1641 H= 410:15 +CPU= 25.399
  03:47:48 STEP 1642 H= 410:30 +CPU= 25.010
  03:48:15 STEP 1643 H= 410:45 +CPU= 25.044
  03:48:48 STEP 1644 H= 411:00 +CPU= 30.829
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001644
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001644
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001644
  03:49:15 STEP 1645 H= 411:15 +CPU= 25.326
  03:49:42 STEP 1646 H= 411:30 +CPU= 25.201
  03:50:10 STEP 1647 H= 411:45 +CPU= 25.331
  03:50:38 STEP 1648 H= 412:00 +CPU= 25.996
  03:51:06 STEP 1649 H= 412:15 +CPU= 25.119
  03:51:33 STEP 1650 H= 412:30 +CPU= 25.388
  03:52:00 STEP 1651 H= 412:45 +CPU= 25.427
  03:52:28 STEP 1652 H= 413:00 +CPU= 25.772
  03:52:56 STEP 1653 H= 413:15 +CPU= 25.099
  03:53:23 STEP 1654 H= 413:30 +CPU= 25.073
  03:53:50 STEP 1655 H= 413:45 +CPU= 25.256
  03:54:23 STEP 1656 H= 414:00 +CPU= 31.340
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001656
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001656
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001656
  03:54:50 STEP 1657 H= 414:15 +CPU= 24.968
  03:55:17 STEP 1658 H= 414:30 +CPU= 25.027
  03:55:45 STEP 1659 H= 414:45 +CPU= 24.961
  03:56:13 STEP 1660 H= 415:00 +CPU= 25.672
  03:56:36 STEP 1661 H= 415:15 +CPU= 21.993
  03:56:58 STEP 1662 H= 415:30 +CPU= 20.123
  03:57:20 STEP 1663 H= 415:45 +CPU= 20.094
  03:57:42 STEP 1664 H= 416:00 +CPU= 20.608
  03:58:03 STEP 1665 H= 416:15 +CPU= 20.287
  03:58:25 STEP 1666 H= 416:30 +CPU= 20.136
  03:58:47 STEP 1667 H= 416:45 +CPU= 20.219
  03:59:14 STEP 1668 H= 417:00 +CPU= 25.282
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001668
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001668
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001668
  03:59:35 STEP 1669 H= 417:15 +CPU= 20.130
  03:59:56 STEP 1670 H= 417:30 +CPU= 19.648
  04:00:17 STEP 1671 H= 417:45 +CPU= 20.146
  04:00:39 STEP 1672 H= 418:00 +CPU= 20.631
  04:01:01 STEP 1673 H= 418:15 +CPU= 19.961
  04:01:22 STEP 1674 H= 418:30 +CPU= 20.107
  04:01:44 STEP 1675 H= 418:45 +CPU= 20.305
  04:02:06 STEP 1676 H= 419:00 +CPU= 20.539
  04:02:27 STEP 1677 H= 419:15 +CPU= 19.923
  04:02:49 STEP 1678 H= 419:30 +CPU= 20.165
  04:03:10 STEP 1679 H= 419:45 +CPU= 20.102
  04:03:41 STEP 1680 H= 420:00 +CPU= 28.858
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001680
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001680
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001680
  04:04:03 STEP 1681 H= 420:15 +CPU= 20.191
  04:04:24 STEP 1682 H= 420:30 +CPU= 20.060
  04:04:46 STEP 1683 H= 420:45 +CPU= 20.223
  04:05:09 STEP 1684 H= 421:00 +CPU= 21.133
  04:05:30 STEP 1685 H= 421:15 +CPU= 20.182
  04:05:51 STEP 1686 H= 421:30 +CPU= 19.732
  04:06:13 STEP 1687 H= 421:45 +CPU= 20.307
  04:06:35 STEP 1688 H= 422:00 +CPU= 20.665
  04:06:56 STEP 1689 H= 422:15 +CPU= 20.112
  04:07:18 STEP 1690 H= 422:30 +CPU= 20.168
  04:07:39 STEP 1691 H= 422:45 +CPU= 20.015
  04:08:06 STEP 1692 H= 423:00 +CPU= 25.509
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001692
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001692
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001692
  04:08:27 STEP 1693 H= 423:15 +CPU= 20.283
  04:08:49 STEP 1694 H= 423:30 +CPU= 20.116
  04:09:11 STEP 1695 H= 423:45 +CPU= 20.392
  04:09:34 STEP 1696 H= 424:00 +CPU= 20.847
  04:09:56 STEP 1697 H= 424:15 +CPU= 19.996
  04:10:17 STEP 1698 H= 424:30 +CPU= 20.163
  04:10:40 STEP 1699 H= 424:45 +CPU= 19.891
  04:11:03 STEP 1700 H= 425:00 +CPU= 20.889
  04:11:24 STEP 1701 H= 425:15 +CPU= 20.020
  04:11:46 STEP 1702 H= 425:30 +CPU= 20.193
  04:12:07 STEP 1703 H= 425:45 +CPU= 20.030
  04:12:34 STEP 1704 H= 426:00 +CPU= 25.533
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001704
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001704
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001704
  04:12:56 STEP 1705 H= 426:15 +CPU= 20.075
  04:13:17 STEP 1706 H= 426:30 +CPU= 19.918
  04:13:39 STEP 1707 H= 426:45 +CPU= 20.188
  04:14:01 STEP 1708 H= 427:00 +CPU= 20.684
  04:14:22 STEP 1709 H= 427:15 +CPU= 19.678
  04:14:43 STEP 1710 H= 427:30 +CPU= 20.170
  04:15:04 STEP 1711 H= 427:45 +CPU= 20.084
  04:15:27 STEP 1712 H= 428:00 +CPU= 20.668
  04:15:48 STEP 1713 H= 428:15 +CPU= 20.244
  04:16:09 STEP 1714 H= 428:30 +CPU= 19.970
  04:16:30 STEP 1715 H= 428:45 +CPU= 19.980
  04:16:57 STEP 1716 H= 429:00 +CPU= 25.200
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001716
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001716
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001716
  04:17:18 STEP 1717 H= 429:15 +CPU= 20.084
  04:17:40 STEP 1718 H= 429:30 +CPU= 20.218
  04:18:01 STEP 1719 H= 429:45 +CPU= 20.258
  04:18:23 STEP 1720 H= 430:00 +CPU= 20.391
  04:18:44 STEP 1721 H= 430:15 +CPU= 20.555
  04:19:05 STEP 1722 H= 430:30 +CPU= 20.168
  04:19:27 STEP 1723 H= 430:45 +CPU= 20.155
  04:19:48 STEP 1724 H= 431:00 +CPU= 20.779
  04:20:10 STEP 1725 H= 431:15 +CPU= 20.405
  04:20:32 STEP 1726 H= 431:30 +CPU= 20.397
  04:20:54 STEP 1727 H= 431:45 +CPU= 19.936
  04:21:26 STEP 1728 H= 432:00 +CPU= 28.860
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001728
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001728
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001632
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001632
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001644
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001644
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001644
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001656
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001656
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001656
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001668
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001668
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001668
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001680
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001680
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001680
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001692
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001692
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001692
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001704
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001704
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001704
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001716
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001716
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001716
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a2lr_2016092300_20_1018_12291702_1_r476214054_17.zip
Uploading the intermediate file: upload_file_17.zip
  04:21:49 STEP 1729 H= 432:15 +CPU= 20.275
  04:22:11 STEP 1730 H= 432:30 +CPU= 20.700
  04:22:33 STEP 1731 H= 432:45 +CPU= 19.976
  04:22:55 STEP 1732 H= 433:00 +CPU= 20.517
  04:23:16 STEP 1733 H= 433:15 +CPU= 20.207
  04:23:38 STEP 1734 H= 433:30 +CPU= 20.223
  04:23:59 STEP 1735 H= 433:45 +CPU= 20.282
  04:24:21 STEP 1736 H= 434:00 +CPU= 20.945
  04:24:43 STEP 1737 H= 434:15 +CPU= 20.302
  04:25:05 STEP 1738 H= 434:30 +CPU= 20.095
  04:25:26 STEP 1739 H= 434:45 +CPU= 20.133
  04:25:54 STEP 1740 H= 435:00 +CPU= 25.688
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001740
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001740
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001740
  04:26:17 STEP 1741 H= 435:15 +CPU= 20.317
  04:26:38 STEP 1742 H= 435:30 +CPU= 20.029
  04:27:00 STEP 1743 H= 435:45 +CPU= 20.330
  04:27:22 STEP 1744 H= 436:00 +CPU= 20.529
  04:27:43 STEP 1745 H= 436:15 +CPU= 19.943
  04:28:05 STEP 1746 H= 436:30 +CPU= 19.957
  04:28:26 STEP 1747 H= 436:45 +CPU= 20.197
  04:28:49 STEP 1748 H= 437:00 +CPU= 20.821
  04:29:10 STEP 1749 H= 437:15 +CPU= 20.168
  04:29:32 STEP 1750 H= 437:30 +CPU= 20.155
  04:29:53 STEP 1751 H= 437:45 +CPU= 19.638
  04:30:20 STEP 1752 H= 438:00 +CPU= 25.301
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001752
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001752
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001752
  04:30:42 STEP 1753 H= 438:15 +CPU= 20.126
  04:31:04 STEP 1754 H= 438:30 +CPU= 20.337
  04:31:25 STEP 1755 H= 438:45 +CPU= 20.498
  04:31:48 STEP 1756 H= 439:00 +CPU= 20.811
  04:32:10 STEP 1757 H= 439:15 +CPU= 19.990
  04:32:31 STEP 1758 H= 439:30 +CPU= 19.949
  04:32:52 STEP 1759 H= 439:45 +CPU= 20.253
  04:33:14 STEP 1760 H= 440:00 +CPU= 20.716
  04:33:36 STEP 1761 H= 440:15 +CPU= 20.222
  04:33:57 STEP 1762 H= 440:30 +CPU= 19.793
  04:34:18 STEP 1763 H= 440:45 +CPU= 20.094
  04:34:45 STEP 1764 H= 441:00 +CPU= 25.198
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001764
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001764
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001764
  04:35:07 STEP 1765 H= 441:15 +CPU= 20.320
  04:35:29 STEP 1766 H= 441:30 +CPU= 20.062
  04:35:51 STEP 1767 H= 441:45 +CPU= 20.082
  04:36:13 STEP 1768 H= 442:00 +CPU= 20.578
  04:36:35 STEP 1769 H= 442:15 +CPU= 20.127
  04:36:57 STEP 1770 H= 442:30 +CPU= 20.263
  04:37:18 STEP 1771 H= 442:45 +CPU= 19.668
  04:37:40 STEP 1772 H= 443:00 +CPU= 20.489
  04:38:01 STEP 1773 H= 443:15 +CPU= 20.405
  04:38:22 STEP 1774 H= 443:30 +CPU= 19.843
  04:38:44 STEP 1775 H= 443:45 +CPU= 20.297
  04:39:14 STEP 1776 H= 444:00 +CPU= 28.796
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001776
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001776
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001776
  04:39:35 STEP 1777 H= 444:15 +CPU= 19.985
  04:39:57 STEP 1778 H= 444:30 +CPU= 20.054
  04:40:18 STEP 1779 H= 444:45 +CPU= 20.438
  04:40:40 STEP 1780 H= 445:00 +CPU= 20.964
  04:41:02 STEP 1781 H= 445:15 +CPU= 20.280
  04:41:23 STEP 1782 H= 445:30 +CPU= 19.985
  04:41:45 STEP 1783 H= 445:45 +CPU= 20.061
  04:42:07 STEP 1784 H= 446:00 +CPU= 20.861
  04:42:29 STEP 1785 H= 446:15 +CPU= 20.222
  04:42:51 STEP 1786 H= 446:30 +CPU= 20.207
  04:43:12 STEP 1787 H= 446:45 +CPU= 20.315
  04:43:40 STEP 1788 H= 447:00 +CPU= 25.645
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001788
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001788
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001788
  04:44:02 STEP 1789 H= 447:15 +CPU= 20.254
  04:44:24 STEP 1790 H= 447:30 +CPU= 20.195
  04:44:46 STEP 1791 H= 447:45 +CPU= 20.488
  04:45:09 STEP 1792 H= 448:00 +CPU= 20.843
  04:45:30 STEP 1793 H= 448:15 +CPU= 20.110
  04:45:53 STEP 1794 H= 448:30 +CPU= 20.320
  04:46:14 STEP 1795 H= 448:45 +CPU= 20.358
  04:46:37 STEP 1796 H= 449:00 +CPU= 20.520
  04:46:58 STEP 1797 H= 449:15 +CPU= 20.147
  04:47:20 STEP 1798 H= 449:30 +CPU= 20.162
  04:47:41 STEP 1799 H= 449:45 +CPU= 19.907
  04:48:08 STEP 1800 H= 450:00 +CPU= 25.590
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001800
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001800
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001800
  04:48:30 STEP 1801 H= 450:15 +CPU= 20.455
  04:48:52 STEP 1802 H= 450:30 +CPU= 20.502
  04:49:14 STEP 1803 H= 450:45 +CPU= 20.473
  04:49:37 STEP 1804 H= 451:00 +CPU= 20.838
  04:49:59 STEP 1805 H= 451:15 +CPU= 20.076
  04:50:20 STEP 1806 H= 451:30 +CPU= 20.099
  04:50:41 STEP 1807 H= 451:45 +CPU= 20.170
  04:51:03 STEP 1808 H= 452:00 +CPU= 20.289
  04:51:24 STEP 1809 H= 452:15 +CPU= 20.299
  04:51:46 STEP 1810 H= 452:30 +CPU= 20.286
  04:52:08 STEP 1811 H= 452:45 +CPU= 20.555
  04:52:35 STEP 1812 H= 453:00 +CPU= 25.205
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001812
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001812
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001812
  04:52:56 STEP 1813 H= 453:15 +CPU= 20.118
  04:53:18 STEP 1814 H= 453:30 +CPU= 20.155
  04:53:40 STEP 1815 H= 453:45 +CPU= 20.087
  04:54:02 STEP 1816 H= 454:00 +CPU= 20.512
  04:54:23 STEP 1817 H= 454:15 +CPU= 20.102
  04:54:45 STEP 1818 H= 454:30 +CPU= 20.114
  04:55:06 STEP 1819 H= 454:45 +CPU= 20.073
  04:55:28 STEP 1820 H= 455:00 +CPU= 20.671
  04:55:50 STEP 1821 H= 455:15 +CPU= 20.204
  04:56:12 STEP 1822 H= 455:30 +CPU= 20.474
  04:56:33 STEP 1823 H= 455:45 +CPU= 20.166
  04:57:03 STEP 1824 H= 456:00 +CPU= 28.822
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001824
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001824
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001728
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001728
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001740
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001740
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001740
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001752
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001752
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001752
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001764
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001764
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001764
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001776
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001776
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001776
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001788
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001788
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001788
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001800
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001800
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001800
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001812
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001812
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001812
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a2lr_2016092300_20_1018_12291702_1_r476214054_18.zip
Uploading the intermediate file: upload_file_18.zip
  04:57:26 STEP 1825 H= 456:15 +CPU= 19.998
  04:57:47 STEP 1826 H= 456:30 +CPU= 20.282
  04:58:09 STEP 1827 H= 456:45 +CPU= 20.377
  04:58:31 STEP 1828 H= 457:00 +CPU= 20.403
  04:58:53 STEP 1829 H= 457:15 +CPU= 20.094
  04:59:15 STEP 1830 H= 457:30 +CPU= 19.923
  04:59:36 STEP 1831 H= 457:45 +CPU= 19.952
  04:59:59 STEP 1832 H= 458:00 +CPU= 20.607
  05:00:22 STEP 1833 H= 458:15 +CPU= 20.353
  05:00:43 STEP 1834 H= 458:30 +CPU= 20.097
  05:01:05 STEP 1835 H= 458:45 +CPU= 20.212
  05:01:33 STEP 1836 H= 459:00 +CPU= 25.825
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001836
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001836
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001836
  05:01:54 STEP 1837 H= 459:15 +CPU= 20.105
  05:02:16 STEP 1838 H= 459:30 +CPU= 20.349
  05:02:38 STEP 1839 H= 459:45 +CPU= 20.307
  05:03:00 STEP 1840 H= 460:00 +CPU= 20.946
  05:03:22 STEP 1841 H= 460:15 +CPU= 20.021
  05:03:44 STEP 1842 H= 460:30 +CPU= 20.447
  05:04:06 STEP 1843 H= 460:45 +CPU= 20.515
  05:04:28 STEP 1844 H= 461:00 +CPU= 20.915
  05:04:51 STEP 1845 H= 461:15 +CPU= 20.680
  05:05:12 STEP 1846 H= 461:30 +CPU= 20.653
  05:05:35 STEP 1847 H= 461:45 +CPU= 20.646
  05:06:02 STEP 1848 H= 462:00 +CPU= 25.385
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001848
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001848
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001848
  05:06:24 STEP 1849 H= 462:15 +CPU= 20.134
  05:06:45 STEP 1850 H= 462:30 +CPU= 20.456
  05:07:07 STEP 1851 H= 462:45 +CPU= 20.290
  05:07:30 STEP 1852 H= 463:00 +CPU= 20.606
  05:07:50 STEP 1853 H= 463:15 +CPU= 20.294
  05:08:12 STEP 1854 H= 463:30 +CPU= 20.098
  05:08:33 STEP 1855 H= 463:45 +CPU= 19.927
  05:08:55 STEP 1856 H= 464:00 +CPU= 20.684
  05:09:17 STEP 1857 H= 464:15 +CPU= 20.027
  05:09:38 STEP 1858 H= 464:30 +CPU= 20.508
  05:09:59 STEP 1859 H= 464:45 +CPU= 19.958
  05:10:27 STEP 1860 H= 465:00 +CPU= 25.540
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001860
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001860
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001860
  05:10:49 STEP 1861 H= 465:15 +CPU= 20.250
  05:11:11 STEP 1862 H= 465:30 +CPU= 20.205
  05:11:32 STEP 1863 H= 465:45 +CPU= 19.973
  05:11:54 STEP 1864 H= 466:00 +CPU= 20.838
  05:12:16 STEP 1865 H= 466:15 +CPU= 20.208
  05:12:37 STEP 1866 H= 466:30 +CPU= 20.053
  05:12:58 STEP 1867 H= 466:45 +CPU= 20.388
  05:13:20 STEP 1868 H= 467:00 +CPU= 20.337
  05:13:42 STEP 1869 H= 467:15 +CPU= 20.228
  05:14:03 STEP 1870 H= 467:30 +CPU= 19.944
  05:14:25 STEP 1871 H= 467:45 +CPU= 20.286
  05:14:55 STEP 1872 H= 468:00 +CPU= 28.795
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001872
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001872
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001872
  05:15:16 STEP 1873 H= 468:15 +CPU= 20.084
  05:15:37 STEP 1874 H= 468:30 +CPU= 20.122
  05:15:59 STEP 1875 H= 468:45 +CPU= 20.390
  05:16:21 STEP 1876 H= 469:00 +CPU= 20.878
  05:16:42 STEP 1877 H= 469:15 +CPU= 20.186
  05:17:03 STEP 1878 H= 469:30 +CPU= 20.463
  05:17:25 STEP 1879 H= 469:45 +CPU= 20.169
  05:17:47 STEP 1880 H= 470:00 +CPU= 20.343
  05:18:09 STEP 1881 H= 470:15 +CPU= 20.416
  05:18:32 STEP 1882 H= 470:30 +CPU= 20.164
  05:18:53 STEP 1883 H= 470:45 +CPU= 20.309
  05:19:22 STEP 1884 H= 471:00 +CPU= 25.505
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001884
  05:19:43 STEP 1885 H= 471:15 +CPU= 20.564
  05:20:05 STEP 1886 H= 471:30 +CPU= 20.269
  05:20:27 STEP 1887 H= 471:45 +CPU= 20.391
  05:20:49 STEP 1888 H= 472:00 +CPU= 20.623
  05:21:11 STEP 1889 H= 472:15 +CPU= 20.243
  05:21:33 STEP 1890 H= 472:30 +CPU= 20.177
  05:21:55 STEP 1891 H= 472:45 +CPU= 20.481
  05:22:17 STEP 1892 H= 473:00 +CPU= 20.723
  05:22:39 STEP 1893 H= 473:15 +CPU= 20.201
  05:23:01 STEP 1894 H= 473:30 +CPU= 20.399
  05:23:23 STEP 1895 H= 473:45 +CPU= 20.099
  05:23:50 STEP 1896 H= 474:00 +CPU= 25.507
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001896
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001896
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001896
  05:24:11 STEP 1897 H= 474:15 +CPU= 20.356
  05:24:33 STEP 1898 H= 474:30 +CPU= 20.330
  05:24:55 STEP 1899 H= 474:45 +CPU= 20.140
  05:25:17 STEP 1900 H= 475:00 +CPU= 20.343
  05:25:39 STEP 1901 H= 475:15 +CPU= 20.384
  05:26:00 STEP 1902 H= 475:30 +CPU= 19.725
  05:26:22 STEP 1903 H= 475:45 +CPU= 20.120
  05:26:44 STEP 1904 H= 476:00 +CPU= 20.716
  05:27:05 STEP 1905 H= 476:15 +CPU= 20.198
  05:27:26 STEP 1906 H= 476:30 +CPU= 20.149
  05:27:47 STEP 1907 H= 476:45 +CPU= 20.185
  05:28:15 STEP 1908 H= 477:00 +CPU= 25.657
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001908
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001908
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001908
  05:28:36 STEP 1909 H= 477:15 +CPU= 20.317
  05:28:57 STEP 1910 H= 477:30 +CPU= 20.098
  05:29:17 STEP 1911 H= 477:45 +CPU= 19.695
  05:29:36 STEP 1912 H= 478:00 +CPU= 19.414
  05:29:56 STEP 1913 H= 478:15 +CPU= 19.165
  05:30:16 STEP 1914 H= 478:30 +CPU= 19.634
  05:30:34 STEP 1915 H= 478:45 +CPU= 17.808
  05:30:52 STEP 1916 H= 479:00 +CPU= 18.015
  05:31:10 STEP 1917 H= 479:15 +CPU= 17.940
  05:31:28 STEP 1918 H= 479:30 +CPU= 18.060
  05:31:46 STEP 1919 H= 479:45 +CPU= 17.696
  05:32:12 STEP 1920 H= 480:00 +CPU= 24.147
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  480.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.1916940740E-21      I       0.1926623439E-04      I
ICloud fractionI      0.0000000000E+00      I       0.6133192320E-01      I
I  Relat. Hum. I      0.1138589868E+00      I       0.3506985839E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.1136902685E+04      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    301397860 BYTES IN      7 RECORDS TRANSFERRED IN    0.0007 SECONDS430568.4 Mbytes/s, TOTAL TIME=    0.0078( 9.0%)
 05:32:12 STEP 1920 H= 480:00 +CPU= 24.147
 END CNT3
 NSTEP =  1920 CNT0   000000000  16.965  16.965  18.383   0.923               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN        11440/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    203    0       46      1044068    11440 ************    22.8     1.1    21.7     1.7    39.7
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 9376065.1 9376065.1    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 9373070.4 9373070.4     99.97      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  417    4125.4    4125.4     18.35      0.00
   9 SPCM     - SPECTRAL COMP.                     384     635.7     635.7      2.60      0.00
  10 SCAN2M - PHYSICS                              385   11352.8   11352.8     46.62      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 32     913.4     913.4      0.31      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 17      45.3      45.3      0.01      0.00
  17 GRIDFPOS IN CNT4                               32      23.9      23.9      0.01      0.00
  27 RESTART FILES - WRITING                         8    3653.7    3653.7      0.31      0.00
  28 RESTART FILES - READING                         1    2378.9    2378.9      0.03      0.00
  29 SU4FPOS IN CNT4                                32       0.0       0.0      0.00      0.00
  30 DYNFPOS IN CNT4                                32    5039.4    5039.4      1.72      0.00
  31 POSDDH IN STEPO                                 2     685.5     685.5      0.01      0.00
  37 CPGLAG   - SL COMPUTATIONS                    385    5103.1    5103.1     20.95      0.00
  39 SU0YOMB                                         1     364.8     364.8      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    385      21.5      21.5      0.09      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              384     257.7     257.7      1.06      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              384      51.7      51.7      0.21      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 17       0.2       0.2      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      836     335.9     335.9      3.00      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       800     207.8     207.8      1.77      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        800      82.5      82.5      0.70      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       836     109.1     109.1      0.97      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      27.9      27.9      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            836      58.7      58.7      0.52      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            800      30.2      30.2      0.26      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            836      66.9      66.9      0.60      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            800      38.2      38.2      0.33      0.00
 400 GSTATS                                      45508       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 42288       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME     9376.1 CPU TIME    8803.7 VECTOR TIME     8803.7


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      184.4
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc/slots/15/ICMGGh7zg+001920
Moving to projects directory: /var/lib/boinc/slots/15/ICMSHh7zg+001920
Moving to projects directory: /var/lib/boinc/slots/15/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc/slots/15/NODE.001_01
Adding to the zip: /var/lib/boinc/slots/15/ifs.stat
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001884
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001860
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001836
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001836
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001908
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001836
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001860
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001884
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001908
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMUAh7zg+001908
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001884
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMSHh7zg+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12291702/ICMGGh7zg+001860
Zipping up the final file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a2lr_2016092300_20_1018_12291702_1_r476214054_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
05:35:43 (163): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
14 Jun 2024 06:31:41 1541092 22448520 oifs_43r3_bl_a2lr_2016092300_20_1018_12291702_1 1,727,100 52,165 0.0302
14 Jun 2024 02:12:29 1541092 22448520 oifs_43r3_bl_a2lr_2016092300_20_1018_12291702_1 1,209,600 37,735 0.0312


©2024 climateprediction.net