climateprediction.net home page
Task 22448408

Task 22448408

Name oifs_43r3_bl_a4zh_2016092300_20_1018_12294788_0
Workunit 12294788
Created 13 Jun 2024, 5:21:38 UTC
Sent 13 Jun 2024, 23:41:45 UTC
Report deadline 12 Aug 2024, 23:41:45 UTC
Received 14 Jun 2024, 20:50:24 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1507812
Run time 19 hours 41 min 45 sec
CPU time 17 hours 14 min 53 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 5.39 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,516.45 MB
Peak swap size 5,981.07 MB
Peak disk usage 1,962.55 MB
Stderr
<core_client_version>7.24.1</core_client_version>
<![CDATA[
<stderr_txt>
ojects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001380
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001380
  15:32:01 STEP 1381 H= 345:15 +CPU= 36.464
  15:32:42 STEP 1382 H= 345:30 +CPU= 36.318
  15:33:20 STEP 1383 H= 345:45 +CPU= 35.181
  15:34:02 STEP 1384 H= 346:00 +CPU= 37.792
  15:34:43 STEP 1385 H= 346:15 +CPU= 37.137
  15:35:24 STEP 1386 H= 346:30 +CPU= 36.612
  15:36:05 STEP 1387 H= 346:45 +CPU= 35.984
  15:36:45 STEP 1388 H= 347:00 +CPU= 35.466
  15:37:25 STEP 1389 H= 347:15 +CPU= 36.184
  15:38:04 STEP 1390 H= 347:30 +CPU= 35.375
  15:38:44 STEP 1391 H= 347:45 +CPU= 35.631
  15:39:48 STEP 1392 H= 348:00 +CPU= 49.083
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001392
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001392
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001392
  15:40:28 STEP 1393 H= 348:15 +CPU= 34.836
  15:41:08 STEP 1394 H= 348:30 +CPU= 35.067
  15:41:48 STEP 1395 H= 348:45 +CPU= 35.382
  15:42:29 STEP 1396 H= 349:00 +CPU= 36.165
  15:43:08 STEP 1397 H= 349:15 +CPU= 35.346
  15:43:48 STEP 1398 H= 349:30 +CPU= 35.996
  15:44:28 STEP 1399 H= 349:45 +CPU= 35.116
  15:45:06 STEP 1400 H= 350:00 +CPU= 35.223
  15:45:45 STEP 1401 H= 350:15 +CPU= 34.704
  15:46:25 STEP 1402 H= 350:30 +CPU= 36.340
  15:47:06 STEP 1403 H= 350:45 +CPU= 37.090
  15:47:55 STEP 1404 H= 351:00 +CPU= 43.964
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001404
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001404
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001404
  15:48:35 STEP 1405 H= 351:15 +CPU= 35.156
  15:49:16 STEP 1406 H= 351:30 +CPU= 36.501
  15:49:54 STEP 1407 H= 351:45 +CPU= 34.466
  15:50:34 STEP 1408 H= 352:00 +CPU= 35.270
  15:51:17 STEP 1409 H= 352:15 +CPU= 35.599
  15:51:59 STEP 1410 H= 352:30 +CPU= 36.278
  15:52:40 STEP 1411 H= 352:45 +CPU= 37.427
  15:53:20 STEP 1412 H= 353:00 +CPU= 36.335
  15:53:59 STEP 1413 H= 353:15 +CPU= 34.945
  15:54:39 STEP 1414 H= 353:30 +CPU= 36.184
  15:55:18 STEP 1415 H= 353:45 +CPU= 35.213
  15:56:09 STEP 1416 H= 354:00 +CPU= 45.695
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001416
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001416
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001416
  15:56:47 STEP 1417 H= 354:15 +CPU= 34.510
  15:57:26 STEP 1418 H= 354:30 +CPU= 35.130
  15:58:06 STEP 1419 H= 354:45 +CPU= 35.865
  15:58:47 STEP 1420 H= 355:00 +CPU= 37.041
  15:59:28 STEP 1421 H= 355:15 +CPU= 36.836
  16:00:09 STEP 1422 H= 355:30 +CPU= 36.675
  16:00:49 STEP 1423 H= 355:45 +CPU= 35.561
  16:01:29 STEP 1424 H= 356:00 +CPU= 35.765
  16:02:08 STEP 1425 H= 356:15 +CPU= 34.695
  16:02:50 STEP 1426 H= 356:30 +CPU= 36.854
  16:03:31 STEP 1427 H= 356:45 +CPU= 36.140
  16:04:21 STEP 1428 H= 357:00 +CPU= 44.490
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001428
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001428
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001428
  16:05:01 STEP 1429 H= 357:15 +CPU= 34.718
  16:05:41 STEP 1430 H= 357:30 +CPU= 33.971
  16:06:21 STEP 1431 H= 357:45 +CPU= 35.691
  16:07:03 STEP 1432 H= 358:00 +CPU= 36.474
  16:07:43 STEP 1433 H= 358:15 +CPU= 35.864
  16:08:24 STEP 1434 H= 358:30 +CPU= 36.459
  16:09:02 STEP 1435 H= 358:45 +CPU= 33.811
  16:09:44 STEP 1436 H= 359:00 +CPU= 36.978
  16:10:25 STEP 1437 H= 359:15 +CPU= 35.275
  16:11:07 STEP 1438 H= 359:30 +CPU= 37.538
  16:11:47 STEP 1439 H= 359:45 +CPU= 36.924
  16:12:49 STEP 1440 H= 360:00 +CPU= 47.912
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001440
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001440
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001428
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a4zh_2016092300_20_1018_12294788_0_r1368559108_14.zip
Uploading the intermediate file: upload_file_14.zip
  16:13:30 STEP 1441 H= 360:15 +CPU= 35.679
  16:14:12 STEP 1442 H= 360:30 +CPU= 36.692
  16:14:53 STEP 1443 H= 360:45 +CPU= 36.649
  16:15:34 STEP 1444 H= 361:00 +CPU= 37.019
  16:16:13 STEP 1445 H= 361:15 +CPU= 35.077
  16:16:53 STEP 1446 H= 361:30 +CPU= 36.949
  16:17:32 STEP 1447 H= 361:45 +CPU= 35.452
  16:18:12 STEP 1448 H= 362:00 +CPU= 37.073
  16:18:53 STEP 1449 H= 362:15 +CPU= 36.677
  16:19:33 STEP 1450 H= 362:30 +CPU= 36.356
  16:20:13 STEP 1451 H= 362:45 +CPU= 36.238
  16:21:01 STEP 1452 H= 363:00 +CPU= 43.410
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001452
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001452
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001452
  16:21:40 STEP 1453 H= 363:15 +CPU= 34.754
  16:22:19 STEP 1454 H= 363:30 +CPU= 34.455
  16:22:58 STEP 1455 H= 363:45 +CPU= 34.802
  16:23:39 STEP 1456 H= 364:00 +CPU= 35.673
  16:24:18 STEP 1457 H= 364:15 +CPU= 35.546
  16:24:58 STEP 1458 H= 364:30 +CPU= 36.391
  16:25:39 STEP 1459 H= 364:45 +CPU= 36.966
  16:26:20 STEP 1460 H= 365:00 +CPU= 36.683
  16:26:58 STEP 1461 H= 365:15 +CPU= 35.424
  16:27:38 STEP 1462 H= 365:30 +CPU= 35.790
  16:28:19 STEP 1463 H= 365:45 +CPU= 37.691
  16:29:08 STEP 1464 H= 366:00 +CPU= 44.451
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001464
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001464
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001464
  16:29:47 STEP 1465 H= 366:15 +CPU= 35.735
  16:30:26 STEP 1466 H= 366:30 +CPU= 35.137
  16:31:06 STEP 1467 H= 366:45 +CPU= 35.699
  16:31:45 STEP 1468 H= 367:00 +CPU= 35.310
  16:32:25 STEP 1469 H= 367:15 +CPU= 35.503
  16:33:04 STEP 1470 H= 367:30 +CPU= 35.325
  16:33:43 STEP 1471 H= 367:45 +CPU= 35.615
  16:34:23 STEP 1472 H= 368:00 +CPU= 35.914
  16:35:02 STEP 1473 H= 368:15 +CPU= 35.066
  16:35:41 STEP 1474 H= 368:30 +CPU= 35.565
  16:36:20 STEP 1475 H= 368:45 +CPU= 35.115
  16:37:09 STEP 1476 H= 369:00 +CPU= 44.591
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001476
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001476
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001476
  16:37:49 STEP 1477 H= 369:15 +CPU= 34.827
  16:38:30 STEP 1478 H= 369:30 +CPU= 37.110
  16:39:10 STEP 1479 H= 369:45 +CPU= 36.744
  16:39:51 STEP 1480 H= 370:00 +CPU= 37.310
  16:40:31 STEP 1481 H= 370:15 +CPU= 34.954
  16:41:10 STEP 1482 H= 370:30 +CPU= 35.284
  16:41:49 STEP 1483 H= 370:45 +CPU= 34.451
  16:42:27 STEP 1484 H= 371:00 +CPU= 34.355
  16:43:05 STEP 1485 H= 371:15 +CPU= 34.302
  16:43:44 STEP 1486 H= 371:30 +CPU= 35.314
  16:44:24 STEP 1487 H= 371:45 +CPU= 35.450
  16:45:24 STEP 1488 H= 372:00 +CPU= 46.922
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001488
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001488
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001488
  16:46:04 STEP 1489 H= 372:15 +CPU= 35.974
  16:46:42 STEP 1490 H= 372:30 +CPU= 34.446
  16:47:21 STEP 1491 H= 372:45 +CPU= 35.375
  16:48:03 STEP 1492 H= 373:00 +CPU= 37.611
  16:48:44 STEP 1493 H= 373:15 +CPU= 36.381
  16:49:23 STEP 1494 H= 373:30 +CPU= 35.282
  16:50:02 STEP 1495 H= 373:45 +CPU= 35.515
  16:50:43 STEP 1496 H= 374:00 +CPU= 36.065
  16:51:23 STEP 1497 H= 374:15 +CPU= 35.432
  16:52:05 STEP 1498 H= 374:30 +CPU= 36.945
  16:52:46 STEP 1499 H= 374:45 +CPU= 37.056
  16:53:37 STEP 1500 H= 375:00 +CPU= 45.582
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001500
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001500
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001500
  16:54:17 STEP 1501 H= 375:15 +CPU= 36.335
  16:54:55 STEP 1502 H= 375:30 +CPU= 33.690
  16:55:31 STEP 1503 H= 375:45 +CPU= 30.808
  16:56:11 STEP 1504 H= 376:00 +CPU= 35.044
  16:56:51 STEP 1505 H= 376:15 +CPU= 34.114
  16:57:31 STEP 1506 H= 376:30 +CPU= 35.276
  16:58:10 STEP 1507 H= 376:45 +CPU= 35.419
  16:58:50 STEP 1508 H= 377:00 +CPU= 36.294
  16:59:29 STEP 1509 H= 377:15 +CPU= 35.337
  17:00:08 STEP 1510 H= 377:30 +CPU= 35.269
  17:00:47 STEP 1511 H= 377:45 +CPU= 34.650
  17:01:35 STEP 1512 H= 378:00 +CPU= 43.209
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001512
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001512
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001512
  17:02:15 STEP 1513 H= 378:15 +CPU= 35.351
  17:02:56 STEP 1514 H= 378:30 +CPU= 36.816
  17:03:37 STEP 1515 H= 378:45 +CPU= 36.434
  17:04:18 STEP 1516 H= 379:00 +CPU= 36.995
  17:04:59 STEP 1517 H= 379:15 +CPU= 36.127
  17:05:40 STEP 1518 H= 379:30 +CPU= 36.336
  17:06:20 STEP 1519 H= 379:45 +CPU= 36.808
  17:07:01 STEP 1520 H= 380:00 +CPU= 35.936
  17:07:41 STEP 1521 H= 380:15 +CPU= 36.122
  17:08:19 STEP 1522 H= 380:30 +CPU= 34.772
  17:08:58 STEP 1523 H= 380:45 +CPU= 34.727
  17:09:47 STEP 1524 H= 381:00 +CPU= 44.097
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001524
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001524
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001524
  17:10:26 STEP 1525 H= 381:15 +CPU= 34.354
  17:11:05 STEP 1526 H= 381:30 +CPU= 34.558
  17:11:44 STEP 1527 H= 381:45 +CPU= 34.626
  17:12:24 STEP 1528 H= 382:00 +CPU= 34.578
  17:13:04 STEP 1529 H= 382:15 +CPU= 35.509
  17:13:43 STEP 1530 H= 382:30 +CPU= 35.190
  17:14:23 STEP 1531 H= 382:45 +CPU= 35.203
  17:15:03 STEP 1532 H= 383:00 +CPU= 35.761
  17:15:41 STEP 1533 H= 383:15 +CPU= 33.530
  17:16:21 STEP 1534 H= 383:30 +CPU= 35.077
  17:17:02 STEP 1535 H= 383:45 +CPU= 35.452
  17:18:01 STEP 1536 H= 384:00 +CPU= 46.413
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001536
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001536
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001536
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001452
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001452
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001452
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001464
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001464
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001464
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001476
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001476
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001476
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001488
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001488
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001488
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001500
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001500
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001500
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001512
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001512
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001512
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001524
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001524
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001524
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a4zh_2016092300_20_1018_12294788_0_r1368559108_15.zip
Uploading the intermediate file: upload_file_15.zip
  17:18:42 STEP 1537 H= 384:15 +CPU= 34.457
  17:19:21 STEP 1538 H= 384:30 +CPU= 33.841
  17:19:59 STEP 1539 H= 384:45 +CPU= 34.404
  17:20:39 STEP 1540 H= 385:00 +CPU= 34.969
  17:21:18 STEP 1541 H= 385:15 +CPU= 36.202
  17:21:58 STEP 1542 H= 385:30 +CPU= 35.221
  17:22:37 STEP 1543 H= 385:45 +CPU= 35.727
  17:23:16 STEP 1544 H= 386:00 +CPU= 35.280
  17:23:55 STEP 1545 H= 386:15 +CPU= 34.861
  17:24:34 STEP 1546 H= 386:30 +CPU= 36.497
  17:25:15 STEP 1547 H= 386:45 +CPU= 37.027
  17:26:05 STEP 1548 H= 387:00 +CPU= 45.084
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001548
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001548
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001548
  17:26:43 STEP 1549 H= 387:15 +CPU= 34.646
  17:27:20 STEP 1550 H= 387:30 +CPU= 33.656
  17:27:57 STEP 1551 H= 387:45 +CPU= 33.188
  17:28:37 STEP 1552 H= 388:00 +CPU= 35.885
  17:29:16 STEP 1553 H= 388:15 +CPU= 35.718
  17:29:55 STEP 1554 H= 388:30 +CPU= 35.384
  17:30:34 STEP 1555 H= 388:45 +CPU= 35.158
  17:31:14 STEP 1556 H= 389:00 +CPU= 36.546
  17:31:53 STEP 1557 H= 389:15 +CPU= 34.423
  17:32:33 STEP 1558 H= 389:30 +CPU= 34.922
  17:33:13 STEP 1559 H= 389:45 +CPU= 34.948
  17:34:05 STEP 1560 H= 390:00 +CPU= 45.381
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001560
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001560
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001560
  17:34:47 STEP 1561 H= 390:15 +CPU= 36.552
  17:35:28 STEP 1562 H= 390:30 +CPU= 35.383
  17:36:09 STEP 1563 H= 390:45 +CPU= 35.314
  17:36:51 STEP 1564 H= 391:00 +CPU= 37.118
  17:37:32 STEP 1565 H= 391:15 +CPU= 34.412
  17:38:15 STEP 1566 H= 391:30 +CPU= 36.089
  17:38:55 STEP 1567 H= 391:45 +CPU= 34.529
  17:39:38 STEP 1568 H= 392:00 +CPU= 36.795
  17:40:19 STEP 1569 H= 392:15 +CPU= 35.526
  17:41:01 STEP 1570 H= 392:30 +CPU= 35.677
  17:41:43 STEP 1571 H= 392:45 +CPU= 36.551
  17:42:35 STEP 1572 H= 393:00 +CPU= 45.279
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001572
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001572
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001572
  17:43:15 STEP 1573 H= 393:15 +CPU= 34.888
  17:43:54 STEP 1574 H= 393:30 +CPU= 34.469
  17:44:35 STEP 1575 H= 393:45 +CPU= 34.829
  17:45:15 STEP 1576 H= 394:00 +CPU= 34.119
  17:45:54 STEP 1577 H= 394:15 +CPU= 34.674
  17:46:37 STEP 1578 H= 394:30 +CPU= 36.750
  17:47:18 STEP 1579 H= 394:45 +CPU= 35.650
  17:47:58 STEP 1580 H= 395:00 +CPU= 35.780
  17:48:37 STEP 1581 H= 395:15 +CPU= 34.769
  17:49:17 STEP 1582 H= 395:30 +CPU= 34.728
  17:49:55 STEP 1583 H= 395:45 +CPU= 34.663
  17:50:58 STEP 1584 H= 396:00 +CPU= 45.438
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001584
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001584
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001584
  17:51:39 STEP 1585 H= 396:15 +CPU= 35.557
  17:52:22 STEP 1586 H= 396:30 +CPU= 37.025
  17:53:04 STEP 1587 H= 396:45 +CPU= 36.373
  17:53:46 STEP 1588 H= 397:00 +CPU= 36.302
  17:54:27 STEP 1589 H= 397:15 +CPU= 35.618
  17:55:08 STEP 1590 H= 397:30 +CPU= 35.727
  17:55:50 STEP 1591 H= 397:45 +CPU= 35.836
  17:56:32 STEP 1592 H= 398:00 +CPU= 35.828
  17:57:14 STEP 1593 H= 398:15 +CPU= 36.843
  17:57:53 STEP 1594 H= 398:30 +CPU= 34.549
  17:58:34 STEP 1595 H= 398:45 +CPU= 34.736
  17:59:25 STEP 1596 H= 399:00 +CPU= 43.271
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001596
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001596
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001596
  18:00:04 STEP 1597 H= 399:15 +CPU= 34.238
  18:00:44 STEP 1598 H= 399:30 +CPU= 33.882
  18:01:24 STEP 1599 H= 399:45 +CPU= 33.409
  18:02:07 STEP 1600 H= 400:00 +CPU= 35.419
  18:02:50 STEP 1601 H= 400:15 +CPU= 35.517
  18:03:33 STEP 1602 H= 400:30 +CPU= 37.186
  18:04:14 STEP 1603 H= 400:45 +CPU= 34.968
  18:04:55 STEP 1604 H= 401:00 +CPU= 35.318
  18:05:35 STEP 1605 H= 401:15 +CPU= 34.594
  18:06:13 STEP 1606 H= 401:30 +CPU= 34.459
  18:06:54 STEP 1607 H= 401:45 +CPU= 34.998
  18:07:45 STEP 1608 H= 402:00 +CPU= 44.768
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001608
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001608
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001608
  18:08:26 STEP 1609 H= 402:15 +CPU= 35.341
  18:09:06 STEP 1610 H= 402:30 +CPU= 35.675
  18:09:47 STEP 1611 H= 402:45 +CPU= 36.077
  18:10:31 STEP 1612 H= 403:00 +CPU= 37.561
  18:11:12 STEP 1613 H= 403:15 +CPU= 36.394
  18:11:54 STEP 1614 H= 403:30 +CPU= 35.745
  18:12:33 STEP 1615 H= 403:45 +CPU= 34.961
  18:13:14 STEP 1616 H= 404:00 +CPU= 36.121
  18:13:55 STEP 1617 H= 404:15 +CPU= 36.604
  18:14:36 STEP 1618 H= 404:30 +CPU= 35.437
  18:15:17 STEP 1619 H= 404:45 +CPU= 35.811
  18:16:07 STEP 1620 H= 405:00 +CPU= 44.087
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001620
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001620
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001620
  18:16:47 STEP 1621 H= 405:15 +CPU= 35.364
  18:17:29 STEP 1622 H= 405:30 +CPU= 36.864
  18:18:10 STEP 1623 H= 405:45 +CPU= 36.864
  18:18:53 STEP 1624 H= 406:00 +CPU= 37.386
  18:19:32 STEP 1625 H= 406:15 +CPU= 34.005
  18:20:13 STEP 1626 H= 406:30 +CPU= 35.348
  18:20:53 STEP 1627 H= 406:45 +CPU= 33.712
  18:21:34 STEP 1628 H= 407:00 +CPU= 36.379
  18:22:15 STEP 1629 H= 407:15 +CPU= 36.182
  18:22:56 STEP 1630 H= 407:30 +CPU= 36.493
  18:23:35 STEP 1631 H= 407:45 +CPU= 34.809
  18:24:47 STEP 1632 H= 408:00 +CPU= 45.529
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001632
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001632
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001536
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001536
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001536
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001548
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001548
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001548
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001560
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001560
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001560
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001572
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001572
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001572
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001584
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001584
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001584
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001596
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001596
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001596
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001608
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001608
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001608
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001620
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001620
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001620
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a4zh_2016092300_20_1018_12294788_0_r1368559108_16.zip
Uploading the intermediate file: upload_file_16.zip
  18:25:29 STEP 1633 H= 408:15 +CPU= 33.994
  18:26:09 STEP 1634 H= 408:30 +CPU= 36.192
  18:26:48 STEP 1635 H= 408:45 +CPU= 34.700
  18:27:29 STEP 1636 H= 409:00 +CPU= 36.935
  18:28:08 STEP 1637 H= 409:15 +CPU= 35.001
  18:28:49 STEP 1638 H= 409:30 +CPU= 36.633
  18:29:31 STEP 1639 H= 409:45 +CPU= 37.245
  18:30:11 STEP 1640 H= 410:00 +CPU= 35.927
  18:30:51 STEP 1641 H= 410:15 +CPU= 35.335
  18:31:31 STEP 1642 H= 410:30 +CPU= 35.827
  18:32:12 STEP 1643 H= 410:45 +CPU= 35.759
  18:33:03 STEP 1644 H= 411:00 +CPU= 45.905
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001644
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001644
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001644
  18:33:42 STEP 1645 H= 411:15 +CPU= 35.018
  18:34:21 STEP 1646 H= 411:30 +CPU= 34.395
  18:35:00 STEP 1647 H= 411:45 +CPU= 34.874
  18:35:40 STEP 1648 H= 412:00 +CPU= 35.620
  18:36:20 STEP 1649 H= 412:15 +CPU= 35.694
  18:37:00 STEP 1650 H= 412:30 +CPU= 36.023
  18:37:40 STEP 1651 H= 412:45 +CPU= 35.254
  18:38:20 STEP 1652 H= 413:00 +CPU= 35.973
  18:38:59 STEP 1653 H= 413:15 +CPU= 34.839
  18:39:38 STEP 1654 H= 413:30 +CPU= 35.223
  18:40:17 STEP 1655 H= 413:45 +CPU= 35.089
  18:41:09 STEP 1656 H= 414:00 +CPU= 45.295
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001656
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001656
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001656
  18:41:49 STEP 1657 H= 414:15 +CPU= 35.965
  18:42:30 STEP 1658 H= 414:30 +CPU= 36.806
  18:43:10 STEP 1659 H= 414:45 +CPU= 36.682
  18:43:51 STEP 1660 H= 415:00 +CPU= 36.749
  18:44:31 STEP 1661 H= 415:15 +CPU= 35.554
  18:45:10 STEP 1662 H= 415:30 +CPU= 34.968
  18:45:49 STEP 1663 H= 415:45 +CPU= 35.396
  18:46:29 STEP 1664 H= 416:00 +CPU= 36.209
  18:47:08 STEP 1665 H= 416:15 +CPU= 35.159
  18:47:49 STEP 1666 H= 416:30 +CPU= 36.705
  18:48:28 STEP 1667 H= 416:45 +CPU= 35.109
  18:49:16 STEP 1668 H= 417:00 +CPU= 43.322
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001668
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001668
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001668
  18:49:54 STEP 1669 H= 417:15 +CPU= 34.247
  18:50:33 STEP 1670 H= 417:30 +CPU= 34.631
  18:51:14 STEP 1671 H= 417:45 +CPU= 36.171
  18:51:54 STEP 1672 H= 418:00 +CPU= 35.849
  18:52:34 STEP 1673 H= 418:15 +CPU= 36.091
  18:53:14 STEP 1674 H= 418:30 +CPU= 35.306
  18:53:53 STEP 1675 H= 418:45 +CPU= 34.890
  18:54:34 STEP 1676 H= 419:00 +CPU= 36.680
  18:55:12 STEP 1677 H= 419:15 +CPU= 34.904
  18:55:51 STEP 1678 H= 419:30 +CPU= 35.181
  18:56:31 STEP 1679 H= 419:45 +CPU= 36.031
  18:57:46 STEP 1680 H= 420:00 +CPU= 48.391
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001680
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001680
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001680
  18:58:27 STEP 1681 H= 420:15 +CPU= 36.573
  18:59:09 STEP 1682 H= 420:30 +CPU= 37.659
  18:59:50 STEP 1683 H= 420:45 +CPU= 36.826
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a4zh
(argv4) batchid: 1018
(argv5) wuid: 12294788
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/6
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/6/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/6/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_2efdce5e42fa599209cf23a52cbb355e to: /var/lib/boinc/slots/6/oifs_43r3_bl_a4zh_2016092300_20_1018_12294788.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/6/oifs_43r3_bl_a4zh_2016092300_20_1018_12294788.zip
ic_ancil_file: ic_ancil_12294788
ifsdata_file: ifsdata_12294788
climate_data_file: clim_data_12294788
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc/slots/6/ic_ancil_12294788.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/6/ic_ancil_12294788.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_ea425c7ed6f887d570932d9ccb831b7b to: /var/lib/boinc/slots/6/ifsdata/ifsdata_12294788.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/6/ifsdata/ifsdata_12294788.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/6/159l_2/clim_data_12294788.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/6/159l_2/clim_data_12294788.zip
Checking for progress XML file: /var/lib/boinc/slots/6/progress_file_12294788.xml
Opened progress file ok : /var/lib/boinc/slots/6/progress_file_12294788.xml
-- Model is restarting --
Adjusting last_iter, 1684, to previous model restart step.
Creating progress file: /var/lib/boinc/slots/6/progress_file_12294788.xml
last_cpu_time: 53340
upload_file_number: 17
last_iter: 1681
last_upload: 1468800
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a4zh_2016092300_20_1018_12294788_0_r1368559108
The child process has been launched with process id: 377825
Executing the command: /var/lib/boinc/slots/6/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.001] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:377825:377825] [20240614:190136:1718384496:0.001] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  19:02:24 STEP 1680 H= 420:00 +CPU= 21.588
  19:02:52 STEP 1681 H= 420:15 +CPU= 26.622
  19:03:23 STEP 1682 H= 420:30 +CPU= 29.782
  19:04:03 STEP 1683 H= 420:45 +CPU= 35.917
  19:04:44 STEP 1684 H= 421:00 +CPU= 37.225
  19:05:25 STEP 1685 H= 421:15 +CPU= 36.595
  19:06:05 STEP 1686 H= 421:30 +CPU= 36.120
  19:06:45 STEP 1687 H= 421:45 +CPU= 36.694
  19:07:25 STEP 1688 H= 422:00 +CPU= 36.822
  19:08:05 STEP 1689 H= 422:15 +CPU= 36.108
  19:08:46 STEP 1690 H= 422:30 +CPU= 36.623
  19:09:26 STEP 1691 H= 422:45 +CPU= 35.989
  19:10:16 STEP 1692 H= 423:00 +CPU= 45.427
  19:10:54 STEP 1693 H= 423:15 +CPU= 34.802
  19:11:36 STEP 1694 H= 423:30 +CPU= 37.205
  19:12:18 STEP 1695 H= 423:45 +CPU= 36.940
  19:13:00 STEP 1696 H= 424:00 +CPU= 37.901
  19:13:41 STEP 1697 H= 424:15 +CPU= 36.924
  19:14:21 STEP 1698 H= 424:30 +CPU= 37.209
  19:15:02 STEP 1699 H= 424:45 +CPU= 37.319
  19:15:42 STEP 1700 H= 425:00 +CPU= 36.919
  19:16:24 STEP 1701 H= 425:15 +CPU= 37.610
  19:17:05 STEP 1702 H= 425:30 +CPU= 37.356
  19:17:45 STEP 1703 H= 425:45 +CPU= 36.081
  19:18:33 STEP 1704 H= 426:00 +CPU= 44.070
  19:19:13 STEP 1705 H= 426:15 +CPU= 35.705
  19:19:52 STEP 1706 H= 426:30 +CPU= 35.939
  19:20:33 STEP 1707 H= 426:45 +CPU= 35.995
  19:21:14 STEP 1708 H= 427:00 +CPU= 36.834
  19:21:54 STEP 1709 H= 427:15 +CPU= 35.561
  19:22:33 STEP 1710 H= 427:30 +CPU= 35.923
  19:23:14 STEP 1711 H= 427:45 +CPU= 36.203
  19:23:56 STEP 1712 H= 428:00 +CPU= 37.609
  19:24:36 STEP 1713 H= 428:15 +CPU= 36.183
  19:25:17 STEP 1714 H= 428:30 +CPU= 37.007
  19:25:59 STEP 1715 H= 428:45 +CPU= 37.364
  19:26:49 STEP 1716 H= 429:00 +CPU= 45.157
  19:27:29 STEP 1717 H= 429:15 +CPU= 36.294
  19:28:08 STEP 1718 H= 429:30 +CPU= 35.523
  19:28:47 STEP 1719 H= 429:45 +CPU= 35.839
  19:29:29 STEP 1720 H= 430:00 +CPU= 37.231
  19:30:08 STEP 1721 H= 430:15 +CPU= 35.686
  19:30:49 STEP 1722 H= 430:30 +CPU= 36.788
  19:31:29 STEP 1723 H= 430:45 +CPU= 35.653
  19:32:10 STEP 1724 H= 431:00 +CPU= 37.254
  19:32:49 STEP 1725 H= 431:15 +CPU= 35.799
  19:33:30 STEP 1726 H= 431:30 +CPU= 37.082
  19:34:10 STEP 1727 H= 431:45 +CPU= 35.742
  19:35:09 STEP 1728 H= 432:00 +CPU= 39.601
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001728
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001728
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001632
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001632
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001644
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001644
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001644
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001656
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001656
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001656
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001668
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001668
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001668
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001680
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001680
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001680
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a4zh_2016092300_20_1018_12294788_0_r1368559108_17.zip
Uploading the intermediate file: upload_file_17.zip
  19:35:51 STEP 1729 H= 432:15 +CPU= 34.764
  19:36:32 STEP 1730 H= 432:30 +CPU= 33.315
  19:37:11 STEP 1731 H= 432:45 +CPU= 32.662
  19:37:51 STEP 1732 H= 433:00 +CPU= 34.428
  19:38:31 STEP 1733 H= 433:15 +CPU= 35.165
  19:39:09 STEP 1734 H= 433:30 +CPU= 35.247
  19:39:50 STEP 1735 H= 433:45 +CPU= 35.777
  19:40:30 STEP 1736 H= 434:00 +CPU= 36.323
  19:41:11 STEP 1737 H= 434:15 +CPU= 36.817
  19:41:51 STEP 1738 H= 434:30 +CPU= 36.011
  19:42:31 STEP 1739 H= 434:45 +CPU= 36.216
  19:43:21 STEP 1740 H= 435:00 +CPU= 43.959
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001740
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001740
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001740
  19:44:00 STEP 1741 H= 435:15 +CPU= 35.616
  19:44:37 STEP 1742 H= 435:30 +CPU= 34.733
  19:45:16 STEP 1743 H= 435:45 +CPU= 35.083
  19:45:56 STEP 1744 H= 436:00 +CPU= 36.360
  19:46:35 STEP 1745 H= 436:15 +CPU= 35.720
  19:47:14 STEP 1746 H= 436:30 +CPU= 35.473
  19:47:53 STEP 1747 H= 436:45 +CPU= 35.633
  19:48:33 STEP 1748 H= 437:00 +CPU= 35.904
  19:49:12 STEP 1749 H= 437:15 +CPU= 36.094
  19:49:52 STEP 1750 H= 437:30 +CPU= 36.183
  19:50:31 STEP 1751 H= 437:45 +CPU= 35.720
  19:51:20 STEP 1752 H= 438:00 +CPU= 44.414
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001752
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001752
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001752
  19:51:59 STEP 1753 H= 438:15 +CPU= 34.555
  19:52:37 STEP 1754 H= 438:30 +CPU= 35.012
  19:53:17 STEP 1755 H= 438:45 +CPU= 36.542
  19:53:58 STEP 1756 H= 439:00 +CPU= 36.015
  19:54:38 STEP 1757 H= 439:15 +CPU= 36.304
  19:55:17 STEP 1758 H= 439:30 +CPU= 35.948
  19:55:58 STEP 1759 H= 439:45 +CPU= 36.799
  19:56:39 STEP 1760 H= 440:00 +CPU= 37.116
  19:57:16 STEP 1761 H= 440:15 +CPU= 34.725
  19:57:56 STEP 1762 H= 440:30 +CPU= 35.333
  19:58:36 STEP 1763 H= 440:45 +CPU= 36.323
  19:59:28 STEP 1764 H= 441:00 +CPU= 45.921
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001764
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001764
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001764
  20:00:09 STEP 1765 H= 441:15 +CPU= 36.294
  20:00:50 STEP 1766 H= 441:30 +CPU= 36.787
  20:01:31 STEP 1767 H= 441:45 +CPU= 36.894
  20:02:12 STEP 1768 H= 442:00 +CPU= 36.919
  20:02:52 STEP 1769 H= 442:15 +CPU= 36.432
  20:03:32 STEP 1770 H= 442:30 +CPU= 35.553
  20:04:10 STEP 1771 H= 442:45 +CPU= 34.976
  20:04:51 STEP 1772 H= 443:00 +CPU= 37.138
  20:05:31 STEP 1773 H= 443:15 +CPU= 36.587
  20:06:09 STEP 1774 H= 443:30 +CPU= 34.255
  20:06:50 STEP 1775 H= 443:45 +CPU= 36.932
  20:08:13 STEP 1776 H= 444:00 +CPU= 34.761
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001776
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001776
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001776
  20:08:37 STEP 1777 H= 444:15 +CPU= 22.610
  20:09:03 STEP 1778 H= 444:30 +CPU= 25.047
  20:09:42 STEP 1779 H= 444:45 +CPU= 33.676
  20:10:23 STEP 1780 H= 445:00 +CPU= 36.772
  20:11:03 STEP 1781 H= 445:15 +CPU= 36.219
  20:11:44 STEP 1782 H= 445:30 +CPU= 36.355
  20:12:23 STEP 1783 H= 445:45 +CPU= 36.325
  20:13:04 STEP 1784 H= 446:00 +CPU= 36.931
  20:13:44 STEP 1785 H= 446:15 +CPU= 36.076
  20:14:24 STEP 1786 H= 446:30 +CPU= 35.949
  20:15:02 STEP 1787 H= 446:45 +CPU= 35.379
  20:15:52 STEP 1788 H= 447:00 +CPU= 43.900
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001788
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001788
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001788
  20:16:30 STEP 1789 H= 447:15 +CPU= 34.500
  20:17:09 STEP 1790 H= 447:30 +CPU= 35.858
  20:17:48 STEP 1791 H= 447:45 +CPU= 34.756
  20:18:28 STEP 1792 H= 448:00 +CPU= 36.628
  20:19:09 STEP 1793 H= 448:15 +CPU= 36.863
  20:19:49 STEP 1794 H= 448:30 +CPU= 36.746
  20:20:30 STEP 1795 H= 448:45 +CPU= 35.959
  20:21:11 STEP 1796 H= 449:00 +CPU= 37.289
  20:21:50 STEP 1797 H= 449:15 +CPU= 35.134
  20:22:31 STEP 1798 H= 449:30 +CPU= 36.508
  20:23:10 STEP 1799 H= 449:45 +CPU= 35.235
  20:24:02 STEP 1800 H= 450:00 +CPU= 46.014
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001800
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001800
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001800
  20:24:40 STEP 1801 H= 450:15 +CPU= 34.583
  20:25:19 STEP 1802 H= 450:30 +CPU= 35.524
  20:25:59 STEP 1803 H= 450:45 +CPU= 35.360
  20:26:38 STEP 1804 H= 451:00 +CPU= 36.107
  20:27:17 STEP 1805 H= 451:15 +CPU= 36.138
  20:27:57 STEP 1806 H= 451:30 +CPU= 35.797
  20:28:36 STEP 1807 H= 451:45 +CPU= 35.557
  20:29:16 STEP 1808 H= 452:00 +CPU= 36.786
  20:29:56 STEP 1809 H= 452:15 +CPU= 35.358
  20:30:36 STEP 1810 H= 452:30 +CPU= 35.522
  20:31:15 STEP 1811 H= 452:45 +CPU= 36.170
  20:32:04 STEP 1812 H= 453:00 +CPU= 42.963
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001812
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001812
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001812
  20:32:43 STEP 1813 H= 453:15 +CPU= 35.524
  20:33:21 STEP 1814 H= 453:30 +CPU= 35.339
  20:34:03 STEP 1815 H= 453:45 +CPU= 36.864
  20:34:43 STEP 1816 H= 454:00 +CPU= 36.576
  20:35:23 STEP 1817 H= 454:15 +CPU= 36.866
  20:36:03 STEP 1818 H= 454:30 +CPU= 36.098
  20:36:43 STEP 1819 H= 454:45 +CPU= 36.274
  20:37:24 STEP 1820 H= 455:00 +CPU= 37.341
  20:38:04 STEP 1821 H= 455:15 +CPU= 36.147
  20:38:44 STEP 1822 H= 455:30 +CPU= 36.548
  20:39:25 STEP 1823 H= 455:45 +CPU= 36.638
  20:40:25 STEP 1824 H= 456:00 +CPU= 44.880
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001824
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001824
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001728
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001728
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001740
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001740
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001740
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001752
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001752
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001752
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001764
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001764
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001764
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001776
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001776
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001776
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001788
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001788
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001788
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001800
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001800
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001800
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001812
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001812
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001812
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a4zh_2016092300_20_1018_12294788_0_r1368559108_18.zip
Uploading the intermediate file: upload_file_18.zip
  20:41:03 STEP 1825 H= 456:15 +CPU= 32.052
  20:41:43 STEP 1826 H= 456:30 +CPU= 31.793
  20:42:18 STEP 1827 H= 456:45 +CPU= 30.811
  20:42:56 STEP 1828 H= 457:00 +CPU= 33.683
  20:43:36 STEP 1829 H= 457:15 +CPU= 31.369
  20:44:16 STEP 1830 H= 457:30 +CPU= 34.968
  20:44:55 STEP 1831 H= 457:45 +CPU= 35.448
  20:45:34 STEP 1832 H= 458:00 +CPU= 35.993
  20:46:14 STEP 1833 H= 458:15 +CPU= 36.072
  20:46:53 STEP 1834 H= 458:30 +CPU= 35.370
  20:47:34 STEP 1835 H= 458:45 +CPU= 36.945
  20:48:22 STEP 1836 H= 459:00 +CPU= 43.287
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001836
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001836
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001836
  20:49:02 STEP 1837 H= 459:15 +CPU= 36.209
  20:49:43 STEP 1838 H= 459:30 +CPU= 36.530
  20:50:24 STEP 1839 H= 459:45 +CPU= 36.111
  20:51:04 STEP 1840 H= 460:00 +CPU= 36.248
  20:51:45 STEP 1841 H= 460:15 +CPU= 36.713
  20:52:26 STEP 1842 H= 460:30 +CPU= 36.019
  20:53:04 STEP 1843 H= 460:45 +CPU= 35.174
  20:53:45 STEP 1844 H= 461:00 +CPU= 36.569
  20:54:24 STEP 1845 H= 461:15 +CPU= 34.931
  20:55:03 STEP 1846 H= 461:30 +CPU= 35.256
  20:55:42 STEP 1847 H= 461:45 +CPU= 35.558
  20:56:32 STEP 1848 H= 462:00 +CPU= 44.610
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001848
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001848
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001848
  20:57:09 STEP 1849 H= 462:15 +CPU= 33.786
  20:57:49 STEP 1850 H= 462:30 +CPU= 35.835
  20:58:29 STEP 1851 H= 462:45 +CPU= 35.277
  20:59:10 STEP 1852 H= 463:00 +CPU= 36.923
  20:59:49 STEP 1853 H= 463:15 +CPU= 34.852
  21:00:30 STEP 1854 H= 463:30 +CPU= 35.630
  21:01:11 STEP 1855 H= 463:45 +CPU= 36.211
  21:01:52 STEP 1856 H= 464:00 +CPU= 36.417
  21:02:33 STEP 1857 H= 464:15 +CPU= 36.985
  21:03:12 STEP 1858 H= 464:30 +CPU= 35.634
  21:03:54 STEP 1859 H= 464:45 +CPU= 37.252
  21:04:44 STEP 1860 H= 465:00 +CPU= 44.835
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001860
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001860
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001860
  21:05:22 STEP 1861 H= 465:15 +CPU= 34.813
  21:06:04 STEP 1862 H= 465:30 +CPU= 36.772
  21:06:44 STEP 1863 H= 465:45 +CPU= 36.245
  21:07:26 STEP 1864 H= 466:00 +CPU= 37.256
  21:08:07 STEP 1865 H= 466:15 +CPU= 36.566
  21:08:47 STEP 1866 H= 466:30 +CPU= 36.040
  21:09:26 STEP 1867 H= 466:45 +CPU= 35.856
  21:10:05 STEP 1868 H= 467:00 +CPU= 35.535
  21:10:46 STEP 1869 H= 467:15 +CPU= 35.769
  21:11:25 STEP 1870 H= 467:30 +CPU= 36.434
  21:12:06 STEP 1871 H= 467:45 +CPU= 35.655
  21:14:20 STEP 1872 H= 468:00 +CPU= 37.356
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001872
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001872
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001872
  21:14:48 STEP 1873 H= 468:15 +CPU= 26.151
  21:15:25 STEP 1874 H= 468:30 +CPU= 32.354
  21:16:06 STEP 1875 H= 468:45 +CPU= 36.205
  21:16:46 STEP 1876 H= 469:00 +CPU= 35.631
  21:17:26 STEP 1877 H= 469:15 +CPU= 35.871
  21:18:07 STEP 1878 H= 469:30 +CPU= 36.283
  21:18:47 STEP 1879 H= 469:45 +CPU= 36.353
  21:19:28 STEP 1880 H= 470:00 +CPU= 37.126
  21:20:10 STEP 1881 H= 470:15 +CPU= 37.109
  21:20:52 STEP 1882 H= 470:30 +CPU= 36.887
  21:21:32 STEP 1883 H= 470:45 +CPU= 36.605
  21:22:23 STEP 1884 H= 471:00 +CPU= 43.583
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001884
  21:23:02 STEP 1885 H= 471:15 +CPU= 35.199
  21:23:43 STEP 1886 H= 471:30 +CPU= 36.986
  21:24:24 STEP 1887 H= 471:45 +CPU= 35.321
  21:25:05 STEP 1888 H= 472:00 +CPU= 36.992
  21:25:44 STEP 1889 H= 472:15 +CPU= 36.217
  21:26:25 STEP 1890 H= 472:30 +CPU= 36.110
  21:27:05 STEP 1891 H= 472:45 +CPU= 36.596
  21:27:46 STEP 1892 H= 473:00 +CPU= 37.360
  21:28:27 STEP 1893 H= 473:15 +CPU= 36.777
  21:29:08 STEP 1894 H= 473:30 +CPU= 36.967
  21:29:48 STEP 1895 H= 473:45 +CPU= 36.563
  21:30:39 STEP 1896 H= 474:00 +CPU= 45.216
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001896
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001896
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001896
  21:31:18 STEP 1897 H= 474:15 +CPU= 35.328
  21:31:58 STEP 1898 H= 474:30 +CPU= 36.054
  21:32:37 STEP 1899 H= 474:45 +CPU= 35.108
  21:33:16 STEP 1900 H= 475:00 +CPU= 35.781
  21:33:55 STEP 1901 H= 475:15 +CPU= 34.931
  21:34:35 STEP 1902 H= 475:30 +CPU= 36.362
  21:35:14 STEP 1903 H= 475:45 +CPU= 35.649
  21:35:55 STEP 1904 H= 476:00 +CPU= 37.278
  21:36:34 STEP 1905 H= 476:15 +CPU= 35.325
  21:37:14 STEP 1906 H= 476:30 +CPU= 36.054
  21:37:55 STEP 1907 H= 476:45 +CPU= 37.425
  21:38:45 STEP 1908 H= 477:00 +CPU= 44.784
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001908
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001908
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001908
  21:39:23 STEP 1909 H= 477:15 +CPU= 34.651
  21:40:04 STEP 1910 H= 477:30 +CPU= 37.046
  21:40:45 STEP 1911 H= 477:45 +CPU= 36.583
  21:41:26 STEP 1912 H= 478:00 +CPU= 37.356
  21:42:07 STEP 1913 H= 478:15 +CPU= 36.397
  21:42:47 STEP 1914 H= 478:30 +CPU= 36.284
  21:43:26 STEP 1915 H= 478:45 +CPU= 36.302
  21:44:07 STEP 1916 H= 479:00 +CPU= 36.806
  21:44:47 STEP 1917 H= 479:15 +CPU= 36.347
  21:45:27 STEP 1918 H= 479:30 +CPU= 35.727
  21:46:06 STEP 1919 H= 479:45 +CPU= 35.250
  21:47:51 STEP 1920 H= 480:00 +CPU= 39.058
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  480.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.1824961942E-20      I       0.1030170119E-04      I
ICloud fractionI      0.0000000000E+00      I       0.3997196265E-01      I
I  Relat. Hum. I      0.3133791279E+00      I       0.3856168291E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.7229632469E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    700935172 BYTES IN      7 RECORDS TRANSFERRED IN    0.0003 SECONDS******** Mbytes/s, TOTAL TIME=    0.0141( 2.1%)
 21:47:51 STEP 1920 H= 480:00 +CPU= 39.058
 END CNT3
 NSTEP =  1920 CNT0   000000000  29.714  29.714  35.231   0.843               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN         7150/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    128    0       46      1044068     7150 ************    96.2     1.4    94.9     1.8   179.0
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 9975660.1 9975660.1    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 9950184.0 9950184.0     99.74      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  261    6718.6    6718.6     17.58      0.00
   9 SPCM     - SPECTRAL COMP.                     240    1254.0    1254.0      3.02      0.00
  10 SCAN2M - PHYSICS                              241   18763.6   18763.6     45.33      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 20    6337.4    6337.4      1.27      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 11      41.3      41.3      0.00      0.00
  17 GRIDFPOS IN CNT4                               20      39.8      39.8      0.01      0.00
  27 RESTART FILES - WRITING                         5   25349.6   25349.6      1.27      0.00
  28 RESTART FILES - READING                         1   12167.6   12167.6      0.12      0.00
  29 SU4FPOS IN CNT4                                20       3.1       3.1      0.00      0.00
  30 DYNFPOS IN CNT4                                20   13473.7   13473.7      2.70      0.00
  31 POSDDH IN STEPO                                 2     627.6     627.6      0.01      0.00
  37 CPGLAG   - SL COMPUTATIONS                    241    8337.0    8337.0     20.14      0.00
  39 SU0YOMB                                         1    5206.1    5206.1      0.05      0.00
  51 SCAN2M   - SL COMM. PART 1                    241      35.4      35.4      0.09      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              240     522.6     522.6      1.26      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              240     129.2     129.2      0.31      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 11       0.3       0.3      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      524     606.1     606.1      3.18      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       500     401.6     401.6      2.01      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        500     111.8     111.8      0.56      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       524     210.3     210.3      1.10      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      72.4      72.4      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            524      66.7      66.7      0.35      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            500      32.5      32.5      0.16      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            524      82.2      82.2      0.43      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            500      37.2      37.2      0.19      0.00
 400 GSTATS                                      28864       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 26802       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME     9975.7 CPU TIME    8753.6 VECTOR TIME     8753.6


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      173.7
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc/slots/6/ICMGGh7zg+001920
Moving to projects directory: /var/lib/boinc/slots/6/ICMSHh7zg+001920
Moving to projects directory: /var/lib/boinc/slots/6/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc/slots/6/NODE.001_01
Adding to the zip: /var/lib/boinc/slots/6/ifs.stat
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001836
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001836
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001836
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001860
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001860
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001860
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001884
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001884
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001884
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001908
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001908
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001908
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMGGh7zg+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMSHh7zg+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12294788/ICMUAh7zg+001920
Zipping up the final file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a4zh_2016092300_20_1018_12294788_0_r1368559108_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
21:51:29 (377643): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
14 Jun 2024 19:51:05 1507812 22448408 oifs_43r3_bl_a4zh_2016092300_20_1018_12294788_0 1,727,100 62,093 0.0360
14 Jun 2024 09:45:05 1507812 22448408 oifs_43r3_bl_a4zh_2016092300_20_1018_12294788_0 864,000 27,145 0.0314


©2024 cpdn.org