climateprediction.net home page
Task 22445933

Task 22445933

Name oifs_43r3_bl_a334_2016092300_20_1018_12292327_0
Workunit 12292327
Created 13 Jun 2024, 1:49:13 UTC
Sent 13 Jun 2024, 8:20:59 UTC
Report deadline 12 Aug 2024, 8:20:59 UTC
Received 14 Jun 2024, 12:35:48 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1549573
Run time 21 hours 39 min 51 sec
CPU time 21 hours 38 min 20 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 8.07 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,025.60 MB
Peak swap size 5,981.13 MB
Peak disk usage 1,338.37 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<stderr_txt>
ediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001428
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a334_2016092300_20_1018_12292327_0_r262561884_14.zip
Uploading the intermediate file: upload_file_14.zip
  07:36:09 STEP 1441 H= 360:15 +CPU= 36.239
  07:36:44 STEP 1442 H= 360:30 +CPU= 34.610
  07:37:21 STEP 1443 H= 360:45 +CPU= 36.169
  07:37:58 STEP 1444 H= 361:00 +CPU= 36.264
  07:38:34 STEP 1445 H= 361:15 +CPU= 35.807
  07:39:12 STEP 1446 H= 361:30 +CPU= 37.312
  07:39:48 STEP 1447 H= 361:45 +CPU= 35.313
  07:40:27 STEP 1448 H= 362:00 +CPU= 38.458
  07:41:04 STEP 1449 H= 362:15 +CPU= 35.747
  07:41:39 STEP 1450 H= 362:30 +CPU= 34.264
  07:42:16 STEP 1451 H= 362:45 +CPU= 37.082
  07:43:00 STEP 1452 H= 363:00 +CPU= 42.852
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001452
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001452
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001452
  07:43:36 STEP 1453 H= 363:15 +CPU= 34.861
  07:44:12 STEP 1454 H= 363:30 +CPU= 35.908
  07:44:49 STEP 1455 H= 363:45 +CPU= 35.426
  07:45:28 STEP 1456 H= 364:00 +CPU= 38.797
  07:46:04 STEP 1457 H= 364:15 +CPU= 35.490
  07:46:40 STEP 1458 H= 364:30 +CPU= 35.470
  07:47:19 STEP 1459 H= 364:45 +CPU= 37.820
  07:47:56 STEP 1460 H= 365:00 +CPU= 36.776
  07:48:32 STEP 1461 H= 365:15 +CPU= 35.132
  07:49:07 STEP 1462 H= 365:30 +CPU= 33.881
  07:49:46 STEP 1463 H= 365:45 +CPU= 38.821
  07:50:30 STEP 1464 H= 366:00 +CPU= 43.192
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001464
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001464
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001464
  07:51:10 STEP 1465 H= 366:15 +CPU= 39.678
  07:51:47 STEP 1466 H= 366:30 +CPU= 36.535
  07:52:24 STEP 1467 H= 366:45 +CPU= 35.952
  07:53:01 STEP 1468 H= 367:00 +CPU= 36.533
  07:53:41 STEP 1469 H= 367:15 +CPU= 38.922
  07:54:16 STEP 1470 H= 367:30 +CPU= 34.259
  07:54:53 STEP 1471 H= 367:45 +CPU= 36.531
  07:55:33 STEP 1472 H= 368:00 +CPU= 39.410
  07:56:13 STEP 1473 H= 368:15 +CPU= 39.300
  07:56:51 STEP 1474 H= 368:30 +CPU= 37.348
  07:57:27 STEP 1475 H= 368:45 +CPU= 35.660
  07:58:09 STEP 1476 H= 369:00 +CPU= 41.270
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001476
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001476
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001476
  07:58:44 STEP 1477 H= 369:15 +CPU= 33.946
  07:59:22 STEP 1478 H= 369:30 +CPU= 37.663
  08:00:00 STEP 1479 H= 369:45 +CPU= 37.003
  08:00:37 STEP 1480 H= 370:00 +CPU= 36.397
  08:01:16 STEP 1481 H= 370:15 +CPU= 38.127
  08:01:54 STEP 1482 H= 370:30 +CPU= 37.083
  08:02:30 STEP 1483 H= 370:45 +CPU= 35.205
  08:03:07 STEP 1484 H= 371:00 +CPU= 36.345
  08:03:46 STEP 1485 H= 371:15 +CPU= 37.839
  08:04:21 STEP 1486 H= 371:30 +CPU= 35.216
  08:04:53 STEP 1487 H= 371:45 +CPU= 30.807
  08:05:44 STEP 1488 H= 372:00 +CPU= 49.530
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001488
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001488
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001488
  08:06:21 STEP 1489 H= 372:15 +CPU= 36.795
  08:07:00 STEP 1490 H= 372:30 +CPU= 38.296
  08:07:36 STEP 1491 H= 372:45 +CPU= 35.526
  08:08:12 STEP 1492 H= 373:00 +CPU= 35.393
  08:08:45 STEP 1493 H= 373:15 +CPU= 31.533
  08:09:19 STEP 1494 H= 373:30 +CPU= 33.976
  08:09:55 STEP 1495 H= 373:45 +CPU= 35.193
  08:10:34 STEP 1496 H= 374:00 +CPU= 37.609
  08:11:09 STEP 1497 H= 374:15 +CPU= 34.492
  08:11:46 STEP 1498 H= 374:30 +CPU= 36.989
  08:12:24 STEP 1499 H= 374:45 +CPU= 37.357
  08:13:10 STEP 1500 H= 375:00 +CPU= 44.894
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001500
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001500
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001500
  08:13:49 STEP 1501 H= 375:15 +CPU= 38.484
  08:14:28 STEP 1502 H= 375:30 +CPU= 38.143
  08:15:03 STEP 1503 H= 375:45 +CPU= 34.920
  08:15:41 STEP 1504 H= 376:00 +CPU= 36.684
  08:16:16 STEP 1505 H= 376:15 +CPU= 34.823
  08:16:53 STEP 1506 H= 376:30 +CPU= 35.884
  08:17:29 STEP 1507 H= 376:45 +CPU= 35.987
  08:18:08 STEP 1508 H= 377:00 +CPU= 38.031
  08:18:44 STEP 1509 H= 377:15 +CPU= 35.031
  08:19:20 STEP 1510 H= 377:30 +CPU= 35.017
  08:19:58 STEP 1511 H= 377:45 +CPU= 38.146
  08:20:43 STEP 1512 H= 378:00 +CPU= 44.228
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001512
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001512
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001512
  08:21:20 STEP 1513 H= 378:15 +CPU= 36.540
  08:22:00 STEP 1514 H= 378:30 +CPU= 38.275
  08:22:36 STEP 1515 H= 378:45 +CPU= 35.900
  08:23:14 STEP 1516 H= 379:00 +CPU= 37.156
  08:23:52 STEP 1517 H= 379:15 +CPU= 37.059
  08:24:28 STEP 1518 H= 379:30 +CPU= 35.516
  08:25:05 STEP 1519 H= 379:45 +CPU= 35.984
  08:25:41 STEP 1520 H= 380:00 +CPU= 35.497
  08:26:14 STEP 1521 H= 380:15 +CPU= 33.108
  08:26:53 STEP 1522 H= 380:30 +CPU= 37.626
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a334
(argv4) batchid: 1018
(argv5) wuid: 12292327
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/20
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/20/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/20/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_927d21112ca47581c32928c26db0ea5e to: /var/lib/boinc-client/slots/20/oifs_43r3_bl_a334_2016092300_20_1018_12292327.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/20/oifs_43r3_bl_a334_2016092300_20_1018_12292327.zip
ic_ancil_file: ic_ancil_12292327
ifsdata_file: ifsdata_12292327
climate_data_file: clim_data_12292327
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/20/ic_ancil_12292327.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/20/ic_ancil_12292327.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_f3f7b3d14422e5c8659ca8b5752e7ef3 to: /var/lib/boinc-client/slots/20/ifsdata/ifsdata_12292327.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/20/ifsdata/ifsdata_12292327.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/20/159l_2/clim_data_12292327.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/20/159l_2/clim_data_12292327.zip
Checking for progress XML file: /var/lib/boinc-client/slots/20/progress_file_12292327.xml
Opened progress file ok : /var/lib/boinc-client/slots/20/progress_file_12292327.xml
-- Model is restarting --
Adjusting last_iter, 1523, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/20/progress_file_12292327.xml
last_cpu_time: 59791
upload_file_number: 15
last_iter: 1489
last_upload: 1296000
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a334_2016092300_20_1018_12292327_0_r262561884
The child process has been launched with process id: 2513
Executing the command: /var/lib/boinc-client/slots/20/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:hzfinn-5950x:1:1:2513:2513] [20240614:082832:1718346512:0.001] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  08:29:20 STEP 1488 H= 372:00 +CPU= 38.501
  08:29:59 STEP 1489 H= 372:15 +CPU= 38.295
  08:30:45 STEP 1490 H= 372:30 +CPU= 45.151
  08:31:22 STEP 1491 H= 372:45 +CPU= 37.004
  08:32:07 STEP 1492 H= 373:00 +CPU= 44.345
  08:32:47 STEP 1493 H= 373:15 +CPU= 39.701
  08:33:26 STEP 1494 H= 373:30 +CPU= 38.877
  08:34:05 STEP 1495 H= 373:45 +CPU= 38.363
  08:34:44 STEP 1496 H= 374:00 +CPU= 38.956
  08:35:27 STEP 1497 H= 374:15 +CPU= 42.071
  08:36:06 STEP 1498 H= 374:30 +CPU= 38.476
  08:36:45 STEP 1499 H= 374:45 +CPU= 38.571
  08:37:37 STEP 1500 H= 375:00 +CPU= 52.364
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001500
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001500
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001500
  08:38:19 STEP 1501 H= 375:15 +CPU= 41.356
  08:39:03 STEP 1502 H= 375:30 +CPU= 44.049
  08:39:45 STEP 1503 H= 375:45 +CPU= 41.213
  08:40:25 STEP 1504 H= 376:00 +CPU= 40.116
  08:41:03 STEP 1505 H= 376:15 +CPU= 36.646
  08:41:48 STEP 1506 H= 376:30 +CPU= 44.761
  08:42:29 STEP 1507 H= 376:45 +CPU= 41.045
  08:43:12 STEP 1508 H= 377:00 +CPU= 42.224
  08:43:50 STEP 1509 H= 377:15 +CPU= 37.350
  08:44:35 STEP 1510 H= 377:30 +CPU= 44.774
  08:45:15 STEP 1511 H= 377:45 +CPU= 39.913
  08:46:06 STEP 1512 H= 378:00 +CPU= 50.136
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001512
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001512
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001512
  08:46:49 STEP 1513 H= 378:15 +CPU= 43.303
  08:47:32 STEP 1514 H= 378:30 +CPU= 42.600
  08:48:17 STEP 1515 H= 378:45 +CPU= 43.837
  08:49:03 STEP 1516 H= 379:00 +CPU= 46.336
  08:49:45 STEP 1517 H= 379:15 +CPU= 41.708
  08:50:25 STEP 1518 H= 379:30 +CPU= 39.425
  08:51:07 STEP 1519 H= 379:45 +CPU= 41.110
  08:51:49 STEP 1520 H= 380:00 +CPU= 41.502
  08:52:32 STEP 1521 H= 380:15 +CPU= 42.505
  08:53:12 STEP 1522 H= 380:30 +CPU= 40.375
  08:53:53 STEP 1523 H= 380:45 +CPU= 39.999
  08:54:43 STEP 1524 H= 381:00 +CPU= 49.316
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001524
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001524
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001524
  08:55:29 STEP 1525 H= 381:15 +CPU= 45.426
  08:56:07 STEP 1526 H= 381:30 +CPU= 37.727
  08:56:48 STEP 1527 H= 381:45 +CPU= 41.447
  08:57:32 STEP 1528 H= 382:00 +CPU= 42.989
  08:58:15 STEP 1529 H= 382:15 +CPU= 42.888
  08:58:54 STEP 1530 H= 382:30 +CPU= 38.470
  08:59:35 STEP 1531 H= 382:45 +CPU= 40.508
  09:00:16 STEP 1532 H= 383:00 +CPU= 40.208
  09:00:55 STEP 1533 H= 383:15 +CPU= 39.434
  09:01:34 STEP 1534 H= 383:30 +CPU= 37.947
  09:02:15 STEP 1535 H= 383:45 +CPU= 41.339
  09:03:07 STEP 1536 H= 384:00 +CPU= 51.475
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001536
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001536
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001524
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a334_2016092300_20_1018_12292327_0_r262561884_15.zip
Uploading the intermediate file: upload_file_15.zip
  09:03:47 STEP 1537 H= 384:15 +CPU= 38.510
  09:04:29 STEP 1538 H= 384:30 +CPU= 41.482
  09:05:13 STEP 1539 H= 384:45 +CPU= 43.669
  09:05:52 STEP 1540 H= 385:00 +CPU= 39.113
  09:06:35 STEP 1541 H= 385:15 +CPU= 42.189
  09:07:13 STEP 1542 H= 385:30 +CPU= 38.166
  09:07:55 STEP 1543 H= 385:45 +CPU= 41.708
  09:08:41 STEP 1544 H= 386:00 +CPU= 45.030
  09:09:24 STEP 1545 H= 386:15 +CPU= 42.378
  09:10:10 STEP 1546 H= 386:30 +CPU= 45.717
  09:10:52 STEP 1547 H= 386:45 +CPU= 41.456
  09:11:43 STEP 1548 H= 387:00 +CPU= 50.981
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001548
  09:12:24 STEP 1549 H= 387:15 +CPU= 40.205
  09:13:06 STEP 1550 H= 387:30 +CPU= 42.146
  09:13:48 STEP 1551 H= 387:45 +CPU= 41.780
  09:14:30 STEP 1552 H= 388:00 +CPU= 41.494
  09:15:08 STEP 1553 H= 388:15 +CPU= 37.742
  09:15:51 STEP 1554 H= 388:30 +CPU= 42.068
  09:16:32 STEP 1555 H= 388:45 +CPU= 40.691
  09:17:11 STEP 1556 H= 389:00 +CPU= 38.747
  09:17:52 STEP 1557 H= 389:15 +CPU= 40.284
  09:18:34 STEP 1558 H= 389:30 +CPU= 41.746
  09:19:13 STEP 1559 H= 389:45 +CPU= 38.505
  09:19:59 STEP 1560 H= 390:00 +CPU= 45.995
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001560
  09:20:39 STEP 1561 H= 390:15 +CPU= 38.804
  09:21:19 STEP 1562 H= 390:30 +CPU= 40.112
  09:22:01 STEP 1563 H= 390:45 +CPU= 41.346
  09:22:40 STEP 1564 H= 391:00 +CPU= 38.991
  09:23:21 STEP 1565 H= 391:15 +CPU= 40.187
  09:24:05 STEP 1566 H= 391:30 +CPU= 44.258
  09:24:47 STEP 1567 H= 391:45 +CPU= 41.143
  09:25:30 STEP 1568 H= 392:00 +CPU= 42.989
  09:26:11 STEP 1569 H= 392:15 +CPU= 40.508
  09:26:55 STEP 1570 H= 392:30 +CPU= 43.719
  09:27:38 STEP 1571 H= 392:45 +CPU= 42.505
  09:28:27 STEP 1572 H= 393:00 +CPU= 48.872
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001572
  09:29:11 STEP 1573 H= 393:15 +CPU= 43.561
  09:29:53 STEP 1574 H= 393:30 +CPU= 41.531
  09:30:36 STEP 1575 H= 393:45 +CPU= 42.313
  09:31:20 STEP 1576 H= 394:00 +CPU= 43.494
  09:32:04 STEP 1577 H= 394:15 +CPU= 43.269
  09:32:42 STEP 1578 H= 394:30 +CPU= 37.793
  09:33:27 STEP 1579 H= 394:45 +CPU= 44.273
  09:34:09 STEP 1580 H= 395:00 +CPU= 41.852
  09:34:53 STEP 1581 H= 395:15 +CPU= 44.173
  09:35:37 STEP 1582 H= 395:30 +CPU= 43.361
  09:36:14 STEP 1583 H= 395:45 +CPU= 36.276
  09:37:10 STEP 1584 H= 396:00 +CPU= 55.484
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001584
  09:37:50 STEP 1585 H= 396:15 +CPU= 39.735
  09:38:25 STEP 1586 H= 396:30 +CPU= 34.893
  09:39:09 STEP 1587 H= 396:45 +CPU= 43.049
  09:39:49 STEP 1588 H= 397:00 +CPU= 39.687
  09:40:26 STEP 1589 H= 397:15 +CPU= 36.639
  09:41:05 STEP 1590 H= 397:30 +CPU= 38.698
  09:41:49 STEP 1591 H= 397:45 +CPU= 42.783
  09:42:31 STEP 1592 H= 398:00 +CPU= 41.975
  09:43:12 STEP 1593 H= 398:15 +CPU= 40.862
  09:43:55 STEP 1594 H= 398:30 +CPU= 41.671
  09:44:34 STEP 1595 H= 398:45 +CPU= 39.056
  09:45:26 STEP 1596 H= 399:00 +CPU= 51.196
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001596
  09:46:11 STEP 1597 H= 399:15 +CPU= 44.961
  09:46:53 STEP 1598 H= 399:30 +CPU= 41.766
  09:47:35 STEP 1599 H= 399:45 +CPU= 41.498
  09:48:15 STEP 1600 H= 400:00 +CPU= 39.364
  09:48:56 STEP 1601 H= 400:15 +CPU= 40.954
  09:49:40 STEP 1602 H= 400:30 +CPU= 43.079
  09:50:18 STEP 1603 H= 400:45 +CPU= 37.407
  09:51:00 STEP 1604 H= 401:00 +CPU= 42.051
  09:51:43 STEP 1605 H= 401:15 +CPU= 41.797
  09:52:23 STEP 1606 H= 401:30 +CPU= 40.532
  09:52:52 STEP 1607 H= 401:45 +CPU= 28.823
  09:53:42 STEP 1608 H= 402:00 +CPU= 49.157
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001608
  09:54:22 STEP 1609 H= 402:15 +CPU= 39.918
  09:55:07 STEP 1610 H= 402:30 +CPU= 44.398
  09:55:50 STEP 1611 H= 402:45 +CPU= 42.628
  09:56:34 STEP 1612 H= 403:00 +CPU= 43.341
  09:57:17 STEP 1613 H= 403:15 +CPU= 43.178
  09:57:59 STEP 1614 H= 403:30 +CPU= 40.962
  09:58:41 STEP 1615 H= 403:45 +CPU= 41.898
  09:59:27 STEP 1616 H= 404:00 +CPU= 45.160
  10:00:06 STEP 1617 H= 404:15 +CPU= 39.048
  10:00:49 STEP 1618 H= 404:30 +CPU= 42.025
  10:01:28 STEP 1619 H= 404:45 +CPU= 38.822
  10:02:21 STEP 1620 H= 405:00 +CPU= 52.793
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001620
  10:03:01 STEP 1621 H= 405:15 +CPU= 39.293
  10:03:41 STEP 1622 H= 405:30 +CPU= 39.775
  10:04:23 STEP 1623 H= 405:45 +CPU= 41.686
  10:05:04 STEP 1624 H= 406:00 +CPU= 40.961
  10:05:44 STEP 1625 H= 406:15 +CPU= 38.585
  10:06:25 STEP 1626 H= 406:30 +CPU= 41.317
  10:07:04 STEP 1627 H= 406:45 +CPU= 38.754
  10:07:51 STEP 1628 H= 407:00 +CPU= 46.105
  10:08:29 STEP 1629 H= 407:15 +CPU= 38.007
  10:09:07 STEP 1630 H= 407:30 +CPU= 37.805
  10:09:47 STEP 1631 H= 407:45 +CPU= 38.540
  10:10:45 STEP 1632 H= 408:00 +CPU= 57.732
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001620
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a334_2016092300_20_1018_12292327_0_r262561884_16.zip
Uploading the intermediate file: upload_file_16.zip
  10:11:24 STEP 1633 H= 408:15 +CPU= 38.964
  10:12:08 STEP 1634 H= 408:30 +CPU= 42.909
  10:12:47 STEP 1635 H= 408:45 +CPU= 38.982
  10:13:27 STEP 1636 H= 409:00 +CPU= 39.797
  10:14:09 STEP 1637 H= 409:15 +CPU= 41.516
  10:14:52 STEP 1638 H= 409:30 +CPU= 42.354
  10:15:37 STEP 1639 H= 409:45 +CPU= 44.435
  10:16:20 STEP 1640 H= 410:00 +CPU= 42.614
  10:17:05 STEP 1641 H= 410:15 +CPU= 44.372
  10:17:45 STEP 1642 H= 410:30 +CPU= 40.030
  10:18:26 STEP 1643 H= 410:45 +CPU= 40.366
  10:19:18 STEP 1644 H= 411:00 +CPU= 50.977
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001644
  10:20:02 STEP 1645 H= 411:15 +CPU= 44.018
  10:20:42 STEP 1646 H= 411:30 +CPU= 39.356
  10:21:24 STEP 1647 H= 411:45 +CPU= 41.844
  10:22:07 STEP 1648 H= 412:00 +CPU= 42.611
  10:22:48 STEP 1649 H= 412:15 +CPU= 40.377
  10:23:29 STEP 1650 H= 412:30 +CPU= 41.027
  10:24:12 STEP 1651 H= 412:45 +CPU= 42.719
  10:24:55 STEP 1652 H= 413:00 +CPU= 42.205
  10:25:40 STEP 1653 H= 413:15 +CPU= 44.478
  10:26:20 STEP 1654 H= 413:30 +CPU= 40.264
  10:27:02 STEP 1655 H= 413:45 +CPU= 40.854
  10:27:55 STEP 1656 H= 414:00 +CPU= 52.434
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001656
  10:28:37 STEP 1657 H= 414:15 +CPU= 42.014
  10:29:16 STEP 1658 H= 414:30 +CPU= 38.336
  10:29:59 STEP 1659 H= 414:45 +CPU= 43.370
  10:30:39 STEP 1660 H= 415:00 +CPU= 39.263
  10:31:20 STEP 1661 H= 415:15 +CPU= 40.732
  10:32:05 STEP 1662 H= 415:30 +CPU= 44.917
  10:32:45 STEP 1663 H= 415:45 +CPU= 39.581
  10:33:30 STEP 1664 H= 416:00 +CPU= 44.736
  10:34:12 STEP 1665 H= 416:15 +CPU= 40.705
  10:34:52 STEP 1666 H= 416:30 +CPU= 40.092
  10:35:32 STEP 1667 H= 416:45 +CPU= 39.606
  10:36:20 STEP 1668 H= 417:00 +CPU= 47.821
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001668
  10:37:05 STEP 1669 H= 417:15 +CPU= 43.981
  10:37:46 STEP 1670 H= 417:30 +CPU= 40.722
  10:38:29 STEP 1671 H= 417:45 +CPU= 42.758
  10:39:08 STEP 1672 H= 418:00 +CPU= 38.569
  10:39:49 STEP 1673 H= 418:15 +CPU= 40.357
  10:40:28 STEP 1674 H= 418:30 +CPU= 39.053
  10:41:08 STEP 1675 H= 418:45 +CPU= 39.526
  10:41:52 STEP 1676 H= 419:00 +CPU= 44.135
  10:42:33 STEP 1677 H= 419:15 +CPU= 39.896
  10:43:14 STEP 1678 H= 419:30 +CPU= 40.674
  10:43:57 STEP 1679 H= 419:45 +CPU= 42.520
  10:44:52 STEP 1680 H= 420:00 +CPU= 54.905
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001680
  10:45:33 STEP 1681 H= 420:15 +CPU= 40.250
  10:46:13 STEP 1682 H= 420:30 +CPU= 39.696
  10:46:55 STEP 1683 H= 420:45 +CPU= 41.243
  10:47:39 STEP 1684 H= 421:00 +CPU= 43.389
  10:48:17 STEP 1685 H= 421:15 +CPU= 38.180
  10:48:55 STEP 1686 H= 421:30 +CPU= 37.532
  10:49:37 STEP 1687 H= 421:45 +CPU= 41.053
  10:50:19 STEP 1688 H= 422:00 +CPU= 42.027
  10:51:02 STEP 1689 H= 422:15 +CPU= 42.421
  10:51:45 STEP 1690 H= 422:30 +CPU= 42.712
  10:52:22 STEP 1691 H= 422:45 +CPU= 36.011
  10:53:14 STEP 1692 H= 423:00 +CPU= 51.663
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001692
  10:53:57 STEP 1693 H= 423:15 +CPU= 42.998
  10:54:39 STEP 1694 H= 423:30 +CPU= 41.379
  10:55:22 STEP 1695 H= 423:45 +CPU= 42.780
  10:56:01 STEP 1696 H= 424:00 +CPU= 38.121
  10:56:40 STEP 1697 H= 424:15 +CPU= 38.661
  10:57:23 STEP 1698 H= 424:30 +CPU= 42.459
  10:58:02 STEP 1699 H= 424:45 +CPU= 38.804
  10:58:44 STEP 1700 H= 425:00 +CPU= 41.631
  10:59:22 STEP 1701 H= 425:15 +CPU= 38.215
  11:00:05 STEP 1702 H= 425:30 +CPU= 41.622
  11:00:45 STEP 1703 H= 425:45 +CPU= 39.864
  11:01:36 STEP 1704 H= 426:00 +CPU= 50.901
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001704
  11:02:19 STEP 1705 H= 426:15 +CPU= 42.332
  11:03:03 STEP 1706 H= 426:30 +CPU= 43.028
  11:03:47 STEP 1707 H= 426:45 +CPU= 44.311
  11:04:29 STEP 1708 H= 427:00 +CPU= 41.450
  11:05:11 STEP 1709 H= 427:15 +CPU= 41.797
  11:05:52 STEP 1710 H= 427:30 +CPU= 40.121
  11:06:35 STEP 1711 H= 427:45 +CPU= 42.521
  11:07:16 STEP 1712 H= 428:00 +CPU= 41.336
  11:08:00 STEP 1713 H= 428:15 +CPU= 43.141
  11:08:36 STEP 1714 H= 428:30 +CPU= 36.197
  11:09:21 STEP 1715 H= 428:45 +CPU= 44.397
  11:10:11 STEP 1716 H= 429:00 +CPU= 48.912
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001716
  11:10:53 STEP 1717 H= 429:15 +CPU= 42.131
  11:11:30 STEP 1718 H= 429:30 +CPU= 36.568
  11:12:07 STEP 1719 H= 429:45 +CPU= 36.052
  11:12:47 STEP 1720 H= 430:00 +CPU= 39.920
  11:13:32 STEP 1721 H= 430:15 +CPU= 44.358
  11:14:12 STEP 1722 H= 430:30 +CPU= 40.304
  11:14:56 STEP 1723 H= 430:45 +CPU= 42.966
  11:15:35 STEP 1724 H= 431:00 +CPU= 39.334
  11:16:19 STEP 1725 H= 431:15 +CPU= 42.893
  11:16:59 STEP 1726 H= 431:30 +CPU= 39.457
  11:17:35 STEP 1727 H= 431:45 +CPU= 36.230
  11:18:37 STEP 1728 H= 432:00 +CPU= 61.250
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001716
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a334_2016092300_20_1018_12292327_0_r262561884_17.zip
Uploading the intermediate file: upload_file_17.zip
  11:19:17 STEP 1729 H= 432:15 +CPU= 39.658
  11:20:02 STEP 1730 H= 432:30 +CPU= 44.012
  11:20:42 STEP 1731 H= 432:45 +CPU= 39.484
  11:21:26 STEP 1732 H= 433:00 +CPU= 44.173
  11:22:07 STEP 1733 H= 433:15 +CPU= 40.701
  11:22:50 STEP 1734 H= 433:30 +CPU= 42.471
  11:23:29 STEP 1735 H= 433:45 +CPU= 39.077
  11:24:12 STEP 1736 H= 434:00 +CPU= 41.876
  11:24:54 STEP 1737 H= 434:15 +CPU= 42.072
  11:25:38 STEP 1738 H= 434:30 +CPU= 43.371
  11:26:17 STEP 1739 H= 434:45 +CPU= 38.509
  11:27:08 STEP 1740 H= 435:00 +CPU= 50.309
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001740
  11:27:51 STEP 1741 H= 435:15 +CPU= 42.703
  11:28:32 STEP 1742 H= 435:30 +CPU= 41.275
  11:29:12 STEP 1743 H= 435:45 +CPU= 39.314
  11:29:56 STEP 1744 H= 436:00 +CPU= 43.334
  11:30:38 STEP 1745 H= 436:15 +CPU= 41.353
  11:31:22 STEP 1746 H= 436:30 +CPU= 43.353
  11:32:01 STEP 1747 H= 436:45 +CPU= 39.175
  11:32:45 STEP 1748 H= 437:00 +CPU= 43.074
  11:33:27 STEP 1749 H= 437:15 +CPU= 42.025
  11:34:06 STEP 1750 H= 437:30 +CPU= 38.706
  11:34:48 STEP 1751 H= 437:45 +CPU= 41.399
  11:35:37 STEP 1752 H= 438:00 +CPU= 48.486
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001752
  11:36:21 STEP 1753 H= 438:15 +CPU= 43.644
  11:37:06 STEP 1754 H= 438:30 +CPU= 44.048
  11:37:50 STEP 1755 H= 438:45 +CPU= 44.061
  11:38:34 STEP 1756 H= 439:00 +CPU= 43.772
  11:39:18 STEP 1757 H= 439:15 +CPU= 43.010
  11:39:58 STEP 1758 H= 439:30 +CPU= 39.515
  11:40:38 STEP 1759 H= 439:45 +CPU= 39.279
  11:41:21 STEP 1760 H= 440:00 +CPU= 43.122
  11:42:03 STEP 1761 H= 440:15 +CPU= 41.063
  11:42:38 STEP 1762 H= 440:30 +CPU= 34.835
  11:43:21 STEP 1763 H= 440:45 +CPU= 41.999
  11:44:11 STEP 1764 H= 441:00 +CPU= 49.499
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001764
  11:44:50 STEP 1765 H= 441:15 +CPU= 39.087
  11:45:32 STEP 1766 H= 441:30 +CPU= 41.423
  11:46:16 STEP 1767 H= 441:45 +CPU= 43.645
  11:47:01 STEP 1768 H= 442:00 +CPU= 44.068
  11:47:40 STEP 1769 H= 442:15 +CPU= 38.754
  11:48:24 STEP 1770 H= 442:30 +CPU= 43.514
  11:49:04 STEP 1771 H= 442:45 +CPU= 39.432
  11:49:46 STEP 1772 H= 443:00 +CPU= 41.865
  11:50:24 STEP 1773 H= 443:15 +CPU= 38.212
  11:51:04 STEP 1774 H= 443:30 +CPU= 39.566
  11:51:44 STEP 1775 H= 443:45 +CPU= 39.372
  11:52:36 STEP 1776 H= 444:00 +CPU= 51.327
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001776
  11:53:19 STEP 1777 H= 444:15 +CPU= 42.525
  11:54:00 STEP 1778 H= 444:30 +CPU= 40.182
  11:54:45 STEP 1779 H= 444:45 +CPU= 44.351
  11:55:27 STEP 1780 H= 445:00 +CPU= 42.022
  11:56:10 STEP 1781 H= 445:15 +CPU= 41.632
  11:56:49 STEP 1782 H= 445:30 +CPU= 39.490
  11:57:29 STEP 1783 H= 445:45 +CPU= 39.227
  11:58:09 STEP 1784 H= 446:00 +CPU= 39.328
  11:58:51 STEP 1785 H= 446:15 +CPU= 41.451
  11:59:34 STEP 1786 H= 446:30 +CPU= 42.010
  12:00:16 STEP 1787 H= 446:45 +CPU= 42.437
  12:01:06 STEP 1788 H= 447:00 +CPU= 48.903
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001788
  12:01:42 STEP 1789 H= 447:15 +CPU= 35.970
  12:02:21 STEP 1790 H= 447:30 +CPU= 38.149
  12:03:01 STEP 1791 H= 447:45 +CPU= 40.282
  12:03:43 STEP 1792 H= 448:00 +CPU= 40.596
  12:04:23 STEP 1793 H= 448:15 +CPU= 40.495
  12:05:08 STEP 1794 H= 448:30 +CPU= 44.255
  12:05:46 STEP 1795 H= 448:45 +CPU= 37.266
  12:06:29 STEP 1796 H= 449:00 +CPU= 42.645
  12:07:08 STEP 1797 H= 449:15 +CPU= 38.373
  12:07:45 STEP 1798 H= 449:30 +CPU= 36.819
  12:08:27 STEP 1799 H= 449:45 +CPU= 41.765
  12:09:20 STEP 1800 H= 450:00 +CPU= 52.561
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001800
  12:09:59 STEP 1801 H= 450:15 +CPU= 38.528
  12:10:42 STEP 1802 H= 450:30 +CPU= 42.726
  12:11:23 STEP 1803 H= 450:45 +CPU= 40.313
  12:12:05 STEP 1804 H= 451:00 +CPU= 42.072
  12:12:50 STEP 1805 H= 451:15 +CPU= 43.707
  12:13:34 STEP 1806 H= 451:30 +CPU= 44.080
  12:14:16 STEP 1807 H= 451:45 +CPU= 40.947
  12:15:00 STEP 1808 H= 452:00 +CPU= 44.230
  12:15:45 STEP 1809 H= 452:15 +CPU= 44.078
  12:16:25 STEP 1810 H= 452:30 +CPU= 39.626
  12:17:07 STEP 1811 H= 452:45 +CPU= 41.305
  12:17:55 STEP 1812 H= 453:00 +CPU= 48.119
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001812
  12:18:35 STEP 1813 H= 453:15 +CPU= 39.395
  12:19:17 STEP 1814 H= 453:30 +CPU= 40.769
  12:19:56 STEP 1815 H= 453:45 +CPU= 39.140
  12:20:42 STEP 1816 H= 454:00 +CPU= 45.580
  12:21:20 STEP 1817 H= 454:15 +CPU= 37.822
  12:21:57 STEP 1818 H= 454:30 +CPU= 36.094
  12:22:34 STEP 1819 H= 454:45 +CPU= 36.709
  12:23:19 STEP 1820 H= 455:00 +CPU= 43.859
  12:23:59 STEP 1821 H= 455:15 +CPU= 39.186
  12:24:35 STEP 1822 H= 455:30 +CPU= 35.886
  12:25:19 STEP 1823 H= 455:45 +CPU= 43.048
  12:26:16 STEP 1824 H= 456:00 +CPU= 56.622
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001812
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a334_2016092300_20_1018_12292327_0_r262561884_18.zip
Uploading the intermediate file: upload_file_18.zip
  12:26:53 STEP 1825 H= 456:15 +CPU= 36.633
  12:27:33 STEP 1826 H= 456:30 +CPU= 39.889
  12:28:15 STEP 1827 H= 456:45 +CPU= 41.184
  12:28:58 STEP 1828 H= 457:00 +CPU= 42.497
  12:29:38 STEP 1829 H= 457:15 +CPU= 39.587
  12:30:22 STEP 1830 H= 457:30 +CPU= 43.729
  12:31:05 STEP 1831 H= 457:45 +CPU= 42.337
  12:31:48 STEP 1832 H= 458:00 +CPU= 42.948
  12:32:32 STEP 1833 H= 458:15 +CPU= 43.376
  12:33:14 STEP 1834 H= 458:30 +CPU= 41.012
  12:33:51 STEP 1835 H= 458:45 +CPU= 37.171
  12:34:40 STEP 1836 H= 459:00 +CPU= 47.629
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001836
  12:35:17 STEP 1837 H= 459:15 +CPU= 37.435
  12:35:58 STEP 1838 H= 459:30 +CPU= 40.249
  12:36:37 STEP 1839 H= 459:45 +CPU= 38.347
  12:37:19 STEP 1840 H= 460:00 +CPU= 41.305
  12:38:02 STEP 1841 H= 460:15 +CPU= 43.148
  12:38:47 STEP 1842 H= 460:30 +CPU= 44.224
  12:39:28 STEP 1843 H= 460:45 +CPU= 40.346
  12:40:08 STEP 1844 H= 461:00 +CPU= 39.834
  12:40:45 STEP 1845 H= 461:15 +CPU= 36.537
  12:41:28 STEP 1846 H= 461:30 +CPU= 42.839
  12:42:10 STEP 1847 H= 461:45 +CPU= 40.924
  12:42:56 STEP 1848 H= 462:00 +CPU= 46.081
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001848
  12:43:39 STEP 1849 H= 462:15 +CPU= 42.716
  12:44:22 STEP 1850 H= 462:30 +CPU= 41.831
  12:45:04 STEP 1851 H= 462:45 +CPU= 41.884
  12:45:50 STEP 1852 H= 463:00 +CPU= 45.321
  12:46:35 STEP 1853 H= 463:15 +CPU= 45.261
  12:47:21 STEP 1854 H= 463:30 +CPU= 45.122
  12:48:02 STEP 1855 H= 463:45 +CPU= 40.942
  12:48:44 STEP 1856 H= 464:00 +CPU= 41.009
  12:49:25 STEP 1857 H= 464:15 +CPU= 40.913
  12:50:10 STEP 1858 H= 464:30 +CPU= 43.967
  12:50:55 STEP 1859 H= 464:45 +CPU= 45.011
  12:51:46 STEP 1860 H= 465:00 +CPU= 50.009
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001860
  12:52:28 STEP 1861 H= 465:15 +CPU= 42.113
  12:53:09 STEP 1862 H= 465:30 +CPU= 40.343
  12:53:48 STEP 1863 H= 465:45 +CPU= 38.437
  12:54:29 STEP 1864 H= 466:00 +CPU= 40.919
  12:55:09 STEP 1865 H= 466:15 +CPU= 39.422
  12:55:50 STEP 1866 H= 466:30 +CPU= 40.757
  12:56:32 STEP 1867 H= 466:45 +CPU= 42.066
  12:57:16 STEP 1868 H= 467:00 +CPU= 42.840
  12:57:55 STEP 1869 H= 467:15 +CPU= 38.089
  12:58:33 STEP 1870 H= 467:30 +CPU= 37.958
  12:59:15 STEP 1871 H= 467:45 +CPU= 41.545
  13:00:09 STEP 1872 H= 468:00 +CPU= 53.287
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001872
  13:00:50 STEP 1873 H= 468:15 +CPU= 40.193
  13:01:29 STEP 1874 H= 468:30 +CPU= 39.190
  13:02:12 STEP 1875 H= 468:45 +CPU= 42.048
  13:02:55 STEP 1876 H= 469:00 +CPU= 42.376
  13:03:36 STEP 1877 H= 469:15 +CPU= 41.189
  13:04:17 STEP 1878 H= 469:30 +CPU= 39.663
  13:04:59 STEP 1879 H= 469:45 +CPU= 41.740
  13:05:40 STEP 1880 H= 470:00 +CPU= 41.067
  13:06:20 STEP 1881 H= 470:15 +CPU= 39.836
  13:06:59 STEP 1882 H= 470:30 +CPU= 37.699
  13:07:39 STEP 1883 H= 470:45 +CPU= 39.802
  13:08:25 STEP 1884 H= 471:00 +CPU= 45.960
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001884
  13:09:05 STEP 1885 H= 471:15 +CPU= 39.279
  13:09:45 STEP 1886 H= 471:30 +CPU= 38.787
  13:10:23 STEP 1887 H= 471:45 +CPU= 38.110
  13:11:06 STEP 1888 H= 472:00 +CPU= 42.384
  13:11:47 STEP 1889 H= 472:15 +CPU= 40.469
  13:12:30 STEP 1890 H= 472:30 +CPU= 42.771
  13:13:10 STEP 1891 H= 472:45 +CPU= 39.503
  13:13:50 STEP 1892 H= 473:00 +CPU= 39.683
  13:14:29 STEP 1893 H= 473:15 +CPU= 38.840
  13:15:10 STEP 1894 H= 473:30 +CPU= 40.477
  13:15:51 STEP 1895 H= 473:45 +CPU= 40.696
  13:16:42 STEP 1896 H= 474:00 +CPU= 50.102
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001896
  13:17:19 STEP 1897 H= 474:15 +CPU= 36.955
  13:18:00 STEP 1898 H= 474:30 +CPU= 39.942
  13:18:46 STEP 1899 H= 474:45 +CPU= 46.023
  13:19:27 STEP 1900 H= 475:00 +CPU= 39.960
  13:20:05 STEP 1901 H= 475:15 +CPU= 37.877
  13:20:44 STEP 1902 H= 475:30 +CPU= 39.147
  13:21:28 STEP 1903 H= 475:45 +CPU= 42.718
  13:22:13 STEP 1904 H= 476:00 +CPU= 45.264
  13:22:57 STEP 1905 H= 476:15 +CPU= 43.499
  13:23:36 STEP 1906 H= 476:30 +CPU= 37.883
  13:24:16 STEP 1907 H= 476:45 +CPU= 40.429
  13:25:09 STEP 1908 H= 477:00 +CPU= 51.713
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001908
  13:25:50 STEP 1909 H= 477:15 +CPU= 41.210
  13:26:34 STEP 1910 H= 477:30 +CPU= 42.774
  13:27:20 STEP 1911 H= 477:45 +CPU= 46.039
  13:28:00 STEP 1912 H= 478:00 +CPU= 39.119
  13:28:42 STEP 1913 H= 478:15 +CPU= 41.382
  13:29:21 STEP 1914 H= 478:30 +CPU= 38.819
  13:30:00 STEP 1915 H= 478:45 +CPU= 39.053
  13:30:38 STEP 1916 H= 479:00 +CPU= 37.266
  13:31:17 STEP 1917 H= 479:15 +CPU= 38.128
  13:31:56 STEP 1918 H= 479:30 +CPU= 39.058
  13:32:38 STEP 1919 H= 479:45 +CPU= 41.658
  13:33:30 STEP 1920 H= 480:00 +CPU= 50.626
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  480.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I      0.4291972677E-21      I       0.2003701895E-06      I
ICloud fractionI      0.0000000000E+00      I       0.6142217142E-02      I
I  Relat. Hum. I      0.2533885585E+00      I       0.2886515580E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.1169077109E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    316622148 BYTES IN      7 RECORDS TRANSFERRED IN    0.0007 SECONDS452317.4 Mbytes/s, TOTAL TIME=    0.0125( 5.6%)
 13:33:30 STEP 1920 H= 480:00 +CPU= 50.626
 END CNT3
 NSTEP =  1920 CNT0   000000000  36.790  36.790  37.298   0.986               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN        12870/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    228    0       46      1044068    12870 ************    41.9     4.1    37.9     6.7    71.0
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********     99.94      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  469    8733.5    8733.5     22.38      0.00
   9 SPCM     - SPECTRAL COMP.                     432    1896.0    1896.0      4.48      0.00
  10 SCAN2M - PHYSICS                              433   16623.3   16623.3     39.34      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 36    1624.3    1624.3      0.32      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 19      96.6      96.6      0.01      0.00
  17 GRIDFPOS IN CNT4                               36      45.6      45.6      0.01      0.00
  27 RESTART FILES - WRITING                         9    6497.2    6497.2      0.32      0.00
  28 RESTART FILES - READING                         1    9374.8    9374.8      0.05      0.00
  29 SU4FPOS IN CNT4                                36       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                36    7438.0    7438.0      1.46      0.00
  31 POSDDH IN STEPO                                 2       1.6       1.6      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    433    8840.7    8840.7     20.92      0.00
  39 SU0YOMB                                         1     483.0     483.0      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    433      42.2      42.2      0.10      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              432     921.8     921.8      2.18      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              432     116.9     116.9      0.28      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 19       0.7       0.7      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      940     583.1     583.1      3.00      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       900     318.6     318.6      1.57      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        900     129.7     129.7      0.64      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       940     184.4     184.4      0.95      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      39.2      39.2      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            940     139.4     139.4      0.72      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            900      80.7      80.7      0.40      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            940     153.0     153.0      0.79      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            900      77.4      77.4      0.38      0.00
 400 GSTATS                                      51056       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 47450       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    18298.1 CPU TIME   18109.5 VECTOR TIME    18109.5


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY       94.5
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/slots/20/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/20/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMUAh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMSHh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292327/ICMGGh7zg+001860
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a334_2016092300_20_1018_12292327_0_r262561884_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
13:36:59 (2184): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
14 Jun 2024 11:35:27 1549573 22445933 oifs_43r3_bl_a334_2016092300_20_1018_12292327_0 1,727,100 77,900 0.0451
14 Jun 2024 00:53:00 1549573 22445933 oifs_43r3_bl_a334_2016092300_20_1018_12292327_0 864,000 38,518 0.0446


©2024 climateprediction.net