climateprediction.net home page
Task 22278224

Task 22278224

Name oifs_43r3_ps_0839_2006050100_123_975_12192483_0
Workunit 12192483
Created 23 Dec 2022, 14:13:19 UTC
Sent 19 Jan 2023, 16:27:29 UTC
Report deadline 20 Mar 2023, 16:27:29 UTC
Received 25 Jan 2023, 6:37:57 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1478188
Run time 1 days 3 hours 25 min 2 sec
CPU time 1 days 3 hours 0 min 39 sec
Validate state Invalid
Credit 13,426.56
Device peak FLOPS 4.20 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,403.37 MB
Peak swap size 4,978.08 MB
Peak disk usage 7,168.77 MB
Stderr
<core_client_version>7.20.2</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
PU= 19.996
  07:12:40 STEP 2340 H=2340:00 +CPU= 35.476
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002340
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002340
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002340
  07:13:01 STEP 2341 H=2341:00 +CPU= 20.214
  07:13:22 STEP 2342 H=2342:00 +CPU= 21.306
  07:13:55 STEP 2343 H=2343:00 +CPU= 32.466
  07:14:16 STEP 2344 H=2344:00 +CPU= 20.866
  07:14:39 STEP 2345 H=2345:00 +CPU= 22.885
  07:15:26 STEP 2346 H=2346:00 +CPU= 46.397
  07:15:55 STEP 2347 H=2347:00 +CPU= 28.182
  07:16:22 STEP 2348 H=2348:00 +CPU= 27.277
  07:17:09 STEP 2349 H=2349:00 +CPU= 46.564
  07:17:38 STEP 2350 H=2350:00 +CPU= 28.116
  07:18:06 STEP 2351 H=2351:00 +CPU= 27.919
  07:18:57 STEP 2352 H=2352:00 +CPU= 50.401
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002352
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002352
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002352
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002328
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002328
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002328
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002340
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002340
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002340
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0839_2006050100_123_975_12192483_0_r1131672709_97.zip
Uploading the intermediate file: upload_file_97.zip
  07:19:24 STEP 2353 H=2353:00 +CPU= 26.082
  07:19:50 STEP 2354 H=2354:00 +CPU= 25.572
  07:20:32 STEP 2355 H=2355:00 +CPU= 42.130
  07:21:00 STEP 2356 H=2356:00 +CPU= 27.412
  07:21:28 STEP 2357 H=2357:00 +CPU= 27.684
  07:22:14 STEP 2358 H=2358:00 +CPU= 45.235
  07:22:42 STEP 2359 H=2359:00 +CPU= 28.059
  07:23:10 STEP 2360 H=2360:00 +CPU= 27.539
  07:23:49 STEP 2361 H=2361:00 +CPU= 38.586
  07:24:16 STEP 2362 H=2362:00 +CPU= 26.971
  07:24:45 STEP 2363 H=2363:00 +CPU= 28.183
  07:25:29 STEP 2364 H=2364:00 +CPU= 43.921
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002364
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002364
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002364
  07:25:57 STEP 2365 H=2365:00 +CPU= 27.661
  07:26:26 STEP 2366 H=2366:00 +CPU= 28.064
  07:27:07 STEP 2367 H=2367:00 +CPU= 41.104
  07:27:35 STEP 2368 H=2368:00 +CPU= 27.056
  07:28:00 STEP 2369 H=2369:00 +CPU= 25.516
  07:28:45 STEP 2370 H=2370:00 +CPU= 44.337
  07:29:13 STEP 2371 H=2371:00 +CPU= 27.347
  07:29:43 STEP 2372 H=2372:00 +CPU= 29.769
  07:30:26 STEP 2373 H=2373:00 +CPU= 42.975
  07:30:54 STEP 2374 H=2374:00 +CPU= 27.568
  07:31:23 STEP 2375 H=2375:00 +CPU= 28.159
  07:32:16 STEP 2376 H=2376:00 +CPU= 52.595
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002376
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002376
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002376
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002352
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002352
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002352
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002364
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002364
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002364
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0839_2006050100_123_975_12192483_0_r1131672709_98.zip
Uploading the intermediate file: upload_file_98.zip
  07:32:42 STEP 2377 H=2377:00 +CPU= 24.971
  07:33:10 STEP 2378 H=2378:00 +CPU= 27.909
  07:33:53 STEP 2379 H=2379:00 +CPU= 42.411
  07:34:19 STEP 2380 H=2380:00 +CPU= 26.013
  07:34:43 STEP 2381 H=2381:00 +CPU= 23.199
  07:35:27 STEP 2382 H=2382:00 +CPU= 43.904
  07:35:56 STEP 2383 H=2383:00 +CPU= 28.889
  07:36:27 STEP 2384 H=2384:00 +CPU= 30.464
  07:37:11 STEP 2385 H=2385:00 +CPU= 43.552
  07:37:38 STEP 2386 H=2386:00 +CPU= 26.218
  07:38:01 STEP 2387 H=2387:00 +CPU= 23.622
  07:38:49 STEP 2388 H=2388:00 +CPU= 47.286
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002388
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002388
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002388
  07:39:18 STEP 2389 H=2389:00 +CPU= 27.989
  07:39:45 STEP 2390 H=2390:00 +CPU= 27.391
  07:40:29 STEP 2391 H=2391:00 +CPU= 43.542
  07:40:59 STEP 2392 H=2392:00 +CPU= 29.593
  07:41:27 STEP 2393 H=2393:00 +CPU= 27.386
  07:42:14 STEP 2394 H=2394:00 +CPU= 46.508
  07:42:41 STEP 2395 H=2395:00 +CPU= 26.812
  07:43:10 STEP 2396 H=2396:00 +CPU= 28.655
  07:43:52 STEP 2397 H=2397:00 +CPU= 40.643
  07:44:20 STEP 2398 H=2398:00 +CPU= 28.053
  07:44:50 STEP 2399 H=2399:00 +CPU= 29.816
  07:45:45 STEP 2400 H=2400:00 +CPU= 53.346
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002400
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002400
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002400
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002376
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002376
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002376
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002388
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002388
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002388
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0839_2006050100_123_975_12192483_0_r1131672709_99.zip
Uploading the intermediate file: upload_file_99.zip
  07:46:11 STEP 2401 H=2401:00 +CPU= 26.116
  07:46:39 STEP 2402 H=2402:00 +CPU= 27.364
  07:47:22 STEP 2403 H=2403:00 +CPU= 42.999
  07:47:50 STEP 2404 H=2404:00 +CPU= 27.290
  07:48:16 STEP 2405 H=2405:00 +CPU= 25.406
  07:48:59 STEP 2406 H=2406:00 +CPU= 42.471
  07:49:24 STEP 2407 H=2407:00 +CPU= 24.802
  07:49:51 STEP 2408 H=2408:00 +CPU= 27.185
  07:50:33 STEP 2409 H=2409:00 +CPU= 40.954
  07:51:02 STEP 2410 H=2410:00 +CPU= 28.754
  07:51:27 STEP 2411 H=2411:00 +CPU= 24.736
  07:52:11 STEP 2412 H=2412:00 +CPU= 43.520
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002412
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002412
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002412
  07:52:39 STEP 2413 H=2413:00 +CPU= 27.923
  07:53:08 STEP 2414 H=2414:00 +CPU= 28.717
  07:53:57 STEP 2415 H=2415:00 +CPU= 48.129
  07:54:24 STEP 2416 H=2416:00 +CPU= 26.659
  07:54:50 STEP 2417 H=2417:00 +CPU= 25.542
  07:55:34 STEP 2418 H=2418:00 +CPU= 43.124
  07:55:59 STEP 2419 H=2419:00 +CPU= 25.448
  07:56:30 STEP 2420 H=2420:00 +CPU= 30.364
  07:57:15 STEP 2421 H=2421:00 +CPU= 44.460
  07:57:44 STEP 2422 H=2422:00 +CPU= 28.830
  07:58:13 STEP 2423 H=2423:00 +CPU= 28.532
  07:59:07 STEP 2424 H=2424:00 +CPU= 53.161
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002424
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002424
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002424
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002400
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002400
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002400
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002412
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002412
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002412
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0839_2006050100_123_975_12192483_0_r1131672709_100.zip
Uploading the intermediate file: upload_file_100.zip
  07:59:35 STEP 2425 H=2425:00 +CPU= 27.492
  08:00:05 STEP 2426 H=2426:00 +CPU= 30.552
  08:00:51 STEP 2427 H=2427:00 +CPU= 45.106
  08:01:18 STEP 2428 H=2428:00 +CPU= 26.315
  08:01:45 STEP 2429 H=2429:00 +CPU= 26.530
  08:02:27 STEP 2430 H=2430:00 +CPU= 41.772
  08:02:53 STEP 2431 H=2431:00 +CPU= 25.758
  08:03:20 STEP 2432 H=2432:00 +CPU= 26.493
  08:04:07 STEP 2433 H=2433:00 +CPU= 46.217
  08:04:34 STEP 2434 H=2434:00 +CPU= 26.676
  08:05:00 STEP 2435 H=2435:00 +CPU= 25.611
  08:05:45 STEP 2436 H=2436:00 +CPU= 44.970
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002436
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002436
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002436
  08:06:12 STEP 2437 H=2437:00 +CPU= 27.131
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2006050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0839
(argv4) batchid: 975
(argv5) wuid: 12192483
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /data/boinc/boinc-client/slots/21
Project directory is: /data/boinc/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483
..mkdir for temp folder for results failed
Copying: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /data/boinc/boinc-client/slots/21/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /data/boinc/boinc-client/slots/21/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_bbf14332058794b5dd0180f87f4f35d9 to: /data/boinc/boinc-client/slots/21/oifs_43r3_ps_0839_2006050100_123_975_12192483.zip
Unzipping the namelist zip file: /data/boinc/boinc-client/slots/21/oifs_43r3_ps_0839_2006050100_123_975_12192483.zip
ic_ancil_file: ic_ancil_12192483
ifsdata_file: ifsdata_12192483
climate_data_file: clim_data_12192483
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_866d3e2695cdcc8f6350df59b9a98a4e to: /data/boinc/boinc-client/slots/21/ic_ancil_12192483.zip
Unzipping the IC ancils zip file: /data/boinc/boinc-client/slots/21/ic_ancil_12192483.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_c0f398141abf217ab95cfae8da87c240 to: /data/boinc/boinc-client/slots/21/ifsdata/ifsdata_12192483.zip
Unzipping the ifsdata_zip file: /data/boinc/boinc-client/slots/21/ifsdata/ifsdata_12192483.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /data/boinc/boinc-client/slots/21/159l_2/clim_data_12192483.zip
Unzipping the climate data zip file: /data/boinc/boinc-client/slots/21/159l_2/clim_data_12192483.zip
Checking for progress XML file: /data/boinc/boinc-client/slots/21/progress_file_12192483.xml
Opened progress file ok : /data/boinc/boinc-client/slots/21/progress_file_12192483.xml
-- Model is restarting --
Adjusting last_iter, 2438, to previous model restart step.
Creating progress file: /data/boinc/boinc-client/slots/21/progress_file_12192483.xml
last_cpu_time: 84451
upload_file_number: 101
last_iter: 2423
last_upload: 8726400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0839_2006050100_123_975_12192483_0_r1131672709
The child process has been launched with process id: 19314
Executing the command: /data/boinc/boinc-client/slots/21/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:victor-Precision-7920-Tower:1:1:19314:19314] [20230124:080749:1674547669:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  08:08:46 STEP 2424 H=2424:00 +CPU= 46.169
  08:09:11 STEP 2425 H=2425:00 +CPU= 25.527
  08:09:34 STEP 2426 H=2426:00 +CPU= 21.818
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:39:59 STEP 2427 H=2427:00 +CPU= 38.755
  08:40:30 STEP 2428 H=2428:00 +CPU= 29.002
  08:41:01 STEP 2429 H=2429:00 +CPU= 29.470
  08:41:50 STEP 2430 H=2430:00 +CPU= 45.620
  08:42:19 STEP 2431 H=2431:00 +CPU= 27.782
  08:42:52 STEP 2432 H=2432:00 +CPU= 30.093
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:04:13 STEP 2433 H=2433:00 +CPU= 44.868
  09:04:42 STEP 2434 H=2434:00 +CPU= 27.943
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:07:55 STEP 2435 H=2435:00 +CPU= 26.903
  09:08:44 STEP 2436 H=2436:00 +CPU= 47.264
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002436
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002436
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002436
  09:09:12 STEP 2437 H=2437:00 +CPU= 27.447
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:15:29 STEP 2438 H=2438:00 +CPU= 30.816
  09:16:15 STEP 2439 H=2439:00 +CPU= 44.983
  09:16:42 STEP 2440 H=2440:00 +CPU= 26.838
  09:17:09 STEP 2441 H=2441:00 +CPU= 25.878
  09:17:54 STEP 2442 H=2442:00 +CPU= 44.487
  09:18:19 STEP 2443 H=2443:00 +CPU= 24.581
  09:18:47 STEP 2444 H=2444:00 +CPU= 26.753
  09:19:23 STEP 2445 H=2445:00 +CPU= 36.065
  09:19:49 STEP 2446 H=2446:00 +CPU= 24.563
  09:20:12 STEP 2447 H=2447:00 +CPU= 22.856
  09:21:01 STEP 2448 H=2448:00 +CPU= 47.674
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002448
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002448
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002448
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002424
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002424
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002424
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002436
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002436
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002436
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0839_2006050100_123_975_12192483_0_r1131672709_101.zip
Uploading the intermediate file: upload_file_101.zip
  09:21:26 STEP 2449 H=2449:00 +CPU= 25.001
  09:21:49 STEP 2450 H=2450:00 +CPU= 22.168
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:25:21 STEP 2451 H=2451:00 +CPU= 39.705
  09:25:44 STEP 2452 H=2452:00 +CPU= 22.864
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:29:04 STEP 2453 H=2453:00 +CPU= 22.791
  09:29:46 STEP 2454 H=2454:00 +CPU= 41.750
  09:30:13 STEP 2455 H=2455:00 +CPU= 25.992
  09:30:37 STEP 2456 H=2456:00 +CPU= 23.389
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:38:25 STEP 2457 H=2457:00 +CPU= 44.766
  09:38:53 STEP 2458 H=2458:00 +CPU= 27.309
  09:39:17 STEP 2459 H=2459:00 +CPU= 23.482
  09:40:01 STEP 2460 H=2460:00 +CPU= 43.660
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002460
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002460
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002460
  09:40:28 STEP 2461 H=2461:00 +CPU= 26.274
  09:40:56 STEP 2462 H=2462:00 +CPU= 27.640
  09:41:40 STEP 2463 H=2463:00 +CPU= 42.807
  09:42:06 STEP 2464 H=2464:00 +CPU= 24.728
  09:42:31 STEP 2465 H=2465:00 +CPU= 24.739
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:48:50 STEP 2466 H=2466:00 +CPU= 39.589
  09:49:14 STEP 2467 H=2467:00 +CPU= 23.360
  09:49:36 STEP 2468 H=2468:00 +CPU= 22.153
  09:50:17 STEP 2469 H=2469:00 +CPU= 39.810
  09:50:44 STEP 2470 H=2470:00 +CPU= 26.977
  09:51:08 STEP 2471 H=2471:00 +CPU= 23.180
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:55:40 STEP 2472 H=2472:00 +CPU= 46.199
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002472
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002472
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002472
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002448
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002448
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002448
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002460
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002460
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002460
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0839_2006050100_123_975_12192483_0_r1131672709_102.zip
Uploading the intermediate file: upload_file_102.zip
  09:56:06 STEP 2473 H=2473:00 +CPU= 25.154
  09:56:30 STEP 2474 H=2474:00 +CPU= 23.876
  09:57:12 STEP 2475 H=2475:00 +CPU= 40.486
  09:57:35 STEP 2476 H=2476:00 +CPU= 22.952
  09:57:58 STEP 2477 H=2477:00 +CPU= 22.544
  09:58:41 STEP 2478 H=2478:00 +CPU= 42.299
  09:59:06 STEP 2479 H=2479:00 +CPU= 24.273
  09:59:33 STEP 2480 H=2480:00 +CPU= 26.958
  10:00:18 STEP 2481 H=2481:00 +CPU= 43.522
  10:00:44 STEP 2482 H=2482:00 +CPU= 25.416
  10:01:09 STEP 2483 H=2483:00 +CPU= 25.247
  10:01:52 STEP 2484 H=2484:00 +CPU= 41.351
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002484
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002484
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002484
  10:02:17 STEP 2485 H=2485:00 +CPU= 25.122
  10:02:42 STEP 2486 H=2486:00 +CPU= 24.139
  10:03:21 STEP 2487 H=2487:00 +CPU= 37.353
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:44:33 STEP 2488 H=2488:00 +CPU= 25.195
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:04:20 STEP 2489 H=2489:00 +CPU= 27.577
  11:05:08 STEP 2490 H=2490:00 +CPU= 44.957
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:23:00 STEP 2491 H=2491:00 +CPU= 28.413
  11:23:30 STEP 2492 H=2492:00 +CPU= 28.353
  11:24:16 STEP 2493 H=2493:00 +CPU= 42.331
  11:24:46 STEP 2494 H=2494:00 +CPU= 28.608
  11:25:18 STEP 2495 H=2495:00 +CPU= 29.792
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:30:55 STEP 2496 H=2496:00 +CPU= 57.835
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002496
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002496
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002496
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002472
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002472
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002472
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002484
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002484
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002484
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0839_2006050100_123_975_12192483_0_r1131672709_103.zip
Uploading the intermediate file: upload_file_103.zip
  11:31:26 STEP 2497 H=2497:00 +CPU= 29.536
  11:32:00 STEP 2498 H=2498:00 +CPU= 31.625
  11:32:52 STEP 2499 H=2499:00 +CPU= 48.686
  11:33:23 STEP 2500 H=2500:00 +CPU= 29.451
  11:33:54 STEP 2501 H=2501:00 +CPU= 28.453
  11:34:42 STEP 2502 H=2502:00 +CPU= 45.207
  11:35:14 STEP 2503 H=2503:00 +CPU= 30.202
  11:35:46 STEP 2504 H=2504:00 +CPU= 30.185
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:39:07 STEP 2505 H=2505:00 +CPU= 46.527
  11:39:40 STEP 2506 H=2506:00 +CPU= 31.474
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:43:12 STEP 2507 H=2507:00 +CPU= 29.013
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:42:01 STEP 2508 H=2508:00 +CPU= 51.527
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002508
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002508
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002508
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:53:12 STEP 2509 H=2509:00 +CPU= 28.226
  12:53:43 STEP 2510 H=2510:00 +CPU= 28.598
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:47:45 STEP 2511 H=2511:00 +CPU= 45.128
  13:48:14 STEP 2512 H=2512:00 +CPU= 27.786
  13:48:44 STEP 2513 H=2513:00 +CPU= 29.098
  13:49:33 STEP 2514 H=2514:00 +CPU= 47.218
  13:50:01 STEP 2515 H=2515:00 +CPU= 27.883
  13:50:31 STEP 2516 H=2516:00 +CPU= 28.891
  13:51:15 STEP 2517 H=2517:00 +CPU= 42.725
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:58:03 STEP 2518 H=2518:00 +CPU= 29.438
  13:58:30 STEP 2519 H=2519:00 +CPU= 26.884
  13:59:24 STEP 2520 H=2520:00 +CPU= 51.317
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002520
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002520
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002520
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002496
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002496
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002496
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002508
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002508
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002508
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0839_2006050100_123_975_12192483_0_r1131672709_104.zip
Uploading the intermediate file: upload_file_104.zip
  13:59:54 STEP 2521 H=2521:00 +CPU= 28.907
  14:00:21 STEP 2522 H=2522:00 +CPU= 26.462
  14:01:08 STEP 2523 H=2523:00 +CPU= 46.163
  14:01:36 STEP 2524 H=2524:00 +CPU= 27.322
  14:02:05 STEP 2525 H=2525:00 +CPU= 27.836
  14:02:48 STEP 2526 H=2526:00 +CPU= 42.029
  14:03:15 STEP 2527 H=2527:00 +CPU= 26.536
  14:03:42 STEP 2528 H=2528:00 +CPU= 26.214
  14:04:24 STEP 2529 H=2529:00 +CPU= 40.718
  14:04:53 STEP 2530 H=2530:00 +CPU= 28.145
  14:05:22 STEP 2531 H=2531:00 +CPU= 28.279
  14:06:12 STEP 2532 H=2532:00 +CPU= 48.534
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002532
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002532
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002532
  14:06:37 STEP 2533 H=2533:00 +CPU= 24.268
  14:07:06 STEP 2534 H=2534:00 +CPU= 28.884
  14:07:49 STEP 2535 H=2535:00 +CPU= 41.607
  14:08:18 STEP 2536 H=2536:00 +CPU= 28.446
  14:08:46 STEP 2537 H=2537:00 +CPU= 26.894
  14:09:32 STEP 2538 H=2538:00 +CPU= 45.508
  14:10:02 STEP 2539 H=2539:00 +CPU= 28.611
  14:10:30 STEP 2540 H=2540:00 +CPU= 28.026
  14:11:12 STEP 2541 H=2541:00 +CPU= 40.817
  14:11:38 STEP 2542 H=2542:00 +CPU= 25.676
  14:12:08 STEP 2543 H=2543:00 +CPU= 28.444
  14:13:00 STEP 2544 H=2544:00 +CPU= 50.789
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002544
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002544
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002544
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002520
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002520
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002520
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002532
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002532
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002532
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0839_2006050100_123_975_12192483_0_r1131672709_105.zip
Uploading the intermediate file: upload_file_105.zip
  14:13:25 STEP 2545 H=2545:00 +CPU= 24.783
  14:13:52 STEP 2546 H=2546:00 +CPU= 26.491
  14:14:36 STEP 2547 H=2547:00 +CPU= 42.331
  14:15:04 STEP 2548 H=2548:00 +CPU= 27.823
  14:15:35 STEP 2549 H=2549:00 +CPU= 29.707
  14:16:19 STEP 2550 H=2550:00 +CPU= 43.037
  14:16:47 STEP 2551 H=2551:00 +CPU= 27.762
  14:17:16 STEP 2552 H=2552:00 +CPU= 28.043
  14:17:59 STEP 2553 H=2553:00 +CPU= 41.874
  14:18:28 STEP 2554 H=2554:00 +CPU= 28.225
  14:18:56 STEP 2555 H=2555:00 +CPU= 27.922
  14:19:47 STEP 2556 H=2556:00 +CPU= 49.692
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002556
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002556
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002556
  14:20:13 STEP 2557 H=2557:00 +CPU= 25.007
  14:20:41 STEP 2558 H=2558:00 +CPU= 27.035
  14:21:26 STEP 2559 H=2559:00 +CPU= 44.150
  14:21:55 STEP 2560 H=2560:00 +CPU= 28.297
  14:22:24 STEP 2561 H=2561:00 +CPU= 28.365
  14:23:10 STEP 2562 H=2562:00 +CPU= 45.423
  14:23:40 STEP 2563 H=2563:00 +CPU= 29.509
  14:24:11 STEP 2564 H=2564:00 +CPU= 29.681
  14:24:56 STEP 2565 H=2565:00 +CPU= 44.150
  14:25:24 STEP 2566 H=2566:00 +CPU= 27.529
  14:25:56 STEP 2567 H=2567:00 +CPU= 31.135
  14:26:47 STEP 2568 H=2568:00 +CPU= 49.956
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002568
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002568
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002568
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002544
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002544
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002544
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002556
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002556
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002556
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0839_2006050100_123_975_12192483_0_r1131672709_106.zip
Uploading the intermediate file: upload_file_106.zip
  14:27:15 STEP 2569 H=2569:00 +CPU= 27.012
  14:27:42 STEP 2570 H=2570:00 +CPU= 26.523
  14:28:27 STEP 2571 H=2571:00 +CPU= 43.464
  14:28:58 STEP 2572 H=2572:00 +CPU= 30.060
  14:29:26 STEP 2573 H=2573:00 +CPU= 27.172
  14:30:12 STEP 2574 H=2574:00 +CPU= 45.388
  14:30:37 STEP 2575 H=2575:00 +CPU= 24.371
  14:31:07 STEP 2576 H=2576:00 +CPU= 29.251
  14:31:50 STEP 2577 H=2577:00 +CPU= 41.232
  14:32:17 STEP 2578 H=2578:00 +CPU= 26.973
  14:32:44 STEP 2579 H=2579:00 +CPU= 26.584
  14:33:35 STEP 2580 H=2580:00 +CPU= 49.042
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002580
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002580
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002580
  14:34:03 STEP 2581 H=2581:00 +CPU= 27.961
  14:34:30 STEP 2582 H=2582:00 +CPU= 26.093
  14:35:11 STEP 2583 H=2583:00 +CPU= 40.142
  14:35:40 STEP 2584 H=2584:00 +CPU= 28.010
  14:36:07 STEP 2585 H=2585:00 +CPU= 26.391
  14:36:52 STEP 2586 H=2586:00 +CPU= 43.729
  14:37:20 STEP 2587 H=2587:00 +CPU= 27.178
  14:37:47 STEP 2588 H=2588:00 +CPU= 26.715
  14:38:31 STEP 2589 H=2589:00 +CPU= 42.932
  14:38:58 STEP 2590 H=2590:00 +CPU= 26.326
  14:39:28 STEP 2591 H=2591:00 +CPU= 28.659
  14:40:19 STEP 2592 H=2592:00 +CPU= 50.271
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002592
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002592
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002592
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002568
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002568
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002568
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002580
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002580
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002580
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0839_2006050100_123_975_12192483_0_r1131672709_107.zip
Uploading the intermediate file: upload_file_107.zip
  14:40:48 STEP 2593 H=2593:00 +CPU= 27.415
  14:41:15 STEP 2594 H=2594:00 +CPU= 26.827
  14:41:56 STEP 2595 H=2595:00 +CPU= 39.756
  14:42:24 STEP 2596 H=2596:00 +CPU= 27.774
  14:42:49 STEP 2597 H=2597:00 +CPU= 24.384
  14:43:37 STEP 2598 H=2598:00 +CPU= 46.746
  14:44:02 STEP 2599 H=2599:00 +CPU= 24.257
  14:44:31 STEP 2600 H=2600:00 +CPU= 28.185
  14:45:15 STEP 2601 H=2601:00 +CPU= 42.723
  14:45:43 STEP 2602 H=2602:00 +CPU= 27.522
  14:46:12 STEP 2603 H=2603:00 +CPU= 27.973
  14:47:00 STEP 2604 H=2604:00 +CPU= 46.786
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002604
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002604
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002604
  14:47:27 STEP 2605 H=2605:00 +CPU= 26.774
  14:47:56 STEP 2606 H=2606:00 +CPU= 28.199
  14:48:39 STEP 2607 H=2607:00 +CPU= 41.625
  14:49:04 STEP 2608 H=2608:00 +CPU= 24.736
  14:49:31 STEP 2609 H=2609:00 +CPU= 26.694
  14:50:17 STEP 2610 H=2610:00 +CPU= 44.485
  14:50:47 STEP 2611 H=2611:00 +CPU= 28.974
  14:51:18 STEP 2612 H=2612:00 +CPU= 30.898
  14:52:02 STEP 2613 H=2613:00 +CPU= 42.393
  14:52:30 STEP 2614 H=2614:00 +CPU= 28.002
  14:52:59 STEP 2615 H=2615:00 +CPU= 28.382
  14:53:56 STEP 2616 H=2616:00 +CPU= 55.206
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002616
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002616
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002616
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002592
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002592
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002592
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002604
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002604
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002604
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0839_2006050100_123_975_12192483_0_r1131672709_108.zip
Uploading the intermediate file: upload_file_108.zip
  14:54:25 STEP 2617 H=2617:00 +CPU= 27.496
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:19:56 STEP 2618 H=2618:00 +CPU= 28.510
  16:20:39 STEP 2619 H=2619:00 +CPU= 41.446
  16:21:05 STEP 2620 H=2620:00 +CPU= 25.620
  16:21:31 STEP 2621 H=2621:00 +CPU= 25.937
  16:22:17 STEP 2622 H=2622:00 +CPU= 44.192
  16:22:42 STEP 2623 H=2623:00 +CPU= 25.144
  16:23:09 STEP 2624 H=2624:00 +CPU= 26.306
  16:23:51 STEP 2625 H=2625:00 +CPU= 40.524
  16:24:21 STEP 2626 H=2626:00 +CPU= 29.434
  16:24:45 STEP 2627 H=2627:00 +CPU= 23.476
  16:25:30 STEP 2628 H=2628:00 +CPU= 43.542
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002628
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002628
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002628
  16:25:57 STEP 2629 H=2629:00 +CPU= 27.055
  16:26:26 STEP 2630 H=2630:00 +CPU= 27.849
  16:27:09 STEP 2631 H=2631:00 +CPU= 41.759
  16:27:37 STEP 2632 H=2632:00 +CPU= 27.272
  16:28:06 STEP 2633 H=2633:00 +CPU= 29.067
  16:28:51 STEP 2634 H=2634:00 +CPU= 43.721
  16:29:17 STEP 2635 H=2635:00 +CPU= 25.348
  16:29:43 STEP 2636 H=2636:00 +CPU= 25.636
  16:30:28 STEP 2637 H=2637:00 +CPU= 43.741
  16:30:57 STEP 2638 H=2638:00 +CPU= 27.746
  16:31:21 STEP 2639 H=2639:00 +CPU= 23.386
  16:32:12 STEP 2640 H=2640:00 +CPU= 49.587
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002640
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002640
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002640
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002616
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002616
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002616
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002628
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002628
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002628
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0839_2006050100_123_975_12192483_0_r1131672709_109.zip
Uploading the intermediate file: upload_file_109.zip
  16:32:35 STEP 2641 H=2641:00 +CPU= 23.060
  16:33:00 STEP 2642 H=2642:00 +CPU= 23.990
  16:33:37 STEP 2643 H=2643:00 +CPU= 36.275
  16:34:02 STEP 2644 H=2644:00 +CPU= 24.515
  16:34:23 STEP 2645 H=2645:00 +CPU= 21.647
  16:35:03 STEP 2646 H=2646:00 +CPU= 38.994
  16:35:25 STEP 2647 H=2647:00 +CPU= 21.694
  16:35:47 STEP 2648 H=2648:00 +CPU= 21.570
  16:36:21 STEP 2649 H=2649:00 +CPU= 33.344
  16:36:43 STEP 2650 H=2650:00 +CPU= 21.776
  16:37:08 STEP 2651 H=2651:00 +CPU= 24.795
  16:37:46 STEP 2652 H=2652:00 +CPU= 37.626
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002652
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002652
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002652
  16:38:08 STEP 2653 H=2653:00 +CPU= 21.519
  16:38:32 STEP 2654 H=2654:00 +CPU= 23.206
  16:39:07 STEP 2655 H=2655:00 +CPU= 34.782
  16:39:30 STEP 2656 H=2656:00 +CPU= 22.131
  16:39:52 STEP 2657 H=2657:00 +CPU= 21.975
  16:40:29 STEP 2658 H=2658:00 +CPU= 37.047
  16:40:52 STEP 2659 H=2659:00 +CPU= 21.899
  16:41:14 STEP 2660 H=2660:00 +CPU= 22.399
  16:41:51 STEP 2661 H=2661:00 +CPU= 35.955
  16:42:16 STEP 2662 H=2662:00 +CPU= 24.160
  16:42:40 STEP 2663 H=2663:00 +CPU= 23.442
  16:43:24 STEP 2664 H=2664:00 +CPU= 42.691
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002664
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002664
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002664
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002640
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002640
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002640
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002652
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002652
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002652
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0839_2006050100_123_975_12192483_0_r1131672709_110.zip
Uploading the intermediate file: upload_file_110.zip
  16:43:48 STEP 2665 H=2665:00 +CPU= 23.066
Uploading trickle at timestep: 9590400
  16:44:12 STEP 2666 H=2666:00 +CPU= 23.246
  16:44:48 STEP 2667 H=2667:00 +CPU= 35.289
  16:45:11 STEP 2668 H=2668:00 +CPU= 22.650
  16:45:32 STEP 2669 H=2669:00 +CPU= 21.166
  16:46:09 STEP 2670 H=2670:00 +CPU= 36.788
  16:46:31 STEP 2671 H=2671:00 +CPU= 21.859
  16:46:54 STEP 2672 H=2672:00 +CPU= 22.468
  16:47:30 STEP 2673 H=2673:00 +CPU= 34.836
  16:47:52 STEP 2674 H=2674:00 +CPU= 22.500
  16:48:14 STEP 2675 H=2675:00 +CPU= 21.663
  16:48:53 STEP 2676 H=2676:00 +CPU= 38.470
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002676
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002676
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002676
  16:49:15 STEP 2677 H=2677:00 +CPU= 21.448
  16:49:38 STEP 2678 H=2678:00 +CPU= 22.501
  16:50:15 STEP 2679 H=2679:00 +CPU= 36.938
  16:50:39 STEP 2680 H=2680:00 +CPU= 22.882
  16:51:01 STEP 2681 H=2681:00 +CPU= 22.024
  16:51:39 STEP 2682 H=2682:00 +CPU= 36.831
  16:52:02 STEP 2683 H=2683:00 +CPU= 22.702
  16:52:25 STEP 2684 H=2684:00 +CPU= 22.697
  16:53:04 STEP 2685 H=2685:00 +CPU= 37.454
  16:53:28 STEP 2686 H=2686:00 +CPU= 23.984
  16:53:53 STEP 2687 H=2687:00 +CPU= 23.747
  16:54:39 STEP 2688 H=2688:00 +CPU= 44.400
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002688
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002688
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002688
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002664
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002664
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002664
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002676
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002676
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002676
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0839_2006050100_123_975_12192483_0_r1131672709_111.zip
Uploading the intermediate file: upload_file_111.zip
  16:55:01 STEP 2689 H=2689:00 +CPU= 21.542
  16:55:24 STEP 2690 H=2690:00 +CPU= 22.536
  16:56:02 STEP 2691 H=2691:00 +CPU= 36.800
  16:56:24 STEP 2692 H=2692:00 +CPU= 21.537
  16:56:46 STEP 2693 H=2693:00 +CPU= 21.889
  16:57:23 STEP 2694 H=2694:00 +CPU= 35.910
  16:57:45 STEP 2695 H=2695:00 +CPU= 21.792
  16:58:08 STEP 2696 H=2696:00 +CPU= 23.018
  16:58:44 STEP 2697 H=2697:00 +CPU= 35.723
  16:59:07 STEP 2698 H=2698:00 +CPU= 22.463
  16:59:29 STEP 2699 H=2699:00 +CPU= 21.811
  17:00:08 STEP 2700 H=2700:00 +CPU= 38.603
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002700
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002700
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002700
  17:00:30 STEP 2701 H=2701:00 +CPU= 21.558
  17:00:52 STEP 2702 H=2702:00 +CPU= 22.015
  17:01:28 STEP 2703 H=2703:00 +CPU= 34.909
  17:01:51 STEP 2704 H=2704:00 +CPU= 22.599
  17:02:12 STEP 2705 H=2705:00 +CPU= 21.139
  17:02:48 STEP 2706 H=2706:00 +CPU= 35.731
  17:03:10 STEP 2707 H=2707:00 +CPU= 22.062
  17:03:33 STEP 2708 H=2708:00 +CPU= 22.028
  17:04:07 STEP 2709 H=2709:00 +CPU= 34.371
  17:04:30 STEP 2710 H=2710:00 +CPU= 22.139
  17:04:52 STEP 2711 H=2711:00 +CPU= 21.968
  17:05:33 STEP 2712 H=2712:00 +CPU= 40.252
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002712
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002712
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002712
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002688
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002688
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002688
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002700
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002700
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002700
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0839_2006050100_123_975_12192483_0_r1131672709_112.zip
Uploading the intermediate file: upload_file_112.zip
  17:05:54 STEP 2713 H=2713:00 +CPU= 21.230
  17:06:17 STEP 2714 H=2714:00 +CPU= 22.338
  17:06:52 STEP 2715 H=2715:00 +CPU= 34.936
  17:07:14 STEP 2716 H=2716:00 +CPU= 21.808
  17:07:36 STEP 2717 H=2717:00 +CPU= 21.703
  17:08:12 STEP 2718 H=2718:00 +CPU= 35.628
  17:08:34 STEP 2719 H=2719:00 +CPU= 21.290
  17:08:56 STEP 2720 H=2720:00 +CPU= 21.989
  17:09:30 STEP 2721 H=2721:00 +CPU= 33.741
  17:09:54 STEP 2722 H=2722:00 +CPU= 23.693
  17:10:20 STEP 2723 H=2723:00 +CPU= 24.868
  17:11:00 STEP 2724 H=2724:00 +CPU= 39.674
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002724
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002724
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002724
  17:11:23 STEP 2725 H=2725:00 +CPU= 22.243
  17:11:47 STEP 2726 H=2726:00 +CPU= 24.075
  17:12:24 STEP 2727 H=2727:00 +CPU= 35.701
  17:12:46 STEP 2728 H=2728:00 +CPU= 21.965
  17:13:08 STEP 2729 H=2729:00 +CPU= 21.515
  17:13:45 STEP 2730 H=2730:00 +CPU= 37.183
  17:14:07 STEP 2731 H=2731:00 +CPU= 21.824
  17:14:30 STEP 2732 H=2732:00 +CPU= 22.469
  17:15:06 STEP 2733 H=2733:00 +CPU= 34.701
  17:15:28 STEP 2734 H=2734:00 +CPU= 22.275
  17:15:51 STEP 2735 H=2735:00 +CPU= 22.870
  17:16:34 STEP 2736 H=2736:00 +CPU= 42.188
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002736
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002736
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002736
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002712
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002712
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002712
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002724
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002724
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002724
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0839_2006050100_123_975_12192483_0_r1131672709_113.zip
Uploading the intermediate file: upload_file_113.zip
  17:16:58 STEP 2737 H=2737:00 +CPU= 22.990
  17:17:20 STEP 2738 H=2738:00 +CPU= 22.333
  17:17:57 STEP 2739 H=2739:00 +CPU= 35.502
  17:18:20 STEP 2740 H=2740:00 +CPU= 22.710
  17:18:42 STEP 2741 H=2741:00 +CPU= 22.318
  17:19:20 STEP 2742 H=2742:00 +CPU= 36.693
  17:19:43 STEP 2743 H=2743:00 +CPU= 22.464
  17:20:08 STEP 2744 H=2744:00 +CPU= 23.928
  17:20:46 STEP 2745 H=2745:00 +CPU= 37.144
  17:21:12 STEP 2746 H=2746:00 +CPU= 24.321
  17:21:35 STEP 2747 H=2747:00 +CPU= 22.431
  17:22:16 STEP 2748 H=2748:00 +CPU= 40.357
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002748
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002748
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002748
  17:22:40 STEP 2749 H=2749:00 +CPU= 23.054
  17:23:04 STEP 2750 H=2750:00 +CPU= 23.937
  17:23:40 STEP 2751 H=2751:00 +CPU= 34.820
  17:24:03 STEP 2752 H=2752:00 +CPU= 22.617
  17:24:24 STEP 2753 H=2753:00 +CPU= 21.549
  17:25:01 STEP 2754 H=2754:00 +CPU= 35.947
  17:25:23 STEP 2755 H=2755:00 +CPU= 22.192
  17:25:46 STEP 2756 H=2756:00 +CPU= 22.639
  17:26:23 STEP 2757 H=2757:00 +CPU= 35.689
  17:26:46 STEP 2758 H=2758:00 +CPU= 22.639
  17:27:07 STEP 2759 H=2759:00 +CPU= 21.131
  17:27:50 STEP 2760 H=2760:00 +CPU= 42.032
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002760
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002760
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002760
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002736
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002736
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002736
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002748
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002748
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002748
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0839_2006050100_123_975_12192483_0_r1131672709_114.zip
Uploading the intermediate file: upload_file_114.zip
  17:28:12 STEP 2761 H=2761:00 +CPU= 21.701
  17:28:35 STEP 2762 H=2762:00 +CPU= 23.281
  17:29:11 STEP 2763 H=2763:00 +CPU= 35.490
  17:29:34 STEP 2764 H=2764:00 +CPU= 22.687
  17:29:56 STEP 2765 H=2765:00 +CPU= 21.719
  17:30:32 STEP 2766 H=2766:00 +CPU= 35.836
  17:30:55 STEP 2767 H=2767:00 +CPU= 22.288
  17:31:18 STEP 2768 H=2768:00 +CPU= 22.756
  17:31:53 STEP 2769 H=2769:00 +CPU= 34.643
  17:32:15 STEP 2770 H=2770:00 +CPU= 21.518
  17:32:38 STEP 2771 H=2771:00 +CPU= 22.857
  17:33:17 STEP 2772 H=2772:00 +CPU= 38.256
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002772
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002772
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002772
  17:33:38 STEP 2773 H=2773:00 +CPU= 21.033
  17:34:01 STEP 2774 H=2774:00 +CPU= 22.357
  17:34:43 STEP 2775 H=2775:00 +CPU= 40.888
  17:35:11 STEP 2776 H=2776:00 +CPU= 27.956
  17:35:40 STEP 2777 H=2777:00 +CPU= 28.384
  17:36:25 STEP 2778 H=2778:00 +CPU= 43.816
  17:36:53 STEP 2779 H=2779:00 +CPU= 27.061
  17:37:21 STEP 2780 H=2780:00 +CPU= 27.640
  17:38:04 STEP 2781 H=2781:00 +CPU= 42.500
  17:38:34 STEP 2782 H=2782:00 +CPU= 29.070
  17:39:01 STEP 2783 H=2783:00 +CPU= 26.774
  17:39:53 STEP 2784 H=2784:00 +CPU= 50.748
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002784
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002784
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002784
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002760
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002760
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002760
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002772
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002772
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002772
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0839_2006050100_123_975_12192483_0_r1131672709_115.zip
Uploading the intermediate file: upload_file_115.zip
  17:40:19 STEP 2785 H=2785:00 +CPU= 25.213
  17:40:49 STEP 2786 H=2786:00 +CPU= 28.999
  17:41:31 STEP 2787 H=2787:00 +CPU= 41.958
  17:41:59 STEP 2788 H=2788:00 +CPU= 27.104
  17:42:27 STEP 2789 H=2789:00 +CPU= 26.849
  17:43:14 STEP 2790 H=2790:00 +CPU= 46.214
  17:43:44 STEP 2791 H=2791:00 +CPU= 28.981
  17:44:11 STEP 2792 H=2792:00 +CPU= 27.287
  17:45:02 STEP 2793 H=2793:00 +CPU= 49.201
  17:45:32 STEP 2794 H=2794:00 +CPU= 29.524
  17:45:58 STEP 2795 H=2795:00 +CPU= 24.983
  17:46:46 STEP 2796 H=2796:00 +CPU= 46.531
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002796
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002796
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002796
  17:47:13 STEP 2797 H=2797:00 +CPU= 26.833
  17:47:42 STEP 2798 H=2798:00 +CPU= 28.655
  17:48:24 STEP 2799 H=2799:00 +CPU= 40.565
  17:48:52 STEP 2800 H=2800:00 +CPU= 27.632
  17:49:21 STEP 2801 H=2801:00 +CPU= 28.766
  17:50:09 STEP 2802 H=2802:00 +CPU= 46.717
  17:50:36 STEP 2803 H=2803:00 +CPU= 27.184
  17:51:06 STEP 2804 H=2804:00 +CPU= 29.097
  17:51:50 STEP 2805 H=2805:00 +CPU= 42.851
  17:52:18 STEP 2806 H=2806:00 +CPU= 28.194
  17:52:49 STEP 2807 H=2807:00 +CPU= 29.872
  17:53:40 STEP 2808 H=2808:00 +CPU= 50.474
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002808
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002808
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002808
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002784
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002784
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002784
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002796
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002796
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002796
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0839_2006050100_123_975_12192483_0_r1131672709_116.zip
Uploading the intermediate file: upload_file_116.zip
  17:54:09 STEP 2809 H=2809:00 +CPU= 28.396
  17:54:36 STEP 2810 H=2810:00 +CPU= 26.375
  17:55:17 STEP 2811 H=2811:00 +CPU= 40.176
  17:55:46 STEP 2812 H=2812:00 +CPU= 27.966
  17:56:15 STEP 2813 H=2813:00 +CPU= 28.504
  17:57:01 STEP 2814 H=2814:00 +CPU= 45.444
  17:57:29 STEP 2815 H=2815:00 +CPU= 27.655
  17:57:57 STEP 2816 H=2816:00 +CPU= 27.360
  17:58:38 STEP 2817 H=2817:00 +CPU= 39.882
  17:59:04 STEP 2818 H=2818:00 +CPU= 25.789
  17:59:31 STEP 2819 H=2819:00 +CPU= 25.914
  18:00:21 STEP 2820 H=2820:00 +CPU= 49.230
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002820
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002820
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002820
  18:00:46 STEP 2821 H=2821:00 +CPU= 24.984
  18:01:14 STEP 2822 H=2822:00 +CPU= 26.706
  18:01:56 STEP 2823 H=2823:00 +CPU= 41.765
  18:02:25 STEP 2824 H=2824:00 +CPU= 28.225
  18:02:53 STEP 2825 H=2825:00 +CPU= 27.555
  18:03:40 STEP 2826 H=2826:00 +CPU= 45.510
  18:04:06 STEP 2827 H=2827:00 +CPU= 25.213
  18:04:35 STEP 2828 H=2828:00 +CPU= 28.576
  18:05:19 STEP 2829 H=2829:00 +CPU= 43.752
  18:05:46 STEP 2830 H=2830:00 +CPU= 26.272
  18:06:15 STEP 2831 H=2831:00 +CPU= 27.674
  18:07:08 STEP 2832 H=2832:00 +CPU= 52.231
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMGGhq0f+002832
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMSHhq0f+002832
Moving to projects directory: /data/boinc/boinc-client/slots/21/ICMUAhq0f+002832
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002808
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002808
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002808
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMGGhq0f+002820
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMSHhq0f+002820
Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192483/ICMUAhq0f+002820
Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0839_2006050100_123_975_12192483_0_r1131672709_117.zip
Uploading the intermediate file: upload_file_117.zip
  18:07:35 STEP 2833 H=2833:00 +CPU= 26.814
  18:08:01 STEP 2834 H=2834:00 +CPU= 25.317
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:39:38 1478188 22278224 oifs_43r3_ps_0839_2006050100_123_975_12192483_0 9,590,400 92,251 0.0096
09 May 2023 16:39:38 1478188 22278224 oifs_43r3_ps_0839_2006050100_123_975_12192483_0 9,590,400 92,251 0.0096
09 May 2023 16:37:51 1478188 22278224 oifs_43r3_ps_0839_2006050100_123_975_12192483_0 7,776,000 75,121 0.0097
09 May 2023 16:37:35 1478188 22278224 oifs_43r3_ps_0839_2006050100_123_975_12192483_0 6,912,000 67,826 0.0098
09 May 2023 16:37:18 1478188 22278224 oifs_43r3_ps_0839_2006050100_123_975_12192483_0 6,048,000 60,237 0.0100
09 May 2023 16:36:40 1478188 22278224 oifs_43r3_ps_0839_2006050100_123_975_12192483_0 5,184,000 52,382 0.0101
09 May 2023 16:36:29 1478188 22278224 oifs_43r3_ps_0839_2006050100_123_975_12192483_0 4,320,000 43,865 0.0102
09 May 2023 16:31:35 1478188 22278224 oifs_43r3_ps_0839_2006050100_123_975_12192483_0 2,851,200 28,980 0.0102
09 May 2023 16:31:20 1478188 22278224 oifs_43r3_ps_0839_2006050100_123_975_12192483_0 1,987,200 20,644 0.0104


©2024 cpdn.org