climateprediction.net home page
Task 22279861

Task 22279861

Name oifs_43r3_ps_0397_2008050100_123_977_12194041_0
Workunit 12194041
Created 23 Dec 2022, 18:19:46 UTC
Sent 11 Jan 2023, 13:23:08 UTC
Report deadline 10 Feb 2023, 13:23:08 UTC
Received 12 Jan 2023, 6:23:47 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1537299
Run time 15 hours 39 min 22 sec
CPU time 15 hours 29 min 37 sec
Validate state Invalid
Credit 4,112.64
Device peak FLOPS 1.00 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,419.12 MB
Peak swap size 4,978.08 MB
Peak disk usage 7,707.59 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
ts/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000492
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0397_2008050100_123_977_12194041_0_r319970419_20.zip
Uploading the intermediate file: upload_file_20.zip
  14:22:39 STEP  505 H= 505:00 +CPU= 43.287
  14:23:24 STEP  506 H= 506:00 +CPU= 44.771
  14:24:01 STEP  507 H= 507:00 +CPU= 36.597
  14:24:25 STEP  508 H= 508:00 +CPU= 24.229
  14:24:46 STEP  509 H= 509:00 +CPU= 20.877
  14:25:33 STEP  510 H= 510:00 +CPU= 46.156
  14:26:19 STEP  511 H= 511:00 +CPU= 46.235
  14:27:18 STEP  512 H= 512:00 +CPU= 58.347
  14:28:58 STEP  513 H= 513:00 +CPU= 98.901
  14:30:15 STEP  514 H= 514:00 +CPU= 75.772
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2008050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0397
(argv4) batchid: 977
(argv5) wuid: 12194041
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/37
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/37/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/37/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_c6bae240ca96928cedd1e0ee3354d3fd to: /var/lib/boinc-client/slots/37/oifs_43r3_ps_0397_2008050100_123_977_12194041.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/37/oifs_43r3_ps_0397_2008050100_123_977_12194041.zip
ic_ancil_file: ic_ancil_12194041
ifsdata_file: ifsdata_12194041
climate_data_file: clim_data_12194041
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_6ce2f59a65beff4afa24f9baf519f820 to: /var/lib/boinc-client/slots/37/ic_ancil_12194041.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/37/ic_ancil_12194041.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_c5550fe926504fa72bc1a1ba2d37f6c0 to: /var/lib/boinc-client/slots/37/ifsdata/ifsdata_12194041.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/37/ifsdata/ifsdata_12194041.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/37/159l_2/clim_data_12194041.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/37/159l_2/clim_data_12194041.zip
Checking for progress XML file: /var/lib/boinc-client/slots/37/progress_file_12194041.xml
Opened progress file ok : /var/lib/boinc-client/slots/37/progress_file_12194041.xml
-- Model is restarting --
Adjusting last_iter, 515, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/37/progress_file_12194041.xml
last_cpu_time: 22615
upload_file_number: 21
last_iter: 503
last_upload: 1814400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0397_2008050100_123_977_12194041_0_r319970419
The child process has been launched with process id: 49263
Executing the command: /var/lib/boinc-client/slots/37/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.001] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.001] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.002] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.002] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.002] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.002] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.002] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.002] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.002] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.002] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.002] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:49263:49263] [20230111:143203:1673469123:0.002] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  14:34:16 STEP  504 H= 504:00 +CPU=119.102
  14:35:37 STEP  505 H= 505:00 +CPU= 79.393
  14:36:56 STEP  506 H= 506:00 +CPU= 77.837
  14:38:46 STEP  507 H= 507:00 +CPU=108.864
  14:39:23 STEP  508 H= 508:00 +CPU= 37.224
  14:39:56 STEP  509 H= 509:00 +CPU= 32.874
  14:40:54 STEP  510 H= 510:00 +CPU= 57.860
  14:41:42 STEP  511 H= 511:00 +CPU= 46.944
  14:42:13 STEP  512 H= 512:00 +CPU= 31.189
  14:42:55 STEP  513 H= 513:00 +CPU= 41.945
  14:43:42 STEP  514 H= 514:00 +CPU= 46.317
  14:44:30 STEP  515 H= 515:00 +CPU= 48.585
  14:45:15 STEP  516 H= 516:00 +CPU= 44.586
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000516
  14:45:41 STEP  517 H= 517:00 +CPU= 25.894
  14:46:21 STEP  518 H= 518:00 +CPU= 39.520
  14:47:26 STEP  519 H= 519:00 +CPU= 64.426
  14:47:52 STEP  520 H= 520:00 +CPU= 26.175
  14:48:17 STEP  521 H= 521:00 +CPU= 24.462
  14:49:08 STEP  522 H= 522:00 +CPU= 51.656
  14:49:53 STEP  523 H= 523:00 +CPU= 44.050
  14:50:21 STEP  524 H= 524:00 +CPU= 28.300
  14:50:57 STEP  525 H= 525:00 +CPU= 36.030
  14:51:30 STEP  526 H= 526:00 +CPU= 32.116
  14:52:10 STEP  527 H= 527:00 +CPU= 39.833
  14:53:15 STEP  528 H= 528:00 +CPU= 64.176
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000516
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0397_2008050100_123_977_12194041_0_r319970419_21.zip
Uploading the intermediate file: upload_file_21.zip
  14:53:41 STEP  529 H= 529:00 +CPU= 26.767
  14:54:08 STEP  530 H= 530:00 +CPU= 26.740
  14:54:59 STEP  531 H= 531:00 +CPU= 50.110
  14:55:44 STEP  532 H= 532:00 +CPU= 45.008
  14:56:15 STEP  533 H= 533:00 +CPU= 31.132
  14:56:58 STEP  534 H= 534:00 +CPU= 42.535
  14:57:28 STEP  535 H= 535:00 +CPU= 30.046
  14:58:08 STEP  536 H= 536:00 +CPU= 39.728
  14:58:51 STEP  537 H= 537:00 +CPU= 42.633
  14:59:17 STEP  538 H= 538:00 +CPU= 26.146
  14:59:52 STEP  539 H= 539:00 +CPU= 34.273
  15:00:54 STEP  540 H= 540:00 +CPU= 61.948
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000540
  15:01:26 STEP  541 H= 541:00 +CPU= 32.193
  15:02:04 STEP  542 H= 542:00 +CPU= 37.108
  15:02:59 STEP  543 H= 543:00 +CPU= 55.574
  15:03:21 STEP  544 H= 544:00 +CPU= 21.108
  15:03:42 STEP  545 H= 545:00 +CPU= 21.761
  15:04:21 STEP  546 H= 546:00 +CPU= 38.809
  15:04:57 STEP  547 H= 547:00 +CPU= 35.380
  15:05:42 STEP  548 H= 548:00 +CPU= 44.586
  15:06:26 STEP  549 H= 549:00 +CPU= 43.832
  15:06:51 STEP  550 H= 550:00 +CPU= 24.634
  15:07:24 STEP  551 H= 551:00 +CPU= 33.074
  15:08:42 STEP  552 H= 552:00 +CPU= 77.595
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000540
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0397_2008050100_123_977_12194041_0_r319970419_22.zip
Uploading the intermediate file: upload_file_22.zip
  15:09:05 STEP  553 H= 553:00 +CPU= 23.171
  15:09:27 STEP  554 H= 554:00 +CPU= 22.069
  15:10:00 STEP  555 H= 555:00 +CPU= 32.488
  15:10:44 STEP  556 H= 556:00 +CPU= 43.800
  15:11:27 STEP  557 H= 557:00 +CPU= 42.543
  15:12:08 STEP  558 H= 558:00 +CPU= 40.743
  15:12:32 STEP  559 H= 559:00 +CPU= 23.939
  15:12:59 STEP  560 H= 560:00 +CPU= 27.017
  15:13:54 STEP  561 H= 561:00 +CPU= 55.058
  15:14:18 STEP  562 H= 562:00 +CPU= 23.377
  15:14:39 STEP  563 H= 563:00 +CPU= 21.494
  15:15:18 STEP  564 H= 564:00 +CPU= 38.470
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000564
  15:15:38 STEP  565 H= 565:00 +CPU= 19.738
  15:16:19 STEP  566 H= 566:00 +CPU= 40.852
  15:17:29 STEP  567 H= 567:00 +CPU= 69.977
  15:18:00 STEP  568 H= 568:00 +CPU= 30.335
  15:18:27 STEP  569 H= 569:00 +CPU= 27.044
  15:19:23 STEP  570 H= 570:00 +CPU= 55.399
  15:19:56 STEP  571 H= 571:00 +CPU= 33.435
  15:20:20 STEP  572 H= 572:00 +CPU= 23.256
  15:21:08 STEP  573 H= 573:00 +CPU= 48.489
  15:21:38 STEP  574 H= 574:00 +CPU= 29.383
  15:22:05 STEP  575 H= 575:00 +CPU= 26.689
  15:23:09 STEP  576 H= 576:00 +CPU= 64.210
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000564
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0397_2008050100_123_977_12194041_0_r319970419_23.zip
Uploading the intermediate file: upload_file_23.zip
  15:23:41 STEP  577 H= 577:00 +CPU= 31.193
  15:24:08 STEP  578 H= 578:00 +CPU= 27.055
  15:24:44 STEP  579 H= 579:00 +CPU= 35.678
  15:25:13 STEP  580 H= 580:00 +CPU= 28.817
  15:25:48 STEP  581 H= 581:00 +CPU= 34.955
  15:26:46 STEP  582 H= 582:00 +CPU= 57.651
  15:27:12 STEP  583 H= 583:00 +CPU= 26.680
  15:27:41 STEP  584 H= 584:00 +CPU= 28.074
  15:28:45 STEP  585 H= 585:00 +CPU= 63.815
  15:29:37 STEP  586 H= 586:00 +CPU= 51.907
  15:30:53 STEP  587 H= 587:00 +CPU= 75.235
  15:32:28 STEP  588 H= 588:00 +CPU= 94.279
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000588
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2008050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0397
(argv4) batchid: 977
(argv5) wuid: 12194041
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/37
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/37/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/37/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_c6bae240ca96928cedd1e0ee3354d3fd to: /var/lib/boinc-client/slots/37/oifs_43r3_ps_0397_2008050100_123_977_12194041.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/37/oifs_43r3_ps_0397_2008050100_123_977_12194041.zip
ic_ancil_file: ic_ancil_12194041
ifsdata_file: ifsdata_12194041
climate_data_file: clim_data_12194041
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_6ce2f59a65beff4afa24f9baf519f820 to: /var/lib/boinc-client/slots/37/ic_ancil_12194041.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/37/ic_ancil_12194041.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_c5550fe926504fa72bc1a1ba2d37f6c0 to: /var/lib/boinc-client/slots/37/ifsdata/ifsdata_12194041.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/37/ifsdata/ifsdata_12194041.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/37/159l_2/clim_data_12194041.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/37/159l_2/clim_data_12194041.zip
Checking for progress XML file: /var/lib/boinc-client/slots/37/progress_file_12194041.xml
Opened progress file ok : /var/lib/boinc-client/slots/37/progress_file_12194041.xml
-- Model is restarting --
Adjusting last_iter, 589, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/37/progress_file_12194041.xml
last_cpu_time: 26225
upload_file_number: 24
last_iter: 575
last_upload: 2073600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0397_2008050100_123_977_12194041_0_r319970419
The child process has been launched with process id: 50419
Executing the command: /var/lib/boinc-client/slots/37/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:50419:50419] [20230111:153350:1673472830:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  15:36:11 STEP  576 H= 576:00 +CPU=125.890
  15:37:26 STEP  577 H= 577:00 +CPU= 75.050
  15:38:44 STEP  578 H= 578:00 +CPU= 76.551
  15:40:36 STEP  579 H= 579:00 +CPU=110.493
  15:41:49 STEP  580 H= 580:00 +CPU= 72.400
  15:42:53 STEP  581 H= 581:00 +CPU= 63.523
  15:43:35 STEP  582 H= 582:00 +CPU= 41.843
  15:44:06 STEP  583 H= 583:00 +CPU= 30.239
  15:44:49 STEP  584 H= 584:00 +CPU= 42.863
  15:45:51 STEP  585 H= 585:00 +CPU= 61.458
  15:46:18 STEP  586 H= 586:00 +CPU= 27.425
  15:46:49 STEP  587 H= 587:00 +CPU= 30.333
  15:47:46 STEP  588 H= 588:00 +CPU= 57.513
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000588
  15:48:10 STEP  589 H= 589:00 +CPU= 23.266
  15:48:34 STEP  590 H= 590:00 +CPU= 24.282
  15:49:12 STEP  591 H= 591:00 +CPU= 37.247
  15:49:43 STEP  592 H= 592:00 +CPU= 31.493
  15:50:26 STEP  593 H= 593:00 +CPU= 42.503
  15:51:01 STEP  594 H= 594:00 +CPU= 34.631
  15:51:25 STEP  595 H= 595:00 +CPU= 24.241
  15:51:54 STEP  596 H= 596:00 +CPU= 28.460
  15:52:56 STEP  597 H= 597:00 +CPU= 61.766
  15:53:27 STEP  598 H= 598:00 +CPU= 31.047
  15:53:52 STEP  599 H= 599:00 +CPU= 24.546
  15:54:40 STEP  600 H= 600:00 +CPU= 48.366
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000588
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0397_2008050100_123_977_12194041_0_r319970419_24.zip
Uploading the intermediate file: upload_file_24.zip
  15:55:13 STEP  601 H= 601:00 +CPU= 32.618
  15:56:04 STEP  602 H= 602:00 +CPU= 50.596
  15:57:12 STEP  603 H= 603:00 +CPU= 67.719
  15:57:52 STEP  604 H= 604:00 +CPU= 40.084
  15:58:29 STEP  605 H= 605:00 +CPU= 36.420
  15:59:25 STEP  606 H= 606:00 +CPU= 56.431
  16:00:14 STEP  607 H= 607:00 +CPU= 48.023
  16:01:29 STEP  608 H= 608:00 +CPU= 75.102
  16:03:26 STEP  609 H= 609:00 +CPU=115.265
  16:04:46 STEP  610 H= 610:00 +CPU= 79.132
  16:06:02 STEP  611 H= 611:00 +CPU= 74.578
  16:08:07 STEP  612 H= 612:00 +CPU=124.148
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000612
  16:09:24 STEP  613 H= 613:00 +CPU= 75.989
  16:10:49 STEP  614 H= 614:00 +CPU= 83.564
  16:12:51 STEP  615 H= 615:00 +CPU=119.899
  16:14:07 STEP  616 H= 616:00 +CPU= 76.206
  16:15:26 STEP  617 H= 617:00 +CPU= 77.776
  16:17:25 STEP  618 H= 618:00 +CPU=117.069
  16:18:42 STEP  619 H= 619:00 +CPU= 76.312
  16:20:07 STEP  620 H= 620:00 +CPU= 84.679
  16:22:03 STEP  621 H= 621:00 +CPU=114.070
  16:23:23 STEP  622 H= 622:00 +CPU= 78.733
  16:24:49 STEP  623 H= 623:00 +CPU= 85.142
  16:27:04 STEP  624 H= 624:00 +CPU=133.612
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000612
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0397_2008050100_123_977_12194041_0_r319970419_25.zip
Uploading the intermediate file: upload_file_25.zip
  16:28:20 STEP  625 H= 625:00 +CPU= 75.186
  16:29:45 STEP  626 H= 626:00 +CPU= 83.788
  16:31:40 STEP  627 H= 627:00 +CPU=113.630
  16:32:59 STEP  628 H= 628:00 +CPU= 77.571
  16:34:16 STEP  629 H= 629:00 +CPU= 76.811
  16:36:08 STEP  630 H= 630:00 +CPU=110.504
  16:37:26 STEP  631 H= 631:00 +CPU= 77.506
  16:38:40 STEP  632 H= 632:00 +CPU= 73.056
  16:40:25 STEP  633 H= 633:00 +CPU=104.393
  16:41:41 STEP  634 H= 634:00 +CPU= 74.226
  16:43:06 STEP  635 H= 635:00 +CPU= 85.176
  16:45:15 STEP  636 H= 636:00 +CPU=127.001
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000636
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000636
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000636
  16:46:32 STEP  637 H= 637:00 +CPU= 76.482
  16:47:53 STEP  638 H= 638:00 +CPU= 79.624
  16:49:52 STEP  639 H= 639:00 +CPU=118.158
  16:51:10 STEP  640 H= 640:00 +CPU= 77.801
  16:52:30 STEP  641 H= 641:00 +CPU= 78.530
  16:54:27 STEP  642 H= 642:00 +CPU=116.001
  16:55:43 STEP  643 H= 643:00 +CPU= 75.469
  16:57:06 STEP  644 H= 644:00 +CPU= 81.856
  16:59:03 STEP  645 H= 645:00 +CPU=115.621
  17:00:25 STEP  646 H= 646:00 +CPU= 81.381
  17:01:43 STEP  647 H= 647:00 +CPU= 77.136
  17:03:58 STEP  648 H= 648:00 +CPU=133.481
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000648
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000648
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000636
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0397_2008050100_123_977_12194041_0_r319970419_26.zip
Uploading the intermediate file: upload_file_26.zip
  17:05:15 STEP  649 H= 649:00 +CPU= 76.210
  17:06:35 STEP  650 H= 650:00 +CPU= 78.516
  17:08:23 STEP  651 H= 651:00 +CPU=106.839
  17:09:36 STEP  652 H= 652:00 +CPU= 72.463
  17:10:49 STEP  653 H= 653:00 +CPU= 72.688
  17:12:38 STEP  654 H= 654:00 +CPU=107.898
  17:13:53 STEP  655 H= 655:00 +CPU= 74.164
  17:15:06 STEP  656 H= 656:00 +CPU= 71.750
  17:17:05 STEP  657 H= 657:00 +CPU=117.814
  17:18:24 STEP  658 H= 658:00 +CPU= 78.211
  17:19:35 STEP  659 H= 659:00 +CPU= 70.209
  17:21:28 STEP  660 H= 660:00 +CPU=111.485
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000660
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000660
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000660
  17:22:38 STEP  661 H= 661:00 +CPU= 69.664
  17:23:49 STEP  662 H= 662:00 +CPU= 70.055
  17:25:36 STEP  663 H= 663:00 +CPU=106.382
  17:26:52 STEP  664 H= 664:00 +CPU= 74.570
  17:28:02 STEP  665 H= 665:00 +CPU= 69.618
  17:29:44 STEP  666 H= 666:00 +CPU=101.218
  17:30:55 STEP  667 H= 667:00 +CPU= 70.038
  17:32:01 STEP  668 H= 668:00 +CPU= 65.272
  17:33:48 STEP  669 H= 669:00 +CPU=105.559
  17:34:58 STEP  670 H= 670:00 +CPU= 70.031
  17:36:12 STEP  671 H= 671:00 +CPU= 72.927
  17:38:18 STEP  672 H= 672:00 +CPU=125.586
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000672
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000672
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000660
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0397_2008050100_123_977_12194041_0_r319970419_27.zip
Uploading the intermediate file: upload_file_27.zip
  17:39:28 STEP  673 H= 673:00 +CPU= 68.402
  17:40:38 STEP  674 H= 674:00 +CPU= 69.895
  17:42:18 STEP  675 H= 675:00 +CPU= 98.611
  17:43:33 STEP  676 H= 676:00 +CPU= 74.401
  17:44:48 STEP  677 H= 677:00 +CPU= 74.405
  17:46:28 STEP  678 H= 678:00 +CPU= 99.302
  17:47:36 STEP  679 H= 679:00 +CPU= 67.479
  17:48:48 STEP  680 H= 680:00 +CPU= 70.831
  17:50:30 STEP  681 H= 681:00 +CPU=101.760
  17:51:43 STEP  682 H= 682:00 +CPU= 71.980
  17:52:51 STEP  683 H= 683:00 +CPU= 67.086
  17:54:41 STEP  684 H= 684:00 +CPU=109.276
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000684
  17:55:53 STEP  685 H= 685:00 +CPU= 71.028
  17:57:04 STEP  686 H= 686:00 +CPU= 70.462
  17:58:47 STEP  687 H= 687:00 +CPU=101.323
  17:59:56 STEP  688 H= 688:00 +CPU= 68.036
  18:01:04 STEP  689 H= 689:00 +CPU= 67.267
  18:02:44 STEP  690 H= 690:00 +CPU= 99.787
  18:03:54 STEP  691 H= 691:00 +CPU= 68.678
  18:05:05 STEP  692 H= 692:00 +CPU= 70.942
  18:06:46 STEP  693 H= 693:00 +CPU=100.122
  18:07:58 STEP  694 H= 694:00 +CPU= 70.370
  18:09:10 STEP  695 H= 695:00 +CPU= 72.309
  18:11:11 STEP  696 H= 696:00 +CPU=118.928
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000696
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000696
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000684
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0397_2008050100_123_977_12194041_0_r319970419_28.zip
Uploading the intermediate file: upload_file_28.zip
  18:12:19 STEP  697 H= 697:00 +CPU= 67.109
  18:13:30 STEP  698 H= 698:00 +CPU= 69.897
  18:15:10 STEP  699 H= 699:00 +CPU= 98.568
  18:16:21 STEP  700 H= 700:00 +CPU= 70.746
  18:17:28 STEP  701 H= 701:00 +CPU= 66.646
  18:19:10 STEP  702 H= 702:00 +CPU=100.750
  18:20:18 STEP  703 H= 703:00 +CPU= 67.632
  18:21:25 STEP  704 H= 704:00 +CPU= 66.408
  18:23:09 STEP  705 H= 705:00 +CPU=102.371
  18:24:17 STEP  706 H= 706:00 +CPU= 67.740
  18:25:25 STEP  707 H= 707:00 +CPU= 67.038
  18:27:16 STEP  708 H= 708:00 +CPU=109.620
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000708
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000708
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000708
  18:28:24 STEP  709 H= 709:00 +CPU= 68.053
  18:29:34 STEP  710 H= 710:00 +CPU= 68.806
  18:31:12 STEP  711 H= 711:00 +CPU= 97.575
  18:32:23 STEP  712 H= 712:00 +CPU= 70.009
  18:33:31 STEP  713 H= 713:00 +CPU= 66.639
  18:35:21 STEP  714 H= 714:00 +CPU=109.786
  18:36:30 STEP  715 H= 715:00 +CPU= 67.636
  18:37:40 STEP  716 H= 716:00 +CPU= 70.078
  18:39:20 STEP  717 H= 717:00 +CPU= 98.373
  18:40:30 STEP  718 H= 718:00 +CPU= 69.545
  18:41:40 STEP  719 H= 719:00 +CPU= 68.338
  18:43:41 STEP  720 H= 720:00 +CPU=120.418
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000720
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000720
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000708
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0397_2008050100_123_977_12194041_0_r319970419_29.zip
Uploading the intermediate file: upload_file_29.zip
  18:44:50 STEP  721 H= 721:00 +CPU= 68.389
  18:46:03 STEP  722 H= 722:00 +CPU= 71.926
  18:47:49 STEP  723 H= 723:00 +CPU=105.191
  18:48:58 STEP  724 H= 724:00 +CPU= 67.968
  18:50:09 STEP  725 H= 725:00 +CPU= 70.095
  18:51:53 STEP  726 H= 726:00 +CPU=102.652
  18:53:04 STEP  727 H= 727:00 +CPU= 70.808
  18:54:17 STEP  728 H= 728:00 +CPU= 71.683
  18:55:57 STEP  729 H= 729:00 +CPU= 99.036
  18:57:06 STEP  730 H= 730:00 +CPU= 68.437
  18:58:18 STEP  731 H= 731:00 +CPU= 71.664
  19:00:13 STEP  732 H= 732:00 +CPU=113.234
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000732
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000732
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000732
  19:01:20 STEP  733 H= 733:00 +CPU= 66.152
  19:02:30 STEP  734 H= 734:00 +CPU= 69.685
  19:04:10 STEP  735 H= 735:00 +CPU= 98.885
  19:05:19 STEP  736 H= 736:00 +CPU= 68.466
  19:06:27 STEP  737 H= 737:00 +CPU= 67.678
  19:08:06 STEP  738 H= 738:00 +CPU= 97.946
  19:09:14 STEP  739 H= 739:00 +CPU= 66.913
  19:10:25 STEP  740 H= 740:00 +CPU= 70.104
  19:12:06 STEP  741 H= 741:00 +CPU=100.599
  19:13:17 STEP  742 H= 742:00 +CPU= 70.060
  19:14:27 STEP  743 H= 743:00 +CPU= 69.290
  19:16:30 STEP  744 H= 744:00 +CPU=120.950
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000744
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000744
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000732
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0397_2008050100_123_977_12194041_0_r319970419_30.zip
Uploading the intermediate file: upload_file_30.zip
  19:17:38 STEP  745 H= 745:00 +CPU= 67.089
  19:18:48 STEP  746 H= 746:00 +CPU= 69.444
  19:20:31 STEP  747 H= 747:00 +CPU=101.790
  19:21:42 STEP  748 H= 748:00 +CPU= 70.372
  19:22:47 STEP  749 H= 749:00 +CPU= 64.893
  19:24:30 STEP  750 H= 750:00 +CPU=101.516
  19:25:45 STEP  751 H= 751:00 +CPU= 74.613
  19:26:54 STEP  752 H= 752:00 +CPU= 68.868
  19:28:37 STEP  753 H= 753:00 +CPU=101.372
  19:29:50 STEP  754 H= 754:00 +CPU= 72.262
  19:30:58 STEP  755 H= 755:00 +CPU= 67.684
  19:32:52 STEP  756 H= 756:00 +CPU=112.427
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000756
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000756
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000756
  19:34:02 STEP  757 H= 757:00 +CPU= 69.547
  19:35:12 STEP  758 H= 758:00 +CPU= 69.385
  19:36:50 STEP  759 H= 759:00 +CPU= 96.888
  19:38:02 STEP  760 H= 760:00 +CPU= 71.087
  19:39:10 STEP  761 H= 761:00 +CPU= 66.690
  19:40:54 STEP  762 H= 762:00 +CPU=103.149
  19:42:03 STEP  763 H= 763:00 +CPU= 68.667
  19:43:16 STEP  764 H= 764:00 +CPU= 72.223
  19:44:58 STEP  765 H= 765:00 +CPU=101.208
  19:46:09 STEP  766 H= 766:00 +CPU= 69.970
  19:47:19 STEP  767 H= 767:00 +CPU= 69.200
  19:49:24 STEP  768 H= 768:00 +CPU=122.936
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000768
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000768
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000756
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0397_2008050100_123_977_12194041_0_r319970419_31.zip
Uploading the intermediate file: upload_file_31.zip
  19:50:28 STEP  769 H= 769:00 +CPU= 63.501
  19:51:38 STEP  770 H= 770:00 +CPU= 69.862
  19:53:23 STEP  771 H= 771:00 +CPU=103.502
  19:54:33 STEP  772 H= 772:00 +CPU= 69.431
  19:55:41 STEP  773 H= 773:00 +CPU= 67.923
  19:57:24 STEP  774 H= 774:00 +CPU=101.876
  19:58:32 STEP  775 H= 775:00 +CPU= 67.252
  19:59:42 STEP  776 H= 776:00 +CPU= 69.451
  20:01:21 STEP  777 H= 777:00 +CPU= 98.117
  20:02:31 STEP  778 H= 778:00 +CPU= 69.475
  20:03:41 STEP  779 H= 779:00 +CPU= 68.705
  20:05:31 STEP  780 H= 780:00 +CPU=109.148
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000780
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000780
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000780
  20:06:41 STEP  781 H= 781:00 +CPU= 69.585
  20:07:51 STEP  782 H= 782:00 +CPU= 68.994
  20:09:33 STEP  783 H= 783:00 +CPU=101.591
  20:10:44 STEP  784 H= 784:00 +CPU= 69.540
  20:11:51 STEP  785 H= 785:00 +CPU= 66.956
  20:13:34 STEP  786 H= 786:00 +CPU=102.175
  20:14:41 STEP  787 H= 787:00 +CPU= 65.708
  20:15:50 STEP  788 H= 788:00 +CPU= 69.237
  20:17:41 STEP  789 H= 789:00 +CPU=109.699
  20:19:01 STEP  790 H= 790:00 +CPU= 79.579
  20:20:13 STEP  791 H= 791:00 +CPU= 71.166
  20:22:14 STEP  792 H= 792:00 +CPU=118.601
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000792
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000792
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000780
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0397_2008050100_123_977_12194041_0_r319970419_32.zip
Uploading the intermediate file: upload_file_32.zip
  20:23:21 STEP  793 H= 793:00 +CPU= 67.251
  20:24:33 STEP  794 H= 794:00 +CPU= 70.451
  20:26:12 STEP  795 H= 795:00 +CPU= 98.241
  20:27:23 STEP  796 H= 796:00 +CPU= 70.692
  20:28:31 STEP  797 H= 797:00 +CPU= 66.562
  20:30:10 STEP  798 H= 798:00 +CPU= 98.121
  20:31:21 STEP  799 H= 799:00 +CPU= 70.633
  20:32:30 STEP  800 H= 800:00 +CPU= 68.686
  20:34:08 STEP  801 H= 801:00 +CPU= 97.285
  20:35:21 STEP  802 H= 802:00 +CPU= 72.034
  20:36:29 STEP  803 H= 803:00 +CPU= 67.350
  20:38:18 STEP  804 H= 804:00 +CPU=108.294
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000804
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000804
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000804
  20:39:27 STEP  805 H= 805:00 +CPU= 68.050
  20:40:48 STEP  806 H= 806:00 +CPU= 80.832
  20:42:32 STEP  807 H= 807:00 +CPU=102.789
  20:43:41 STEP  808 H= 808:00 +CPU= 68.390
  20:44:49 STEP  809 H= 809:00 +CPU= 66.967
  20:46:28 STEP  810 H= 810:00 +CPU= 98.419
  20:47:37 STEP  811 H= 811:00 +CPU= 67.738
  20:48:47 STEP  812 H= 812:00 +CPU= 69.998
  20:50:35 STEP  813 H= 813:00 +CPU=106.575
  20:51:49 STEP  814 H= 814:00 +CPU= 73.729
  20:53:03 STEP  815 H= 815:00 +CPU= 73.491
  20:55:05 STEP  816 H= 816:00 +CPU=119.900
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000816
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000816
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000804
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0397_2008050100_123_977_12194041_0_r319970419_33.zip
Uploading the intermediate file: upload_file_33.zip
Uploading trickle at timestep: 2937600
  20:56:17 STEP  817 H= 817:00 +CPU= 71.691
  20:57:29 STEP  818 H= 818:00 +CPU= 71.420
  20:59:10 STEP  819 H= 819:00 +CPU= 99.356
  21:00:30 STEP  820 H= 820:00 +CPU= 78.599
  21:01:46 STEP  821 H= 821:00 +CPU= 75.100
  21:03:27 STEP  822 H= 822:00 +CPU=100.476
  21:04:35 STEP  823 H= 823:00 +CPU= 66.720
  21:05:45 STEP  824 H= 824:00 +CPU= 69.527
  21:07:27 STEP  825 H= 825:00 +CPU=100.476
  21:08:38 STEP  826 H= 826:00 +CPU= 70.594
  21:09:50 STEP  827 H= 827:00 +CPU= 71.537
  21:11:51 STEP  828 H= 828:00 +CPU=119.695
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000828
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000828
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000828
  21:13:06 STEP  829 H= 829:00 +CPU= 74.713
  21:14:18 STEP  830 H= 830:00 +CPU= 70.384
  21:16:01 STEP  831 H= 831:00 +CPU=101.827
  21:17:19 STEP  832 H= 832:00 +CPU= 77.971
  21:18:29 STEP  833 H= 833:00 +CPU= 68.881
  21:20:17 STEP  834 H= 834:00 +CPU=106.303
  21:21:30 STEP  835 H= 835:00 +CPU= 72.680
  21:22:41 STEP  836 H= 836:00 +CPU= 70.523
  21:24:31 STEP  837 H= 837:00 +CPU=108.759
  21:25:45 STEP  838 H= 838:00 +CPU= 72.769
  21:26:55 STEP  839 H= 839:00 +CPU= 69.371
  21:28:57 STEP  840 H= 840:00 +CPU=120.997
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000840
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000840
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000828
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0397_2008050100_123_977_12194041_0_r319970419_34.zip
Uploading the intermediate file: upload_file_34.zip
  21:30:09 STEP  841 H= 841:00 +CPU= 70.735
  21:31:23 STEP  842 H= 842:00 +CPU= 73.770
  21:33:08 STEP  843 H= 843:00 +CPU=102.814
  21:34:21 STEP  844 H= 844:00 +CPU= 72.527
  21:35:30 STEP  845 H= 845:00 +CPU= 68.108
  21:37:12 STEP  846 H= 846:00 +CPU=101.526
  21:38:29 STEP  847 H= 847:00 +CPU= 75.833
  21:39:41 STEP  848 H= 848:00 +CPU= 71.153
  21:41:22 STEP  849 H= 849:00 +CPU=100.547
  21:42:36 STEP  850 H= 850:00 +CPU= 72.679
  21:43:55 STEP  851 H= 851:00 +CPU= 78.862
  21:45:52 STEP  852 H= 852:00 +CPU=115.219
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000852
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000852
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000852
  21:47:08 STEP  853 H= 853:00 +CPU= 75.128
  21:48:24 STEP  854 H= 854:00 +CPU= 74.920
  21:50:06 STEP  855 H= 855:00 +CPU=100.965
  21:51:23 STEP  856 H= 856:00 +CPU= 76.676
  21:52:33 STEP  857 H= 857:00 +CPU= 69.615
  21:54:23 STEP  858 H= 858:00 +CPU=108.902
  21:55:46 STEP  859 H= 859:00 +CPU= 81.949
  21:57:06 STEP  860 H= 860:00 +CPU= 79.091
  21:58:56 STEP  861 H= 861:00 +CPU=108.576
  22:00:12 STEP  862 H= 862:00 +CPU= 75.094
  22:01:27 STEP  863 H= 863:00 +CPU= 74.120
  22:03:36 STEP  864 H= 864:00 +CPU=127.331
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000864
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000864
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000852
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0397_2008050100_123_977_12194041_0_r319970419_35.zip
Uploading the intermediate file: upload_file_35.zip
  22:04:54 STEP  865 H= 865:00 +CPU= 77.736
  22:06:16 STEP  866 H= 866:00 +CPU= 80.604
  22:08:08 STEP  867 H= 867:00 +CPU=111.191
  22:09:25 STEP  868 H= 868:00 +CPU= 76.088
  22:10:38 STEP  869 H= 869:00 +CPU= 72.130
  22:12:25 STEP  870 H= 870:00 +CPU=105.355
  22:13:42 STEP  871 H= 871:00 +CPU= 76.747
  22:15:02 STEP  872 H= 872:00 +CPU= 78.484
  22:16:53 STEP  873 H= 873:00 +CPU=110.575
  22:18:09 STEP  874 H= 874:00 +CPU= 74.872
  22:19:31 STEP  875 H= 875:00 +CPU= 81.151
  22:21:40 STEP  876 H= 876:00 +CPU=127.871
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000876
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000876
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000876
  22:23:02 STEP  877 H= 877:00 +CPU= 80.248
  22:24:20 STEP  878 H= 878:00 +CPU= 77.405
  22:26:13 STEP  879 H= 879:00 +CPU=111.836
  22:27:29 STEP  880 H= 880:00 +CPU= 75.999
  22:28:50 STEP  881 H= 881:00 +CPU= 79.544
  22:30:45 STEP  882 H= 882:00 +CPU=113.846
  22:31:59 STEP  883 H= 883:00 +CPU= 73.443
  22:33:17 STEP  884 H= 884:00 +CPU= 76.899
  22:35:18 STEP  885 H= 885:00 +CPU=119.781
  22:36:39 STEP  886 H= 886:00 +CPU= 79.901
  22:37:53 STEP  887 H= 887:00 +CPU= 73.951
  22:40:05 STEP  888 H= 888:00 +CPU=129.999
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000888
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000888
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000876
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0397_2008050100_123_977_12194041_0_r319970419_36.zip
Uploading the intermediate file: upload_file_36.zip
  22:41:21 STEP  889 H= 889:00 +CPU= 76.094
  22:42:44 STEP  890 H= 890:00 +CPU= 81.672
  22:44:33 STEP  891 H= 891:00 +CPU=107.778
  22:45:55 STEP  892 H= 892:00 +CPU= 80.510
  22:47:12 STEP  893 H= 893:00 +CPU= 75.769
  22:49:02 STEP  894 H= 894:00 +CPU=109.573
  22:50:25 STEP  895 H= 895:00 +CPU= 81.733
  22:51:40 STEP  896 H= 896:00 +CPU= 74.413
  22:53:28 STEP  897 H= 897:00 +CPU=106.231
  22:54:43 STEP  898 H= 898:00 +CPU= 74.884
  22:56:03 STEP  899 H= 899:00 +CPU= 79.283
  22:58:12 STEP  900 H= 900:00 +CPU=127.327
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000900
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000900
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000900
  22:59:28 STEP  901 H= 901:00 +CPU= 75.192
  23:00:52 STEP  902 H= 902:00 +CPU= 82.719
  23:02:54 STEP  903 H= 903:00 +CPU=121.362
  23:04:13 STEP  904 H= 904:00 +CPU= 78.358
  23:05:48 STEP  905 H= 905:00 +CPU= 93.578
  23:07:43 STEP  906 H= 906:00 +CPU=114.145
  23:09:04 STEP  907 H= 907:00 +CPU= 80.014
  23:10:23 STEP  908 H= 908:00 +CPU= 78.441
  23:12:18 STEP  909 H= 909:00 +CPU=113.233
  23:13:39 STEP  910 H= 910:00 +CPU= 80.196
  23:15:00 STEP  911 H= 911:00 +CPU= 79.390
  23:17:19 STEP  912 H= 912:00 +CPU=137.620
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000912
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000912
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMGGhq0f+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMSHhq0f+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194041/ICMUAhq0f+000900
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0397_2008050100_123_977_12194041_0_r319970419_37.zip
Uploading the intermediate file: upload_file_37.zip
  23:18:44 STEP  913 H= 913:00 +CPU= 83.714
  23:19:57 STEP  914 H= 914:00 +CPU= 72.594
  23:21:55 STEP  915 H= 915:00 +CPU=117.222
  23:23:15 STEP  916 H= 916:00 +CPU= 78.502
  23:24:36 STEP  917 H= 917:00 +CPU= 80.221
  23:26:38 STEP  918 H= 918:00 +CPU=120.967
  23:27:54 STEP  919 H= 919:00 +CPU= 75.040
  23:29:17 STEP  920 H= 920:00 +CPU= 81.919
  23:31:11 STEP  921 H= 921:00 +CPU=112.953
  23:32:31 STEP  922 H= 922:00 +CPU= 79.466
  23:33:48 STEP  923 H= 923:00 +CPU= 75.802
  23:35:47 STEP  924 H= 924:00 +CPU=117.377
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMGGhq0f+000924
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMSHhq0f+000924
Moving to projects directory: /var/lib/boinc-client/slots/37/ICMUAhq0f+000924
  23:37:02 STEP  925 H= 925:00 +CPU= 74.157
  23:38:16 STEP  926 H= 926:00 +CPU= 73.449
  23:39:59 STEP  927 H= 927:00 +CPU=102.852
  23:41:22 STEP  928 H= 928:00 +CPU= 81.785
  23:42:31 STEP  929 H= 929:00 +CPU= 68.335
  23:44:19 STEP  930 H= 930:00 +CPU=107.505
  23:45:34 STEP  931 H= 931:00 +CPU= 74.105
  23:46:52 STEP  932 H= 932:00 +CPU= 76.260
  23:48:43 STEP  933 H= 933:00 +CPU=110.699
  23:50:00 STEP  934 H= 934:00 +CPU= 75.344
  23:51:16 STEP  935 H= 935:00 +CPU= 76.002
Abort request received from BOINC client, ending the child process

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:16:16 1537299 22279861 oifs_43r3_ps_0397_2008050100_123_977_12194041_0 2,937,600 45,305 0.0154


©2024 cpdn.org