| Name |
oifs_43r3_ps_0343_2000050100_123_969_12185987_1 |
| Workunit |
12185987 |
| Created |
25 Jan 2023, 11:10:55 UTC |
| Sent |
25 Jan 2023, 11:44:12 UTC |
| Report deadline |
26 Mar 2023, 11:44:12 UTC |
| Received |
16 Feb 2023, 15:32:32 UTC |
| Server state |
Over |
| Outcome |
Computation error |
| Client state |
Compute error |
| Exit status |
1 (0x00000001) Unknown error code |
| Computer ID |
1526341 |
| Run time |
7 hours 6 min 9 sec |
| CPU time |
7 hours 1 min 16 sec |
| Validate state |
Invalid |
| Credit |
2,419.20 |
| Device peak FLOPS |
3.66 GFLOPS |
| Application version |
OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu |
| Peak working set size |
4,395.68 MB |
| Peak swap size |
4,974.33 MB |
| Peak disk usage |
1,210.12 MB |
| Stderr |
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2000050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0343
(argv4) batchid: 969
(argv5) wuid: 12185987
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /home/tommcfarland/slots/0
Project directory is: /home/tommcfarland/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987
Copying: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /home/tommcfarland/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /home/tommcfarland/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_114b3655ac40a6d6882aebbafb71a05d to: /home/tommcfarland/slots/0/oifs_43r3_ps_0343_2000050100_123_969_12185987.zip
Unzipping the namelist zip file: /home/tommcfarland/slots/0/oifs_43r3_ps_0343_2000050100_123_969_12185987.zip
ic_ancil_file: ic_ancil_12185987
ifsdata_file: ifsdata_12185987
climate_data_file: clim_data_12185987
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_2fd1d72dbded281c5c03cf2a203c4b30 to: /home/tommcfarland/slots/0/ic_ancil_12185987.zip
Unzipping the IC ancils zip file: /home/tommcfarland/slots/0/ic_ancil_12185987.zip
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_8a2ab00809c2846c4dce09a660502aec to: /home/tommcfarland/slots/0/ifsdata/ifsdata_12185987.zip
Unzipping the ifsdata_zip file: /home/tommcfarland/slots/0/ifsdata/ifsdata_12185987.zip
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /home/tommcfarland/slots/0/159l_2/clim_data_12185987.zip
Unzipping the climate data zip file: /home/tommcfarland/slots/0/159l_2/clim_data_12185987.zip
Checking for progress XML file: /home/tommcfarland/slots/0/progress_file_12185987.xml
Creating progress file: /home/tommcfarland/slots/0/progress_file_12185987.xml
last_cpu_time: 0
upload_file_number: 0
last_iter: 0
last_upload: 0
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0343_2000050100_123_969_12185987_1_r1360474992
The child process has been launched with process id: 4862
Executing the command: /home/tommcfarland/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:tommcfarland-FX8350-Linux:1:1:4862:4862] [20230215:221610:1676517370:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:46:22 STEP 0 H= 0:00 +CPU= 63.757
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000000
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000000
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000000
01:47:08 STEP 1 H= 1:00 +CPU= 44.320
01:47:48 STEP 2 H= 2:00 +CPU= 38.815
01:48:49 STEP 3 H= 3:00 +CPU= 59.947
01:49:29 STEP 4 H= 4:00 +CPU= 39.975
01:50:07 STEP 5 H= 5:00 +CPU= 37.558
01:51:08 STEP 6 H= 6:00 +CPU= 60.836
01:51:47 STEP 7 H= 7:00 +CPU= 38.884
01:52:26 STEP 8 H= 8:00 +CPU= 39.283
01:53:25 STEP 9 H= 9:00 +CPU= 58.701
01:54:05 STEP 10 H= 10:00 +CPU= 39.679
01:54:43 STEP 11 H= 11:00 +CPU= 37.777
01:55:49 STEP 12 H= 12:00 +CPU= 65.259
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000012
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000012
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000012
01:56:28 STEP 13 H= 13:00 +CPU= 38.241
01:57:08 STEP 14 H= 14:00 +CPU= 39.031
01:58:09 STEP 15 H= 15:00 +CPU= 59.900
01:58:49 STEP 16 H= 16:00 +CPU= 39.327
01:59:27 STEP 17 H= 17:00 +CPU= 38.087
02:00:29 STEP 18 H= 18:00 +CPU= 61.445
02:01:08 STEP 19 H= 19:00 +CPU= 38.543
02:01:46 STEP 20 H= 20:00 +CPU= 38.170
02:02:46 STEP 21 H= 21:00 +CPU= 59.831
02:03:26 STEP 22 H= 22:00 +CPU= 39.134
02:04:05 STEP 23 H= 23:00 +CPU= 38.562
02:05:17 STEP 24 H= 24:00 +CPU= 70.243
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000024
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000024
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000024
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000000
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000000
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000000
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000012
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000012
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000012
Zipping up the intermediate file: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_0343_2000050100_123_969_12185987_1_r1360474992_0.zip
Uploading the intermediate file: upload_file_0.zip
02:05:56 STEP 25 H= 25:00 +CPU= 38.626
02:06:36 STEP 26 H= 26:00 +CPU= 39.161
02:07:36 STEP 27 H= 27:00 +CPU= 59.846
02:08:15 STEP 28 H= 28:00 +CPU= 39.138
02:08:54 STEP 29 H= 29:00 +CPU= 38.030
02:09:54 STEP 30 H= 30:00 +CPU= 59.743
02:10:32 STEP 31 H= 31:00 +CPU= 37.873
02:11:12 STEP 32 H= 32:00 +CPU= 39.363
02:12:12 STEP 33 H= 33:00 +CPU= 60.029
02:12:52 STEP 34 H= 34:00 +CPU= 39.676
02:13:31 STEP 35 H= 35:00 +CPU= 38.394
02:14:38 STEP 36 H= 36:00 +CPU= 66.540
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000036
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000036
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000036
02:15:17 STEP 37 H= 37:00 +CPU= 38.335
03:43:40 STEP 38 H= 38:00 +CPU=116.692
03:44:35 STEP 39 H= 39:00 +CPU= 53.848
03:45:16 STEP 40 H= 40:00 +CPU= 39.509
03:45:54 STEP 41 H= 41:00 +CPU= 37.543
03:46:55 STEP 42 H= 42:00 +CPU= 60.437
03:47:34 STEP 43 H= 43:00 +CPU= 38.681
03:48:13 STEP 44 H= 44:00 +CPU= 38.461
03:49:13 STEP 45 H= 45:00 +CPU= 59.876
03:49:54 STEP 46 H= 46:00 +CPU= 40.325
03:50:33 STEP 47 H= 47:00 +CPU= 38.847
03:51:45 STEP 48 H= 48:00 +CPU= 69.315
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000048
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000048
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000048
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000024
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000024
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000024
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000036
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000036
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000036
Zipping up the intermediate file: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_0343_2000050100_123_969_12185987_1_r1360474992_1.zip
Uploading the intermediate file: upload_file_1.zip
03:52:24 STEP 49 H= 49:00 +CPU= 38.335
03:53:04 STEP 50 H= 50:00 +CPU= 38.915
03:54:05 STEP 51 H= 51:00 +CPU= 60.598
03:54:45 STEP 52 H= 52:00 +CPU= 39.619
03:55:23 STEP 53 H= 53:00 +CPU= 37.618
03:56:24 STEP 54 H= 54:00 +CPU= 60.903
03:57:03 STEP 55 H= 55:00 +CPU= 38.745
03:57:42 STEP 56 H= 56:00 +CPU= 38.734
03:58:44 STEP 57 H= 57:00 +CPU= 61.005
03:59:23 STEP 58 H= 58:00 +CPU= 39.336
04:00:01 STEP 59 H= 59:00 +CPU= 37.662
04:01:08 STEP 60 H= 60:00 +CPU= 65.904
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000060
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000060
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000060
04:01:46 STEP 61 H= 61:00 +CPU= 37.806
04:02:27 STEP 62 H= 62:00 +CPU= 39.275
04:03:28 STEP 63 H= 63:00 +CPU= 59.888
04:04:08 STEP 64 H= 64:00 +CPU= 39.517
04:04:47 STEP 65 H= 65:00 +CPU= 38.470
04:05:48 STEP 66 H= 66:00 +CPU= 60.928
04:06:27 STEP 67 H= 67:00 +CPU= 37.830
04:07:06 STEP 68 H= 68:00 +CPU= 39.010
04:08:06 STEP 69 H= 69:00 +CPU= 59.340
04:08:46 STEP 70 H= 70:00 +CPU= 39.396
04:09:25 STEP 71 H= 71:00 +CPU= 38.803
04:10:36 STEP 72 H= 72:00 +CPU= 69.994
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000072
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000072
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000072
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000048
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000048
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000048
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000060
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000060
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000060
Zipping up the intermediate file: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_0343_2000050100_123_969_12185987_1_r1360474992_2.zip
Uploading the intermediate file: upload_file_2.zip
04:11:16 STEP 73 H= 73:00 +CPU= 39.108
04:11:55 STEP 74 H= 74:00 +CPU= 38.868
04:12:56 STEP 75 H= 75:00 +CPU= 59.902
04:13:35 STEP 76 H= 76:00 +CPU= 39.642
04:14:13 STEP 77 H= 77:00 +CPU= 36.954
04:15:13 STEP 78 H= 78:00 +CPU= 60.228
04:15:52 STEP 79 H= 79:00 +CPU= 38.683
04:16:31 STEP 80 H= 80:00 +CPU= 38.851
04:17:31 STEP 81 H= 81:00 +CPU= 59.442
04:18:10 STEP 82 H= 82:00 +CPU= 38.579
04:18:48 STEP 83 H= 83:00 +CPU= 38.179
04:19:55 STEP 84 H= 84:00 +CPU= 66.324
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000084
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000084
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000084
04:20:34 STEP 85 H= 85:00 +CPU= 38.478
04:21:14 STEP 86 H= 86:00 +CPU= 39.140
04:22:14 STEP 87 H= 87:00 +CPU= 59.414
04:22:55 STEP 88 H= 88:00 +CPU= 39.044
04:23:38 STEP 89 H= 89:00 +CPU= 36.627
04:24:44 STEP 90 H= 90:00 +CPU= 57.933
04:25:26 STEP 91 H= 91:00 +CPU= 38.376
04:26:05 STEP 92 H= 92:00 +CPU= 39.116
04:27:06 STEP 93 H= 93:00 +CPU= 59.927
04:27:46 STEP 94 H= 94:00 +CPU= 39.213
04:28:24 STEP 95 H= 95:00 +CPU= 38.567
04:29:35 STEP 96 H= 96:00 +CPU= 69.233
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000096
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000096
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000096
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000072
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000072
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000072
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000084
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000084
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000084
Zipping up the intermediate file: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_0343_2000050100_123_969_12185987_1_r1360474992_3.zip
Uploading the intermediate file: upload_file_3.zip
04:30:14 STEP 97 H= 97:00 +CPU= 38.338
04:30:54 STEP 98 H= 98:00 +CPU= 39.300
04:31:54 STEP 99 H= 99:00 +CPU= 60.064
04:32:34 STEP 100 H= 100:00 +CPU= 39.304
04:33:11 STEP 101 H= 101:00 +CPU= 37.610
04:34:13 STEP 102 H= 102:00 +CPU= 61.363
04:34:52 STEP 103 H= 103:00 +CPU= 38.952
04:35:31 STEP 104 H= 104:00 +CPU= 38.217
04:36:32 STEP 105 H= 105:00 +CPU= 61.234
04:37:11 STEP 106 H= 106:00 +CPU= 39.047
04:37:50 STEP 107 H= 107:00 +CPU= 38.087
04:38:57 STEP 108 H= 108:00 +CPU= 65.688
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000108
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000108
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000108
04:39:36 STEP 109 H= 109:00 +CPU= 38.232
04:40:15 STEP 110 H= 110:00 +CPU= 38.809
04:41:16 STEP 111 H= 111:00 +CPU= 59.505
04:41:56 STEP 112 H= 112:00 +CPU= 39.717
04:42:33 STEP 113 H= 113:00 +CPU= 37.133
04:43:34 STEP 114 H= 114:00 +CPU= 59.984
04:44:12 STEP 115 H= 115:00 +CPU= 37.859
04:44:51 STEP 116 H= 116:00 +CPU= 38.977
04:45:51 STEP 117 H= 117:00 +CPU= 58.862
04:46:31 STEP 118 H= 118:00 +CPU= 39.603
04:47:10 STEP 119 H= 119:00 +CPU= 39.116
04:48:22 STEP 120 H= 120:00 +CPU= 70.832
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000120
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000120
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000120
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000096
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000096
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000096
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000108
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000108
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000108
Zipping up the intermediate file: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_0343_2000050100_123_969_12185987_1_r1360474992_4.zip
Uploading the intermediate file: upload_file_4.zip
04:49:02 STEP 121 H= 121:00 +CPU= 38.762
04:49:41 STEP 122 H= 122:00 +CPU= 38.677
04:50:41 STEP 123 H= 123:00 +CPU= 59.877
04:51:20 STEP 124 H= 124:00 +CPU= 39.096
04:51:59 STEP 125 H= 125:00 +CPU= 38.159
04:53:01 STEP 126 H= 126:00 +CPU= 61.532
04:53:39 STEP 127 H= 127:00 +CPU= 38.148
04:54:18 STEP 128 H= 128:00 +CPU= 38.721
04:55:17 STEP 129 H= 129:00 +CPU= 59.458
04:55:57 STEP 130 H= 130:00 +CPU= 39.423
04:56:36 STEP 131 H= 131:00 +CPU= 38.811
04:57:42 STEP 132 H= 132:00 +CPU= 65.526
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000132
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000132
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000132
04:58:22 STEP 133 H= 133:00 +CPU= 39.457
04:59:01 STEP 134 H= 134:00 +CPU= 38.848
05:00:02 STEP 135 H= 135:00 +CPU= 59.346
05:00:41 STEP 136 H= 136:00 +CPU= 39.107
05:01:20 STEP 137 H= 137:00 +CPU= 38.449
05:02:21 STEP 138 H= 138:00 +CPU= 60.504
05:02:59 STEP 139 H= 139:00 +CPU= 37.878
05:03:38 STEP 140 H= 140:00 +CPU= 38.606
05:04:38 STEP 141 H= 141:00 +CPU= 59.636
05:05:18 STEP 142 H= 142:00 +CPU= 39.383
05:05:58 STEP 143 H= 143:00 +CPU= 39.628
05:07:08 STEP 144 H= 144:00 +CPU= 68.893
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000144
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000144
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000144
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000120
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000120
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000120
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000132
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000132
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000132
Zipping up the intermediate file: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_0343_2000050100_123_969_12185987_1_r1360474992_5.zip
Uploading the intermediate file: upload_file_5.zip
05:07:47 STEP 145 H= 145:00 +CPU= 38.191
05:08:26 STEP 146 H= 146:00 +CPU= 38.522
05:09:25 STEP 147 H= 147:00 +CPU= 59.015
05:10:04 STEP 148 H= 148:00 +CPU= 39.150
05:10:42 STEP 149 H= 149:00 +CPU= 37.697
05:11:44 STEP 150 H= 150:00 +CPU= 61.018
05:12:23 STEP 151 H= 151:00 +CPU= 38.788
05:13:02 STEP 152 H= 152:00 +CPU= 39.176
05:14:02 STEP 153 H= 153:00 +CPU= 59.650
05:14:42 STEP 154 H= 154:00 +CPU= 39.688
05:15:20 STEP 155 H= 155:00 +CPU= 38.168
05:16:27 STEP 156 H= 156:00 +CPU= 65.938
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000156
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000156
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000156
05:17:06 STEP 157 H= 157:00 +CPU= 38.370
05:17:45 STEP 158 H= 158:00 +CPU= 38.804
05:18:46 STEP 159 H= 159:00 +CPU= 59.847
05:19:26 STEP 160 H= 160:00 +CPU= 39.435
05:20:05 STEP 161 H= 161:00 +CPU= 38.211
05:21:06 STEP 162 H= 162:00 +CPU= 60.569
05:21:45 STEP 163 H= 163:00 +CPU= 39.172
05:22:25 STEP 164 H= 164:00 +CPU= 39.480
05:23:25 STEP 165 H= 165:00 +CPU= 59.811
05:24:04 STEP 166 H= 166:00 +CPU= 39.449
05:24:44 STEP 167 H= 167:00 +CPU= 39.494
05:25:55 STEP 168 H= 168:00 +CPU= 70.021
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000168
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000168
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000168
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000144
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000144
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000144
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000156
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000156
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000156
Zipping up the intermediate file: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_0343_2000050100_123_969_12185987_1_r1360474992_6.zip
Uploading the intermediate file: upload_file_6.zip
05:26:34 STEP 169 H= 169:00 +CPU= 38.502
05:27:13 STEP 170 H= 170:00 +CPU= 38.760
05:28:13 STEP 171 H= 171:00 +CPU= 59.056
05:28:52 STEP 172 H= 172:00 +CPU= 39.510
05:29:30 STEP 173 H= 173:00 +CPU= 37.759
05:30:31 STEP 174 H= 174:00 +CPU= 59.377
05:31:08 STEP 175 H= 175:00 +CPU= 37.547
05:31:47 STEP 176 H= 176:00 +CPU= 38.933
05:32:47 STEP 177 H= 177:00 +CPU= 59.375
05:33:26 STEP 178 H= 178:00 +CPU= 38.717
05:34:06 STEP 179 H= 179:00 +CPU= 39.141
05:35:13 STEP 180 H= 180:00 +CPU= 66.217
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000180
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000180
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000180
05:35:52 STEP 181 H= 181:00 +CPU= 38.858
05:36:32 STEP 182 H= 182:00 +CPU= 38.932
05:37:32 STEP 183 H= 183:00 +CPU= 59.522
05:38:13 STEP 184 H= 184:00 +CPU= 39.984
05:38:51 STEP 185 H= 185:00 +CPU= 37.753
05:39:52 STEP 186 H= 186:00 +CPU= 60.369
05:40:30 STEP 187 H= 187:00 +CPU= 38.160
05:41:09 STEP 188 H= 188:00 +CPU= 38.871
05:42:08 STEP 189 H= 189:00 +CPU= 58.183
05:42:47 STEP 190 H= 190:00 +CPU= 38.554
05:43:27 STEP 191 H= 191:00 +CPU= 40.038
05:44:38 STEP 192 H= 192:00 +CPU= 69.704
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000192
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000192
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000192
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000168
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000168
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000168
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000180
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000180
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000180
Zipping up the intermediate file: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_0343_2000050100_123_969_12185987_1_r1360474992_7.zip
Uploading the intermediate file: upload_file_7.zip
05:45:17 STEP 193 H= 193:00 +CPU= 38.057
05:45:57 STEP 194 H= 194:00 +CPU= 39.077
05:46:57 STEP 195 H= 195:00 +CPU= 59.137
05:47:37 STEP 196 H= 196:00 +CPU= 40.186
05:48:16 STEP 197 H= 197:00 +CPU= 38.871
05:49:17 STEP 198 H= 198:00 +CPU= 60.762
05:49:56 STEP 199 H= 199:00 +CPU= 38.426
05:50:35 STEP 200 H= 200:00 +CPU= 38.941
05:51:35 STEP 201 H= 201:00 +CPU= 59.038
05:52:15 STEP 202 H= 202:00 +CPU= 39.967
05:52:53 STEP 203 H= 203:00 +CPU= 37.918
05:54:00 STEP 204 H= 204:00 +CPU= 66.286
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000204
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000204
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000204
05:54:39 STEP 205 H= 205:00 +CPU= 38.557
05:55:19 STEP 206 H= 206:00 +CPU= 39.369
05:56:19 STEP 207 H= 207:00 +CPU= 59.210
05:56:59 STEP 208 H= 208:00 +CPU= 39.544
05:57:38 STEP 209 H= 209:00 +CPU= 38.275
05:58:39 STEP 210 H= 210:00 +CPU= 60.118
05:59:17 STEP 211 H= 211:00 +CPU= 37.966
05:59:57 STEP 212 H= 212:00 +CPU= 39.435
06:00:56 STEP 213 H= 213:00 +CPU= 59.110
06:01:36 STEP 214 H= 214:00 +CPU= 39.252
06:02:16 STEP 215 H= 215:00 +CPU= 39.686
06:03:26 STEP 216 H= 216:00 +CPU= 69.565
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000216
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000216
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000216
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000192
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000192
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000192
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000204
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000204
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000204
Zipping up the intermediate file: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_0343_2000050100_123_969_12185987_1_r1360474992_8.zip
Uploading the intermediate file: upload_file_8.zip
06:04:05 STEP 217 H= 217:00 +CPU= 37.779
06:04:45 STEP 218 H= 218:00 +CPU= 39.913
06:05:45 STEP 219 H= 219:00 +CPU= 59.594
06:06:25 STEP 220 H= 220:00 +CPU= 39.447
06:07:03 STEP 221 H= 221:00 +CPU= 37.663
06:08:04 STEP 222 H= 222:00 +CPU= 60.266
06:08:43 STEP 223 H= 223:00 +CPU= 39.212
06:09:23 STEP 224 H= 224:00 +CPU= 39.867
06:10:23 STEP 225 H= 225:00 +CPU= 59.413
06:11:04 STEP 226 H= 226:00 +CPU= 40.479
06:11:42 STEP 227 H= 227:00 +CPU= 38.143
06:12:48 STEP 228 H= 228:00 +CPU= 64.949
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000228
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000228
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000228
06:13:26 STEP 229 H= 229:00 +CPU= 38.069
06:14:06 STEP 230 H= 230:00 +CPU= 39.343
06:15:06 STEP 231 H= 231:00 +CPU= 58.780
06:15:45 STEP 232 H= 232:00 +CPU= 39.216
06:16:24 STEP 233 H= 233:00 +CPU= 38.122
06:17:25 STEP 234 H= 234:00 +CPU= 60.199
06:18:04 STEP 235 H= 235:00 +CPU= 39.056
06:18:44 STEP 236 H= 236:00 +CPU= 39.289
06:19:43 STEP 237 H= 237:00 +CPU= 58.851
06:20:23 STEP 238 H= 238:00 +CPU= 39.943
06:21:04 STEP 239 H= 239:00 +CPU= 40.013
06:22:14 STEP 240 H= 240:00 +CPU= 69.599
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000240
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000240
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000240
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000216
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000216
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000216
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000228
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000228
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000228
Zipping up the intermediate file: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_0343_2000050100_123_969_12185987_1_r1360474992_9.zip
Uploading the intermediate file: upload_file_9.zip
Uploading trickle at timestep: 864000
06:22:54 STEP 241 H= 241:00 +CPU= 38.765
06:23:34 STEP 242 H= 242:00 +CPU= 39.802
06:24:35 STEP 243 H= 243:00 +CPU= 59.633
06:25:15 STEP 244 H= 244:00 +CPU= 40.050
06:25:54 STEP 245 H= 245:00 +CPU= 38.678
06:27:02 STEP 246 H= 246:00 +CPU= 58.802
06:27:43 STEP 247 H= 247:00 +CPU= 37.140
06:28:26 STEP 248 H= 248:00 +CPU= 39.077
06:29:32 STEP 249 H= 249:00 +CPU= 59.381
06:30:15 STEP 250 H= 250:00 +CPU= 39.214
06:30:58 STEP 251 H= 251:00 +CPU= 37.786
06:32:09 STEP 252 H= 252:00 +CPU= 65.475
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000252
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000252
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000252
06:32:52 STEP 253 H= 253:00 +CPU= 38.619
06:33:34 STEP 254 H= 254:00 +CPU= 38.829
06:34:39 STEP 255 H= 255:00 +CPU= 58.078
06:35:23 STEP 256 H= 256:00 +CPU= 38.372
06:36:07 STEP 257 H= 257:00 +CPU= 37.291
06:37:12 STEP 258 H= 258:00 +CPU= 59.148
06:37:52 STEP 259 H= 259:00 +CPU= 38.755
06:38:32 STEP 260 H= 260:00 +CPU= 38.910
06:39:32 STEP 261 H= 261:00 +CPU= 58.902
06:40:12 STEP 262 H= 262:00 +CPU= 39.726
06:40:51 STEP 263 H= 263:00 +CPU= 39.352
06:42:05 STEP 264 H= 264:00 +CPU= 69.986
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000264
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000264
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000264
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000240
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000240
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000240
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000252
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000252
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000252
Zipping up the intermediate file: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_0343_2000050100_123_969_12185987_1_r1360474992_10.zip
Uploading the intermediate file: upload_file_10.zip
06:42:44 STEP 265 H= 265:00 +CPU= 37.828
06:43:24 STEP 266 H= 266:00 +CPU= 39.433
06:44:24 STEP 267 H= 267:00 +CPU= 59.991
06:45:04 STEP 268 H= 268:00 +CPU= 39.077
06:45:42 STEP 269 H= 269:00 +CPU= 38.485
06:46:43 STEP 270 H= 270:00 +CPU= 60.637
06:47:22 STEP 271 H= 271:00 +CPU= 38.211
06:48:01 STEP 272 H= 272:00 +CPU= 39.099
06:49:00 STEP 273 H= 273:00 +CPU= 58.727
06:49:39 STEP 274 H= 274:00 +CPU= 38.566
06:50:17 STEP 275 H= 275:00 +CPU= 38.599
06:51:24 STEP 276 H= 276:00 +CPU= 65.956
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000276
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000276
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000276
06:52:03 STEP 277 H= 277:00 +CPU= 38.482
06:52:43 STEP 278 H= 278:00 +CPU= 38.967
06:53:43 STEP 279 H= 279:00 +CPU= 58.954
06:54:23 STEP 280 H= 280:00 +CPU= 39.717
06:55:02 STEP 281 H= 281:00 +CPU= 38.352
06:56:03 STEP 282 H= 282:00 +CPU= 60.544
06:56:42 STEP 283 H= 283:00 +CPU= 38.317
06:57:22 STEP 284 H= 284:00 +CPU= 39.631
06:58:23 STEP 285 H= 285:00 +CPU= 59.607
06:59:02 STEP 286 H= 286:00 +CPU= 39.246
06:59:42 STEP 287 H= 287:00 +CPU= 39.214
07:00:52 STEP 288 H= 288:00 +CPU= 69.352
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000288
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000288
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000288
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000264
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000264
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000264
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000276
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000276
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000276
Zipping up the intermediate file: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_0343_2000050100_123_969_12185987_1_r1360474992_11.zip
Uploading the intermediate file: upload_file_11.zip
07:01:31 STEP 289 H= 289:00 +CPU= 38.418
07:02:10 STEP 290 H= 290:00 +CPU= 39.117
07:03:10 STEP 291 H= 291:00 +CPU= 58.932
07:03:50 STEP 292 H= 292:00 +CPU= 39.558
07:04:29 STEP 293 H= 293:00 +CPU= 38.592
07:05:28 STEP 294 H= 294:00 +CPU= 59.410
07:06:07 STEP 295 H= 295:00 +CPU= 38.350
07:06:47 STEP 296 H= 296:00 +CPU= 39.801
07:07:46 STEP 297 H= 297:00 +CPU= 59.367
07:08:26 STEP 298 H= 298:00 +CPU= 39.093
07:09:04 STEP 299 H= 299:00 +CPU= 38.576
07:10:11 STEP 300 H= 300:00 +CPU= 65.660
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000300
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000300
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000300
07:10:50 STEP 301 H= 301:00 +CPU= 38.407
07:11:30 STEP 302 H= 302:00 +CPU= 39.659
07:12:31 STEP 303 H= 303:00 +CPU= 59.214
07:13:11 STEP 304 H= 304:00 +CPU= 39.977
07:13:51 STEP 305 H= 305:00 +CPU= 38.960
07:14:52 STEP 306 H= 306:00 +CPU= 60.411
07:15:31 STEP 307 H= 307:00 +CPU= 38.269
07:16:12 STEP 308 H= 308:00 +CPU= 40.374
07:17:11 STEP 309 H= 309:00 +CPU= 59.030
07:17:52 STEP 310 H= 310:00 +CPU= 40.329
07:18:32 STEP 311 H= 311:00 +CPU= 39.471
07:19:42 STEP 312 H= 312:00 +CPU= 69.926
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000312
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000312
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000312
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000288
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000288
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000288
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000300
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000300
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000300
Zipping up the intermediate file: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_0343_2000050100_123_969_12185987_1_r1360474992_12.zip
Uploading the intermediate file: upload_file_12.zip
07:20:21 STEP 313 H= 313:00 +CPU= 38.123
07:21:01 STEP 314 H= 314:00 +CPU= 39.279
07:22:00 STEP 315 H= 315:00 +CPU= 58.951
07:22:40 STEP 316 H= 316:00 +CPU= 39.144
07:23:18 STEP 317 H= 317:00 +CPU= 37.797
07:24:17 STEP 318 H= 318:00 +CPU= 59.208
07:24:57 STEP 319 H= 319:00 +CPU= 39.196
07:25:37 STEP 320 H= 320:00 +CPU= 39.798
07:26:37 STEP 321 H= 321:00 +CPU= 59.046
07:27:17 STEP 322 H= 322:00 +CPU= 40.456
07:27:57 STEP 323 H= 323:00 +CPU= 38.767
07:29:03 STEP 324 H= 324:00 +CPU= 65.215
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000324
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000324
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000324
07:29:42 STEP 325 H= 325:00 +CPU= 38.725
07:30:23 STEP 326 H= 326:00 +CPU= 39.971
07:31:23 STEP 327 H= 327:00 +CPU= 59.211
07:32:03 STEP 328 H= 328:00 +CPU= 40.059
07:32:43 STEP 329 H= 329:00 +CPU= 39.075
07:33:43 STEP 330 H= 330:00 +CPU= 59.798
07:34:21 STEP 331 H= 331:00 +CPU= 38.343
07:35:01 STEP 332 H= 332:00 +CPU= 39.784
07:36:01 STEP 333 H= 333:00 +CPU= 59.653
07:36:41 STEP 334 H= 334:00 +CPU= 39.306
07:37:21 STEP 335 H= 335:00 +CPU= 39.766
07:38:33 STEP 336 H= 336:00 +CPU= 69.862
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000336
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000336
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000336
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000312
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000312
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000312
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000324
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000324
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000324
Zipping up the intermediate file: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_0343_2000050100_123_969_12185987_1_r1360474992_13.zip
Uploading the intermediate file: upload_file_13.zip
07:39:11 STEP 337 H= 337:00 +CPU= 37.771
07:39:51 STEP 338 H= 338:00 +CPU= 39.086
07:40:50 STEP 339 H= 339:00 +CPU= 59.312
07:41:30 STEP 340 H= 340:00 +CPU= 39.165
07:42:09 STEP 341 H= 341:00 +CPU= 39.221
07:43:09 STEP 342 H= 342:00 +CPU= 59.793
07:43:48 STEP 343 H= 343:00 +CPU= 38.692
07:44:28 STEP 344 H= 344:00 +CPU= 39.914
07:45:27 STEP 345 H= 345:00 +CPU= 58.980
07:46:08 STEP 346 H= 346:00 +CPU= 39.778
07:46:47 STEP 347 H= 347:00 +CPU= 38.634
07:47:53 STEP 348 H= 348:00 +CPU= 64.669
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000348
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000348
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000348
07:48:33 STEP 349 H= 349:00 +CPU= 39.137
07:49:13 STEP 350 H= 350:00 +CPU= 39.734
07:50:13 STEP 351 H= 351:00 +CPU= 59.038
07:50:54 STEP 352 H= 352:00 +CPU= 40.312
07:51:33 STEP 353 H= 353:00 +CPU= 38.693
07:52:33 STEP 354 H= 354:00 +CPU= 59.791
07:53:13 STEP 355 H= 355:00 +CPU= 39.313
07:53:54 STEP 356 H= 356:00 +CPU= 40.268
07:54:52 STEP 357 H= 357:00 +CPU= 58.090
07:55:33 STEP 358 H= 358:00 +CPU= 40.092
07:56:13 STEP 359 H= 359:00 +CPU= 40.118
07:57:25 STEP 360 H= 360:00 +CPU= 70.849
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000360
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000360
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000360
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000336
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000336
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000336
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000348
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000348
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000348
Zipping up the intermediate file: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_0343_2000050100_123_969_12185987_1_r1360474992_14.zip
Uploading the intermediate file: upload_file_14.zip
07:58:05 STEP 361 H= 361:00 +CPU= 38.823
07:58:45 STEP 362 H= 362:00 +CPU= 39.430
07:59:44 STEP 363 H= 363:00 +CPU= 59.171
08:00:24 STEP 364 H= 364:00 +CPU= 39.513
08:01:03 STEP 365 H= 365:00 +CPU= 39.025
08:02:05 STEP 366 H= 366:00 +CPU= 61.243
08:02:44 STEP 367 H= 367:00 +CPU= 38.896
08:03:24 STEP 368 H= 368:00 +CPU= 40.015
08:04:23 STEP 369 H= 369:00 +CPU= 58.667
08:05:02 STEP 370 H= 370:00 +CPU= 38.963
08:05:41 STEP 371 H= 371:00 +CPU= 38.929
08:06:47 STEP 372 H= 372:00 +CPU= 65.097
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000372
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000372
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000372
08:07:26 STEP 373 H= 373:00 +CPU= 37.951
08:08:06 STEP 374 H= 374:00 +CPU= 39.225
08:09:05 STEP 375 H= 375:00 +CPU= 58.211
08:09:45 STEP 376 H= 376:00 +CPU= 39.785
08:10:24 STEP 377 H= 377:00 +CPU= 39.432
08:11:24 STEP 378 H= 378:00 +CPU= 59.516
08:12:03 STEP 379 H= 379:00 +CPU= 38.291
08:12:43 STEP 380 H= 380:00 +CPU= 40.034
08:13:41 STEP 381 H= 381:00 +CPU= 58.011
08:14:21 STEP 382 H= 382:00 +CPU= 39.843
08:15:01 STEP 383 H= 383:00 +CPU= 39.776
08:16:11 STEP 384 H= 384:00 +CPU= 68.998
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000384
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000384
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000384
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000360
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000360
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000360
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000372
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000372
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000372
Zipping up the intermediate file: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_0343_2000050100_123_969_12185987_1_r1360474992_15.zip
Uploading the intermediate file: upload_file_15.zip
08:16:50 STEP 385 H= 385:00 +CPU= 37.946
08:17:31 STEP 386 H= 386:00 +CPU= 40.034
08:18:29 STEP 387 H= 387:00 +CPU= 58.186
08:19:08 STEP 388 H= 388:00 +CPU= 38.742
08:19:47 STEP 389 H= 389:00 +CPU= 38.842
08:20:48 STEP 390 H= 390:00 +CPU= 59.768
08:21:26 STEP 391 H= 391:00 +CPU= 38.014
08:22:06 STEP 392 H= 392:00 +CPU= 39.747
08:23:06 STEP 393 H= 393:00 +CPU= 58.791
08:23:45 STEP 394 H= 394:00 +CPU= 39.418
08:24:25 STEP 395 H= 395:00 +CPU= 39.208
08:25:31 STEP 396 H= 396:00 +CPU= 65.010
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000396
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000396
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000396
08:26:10 STEP 397 H= 397:00 +CPU= 38.714
08:26:50 STEP 398 H= 398:00 +CPU= 39.569
08:27:48 STEP 399 H= 399:00 +CPU= 57.765
08:28:28 STEP 400 H= 400:00 +CPU= 39.617
08:29:08 STEP 401 H= 401:00 +CPU= 38.876
08:30:08 STEP 402 H= 402:00 +CPU= 59.766
08:30:48 STEP 403 H= 403:00 +CPU= 39.176
08:31:28 STEP 404 H= 404:00 +CPU= 39.663
08:32:28 STEP 405 H= 405:00 +CPU= 59.081
08:33:09 STEP 406 H= 406:00 +CPU= 40.623
08:33:49 STEP 407 H= 407:00 +CPU= 40.043
08:35:01 STEP 408 H= 408:00 +CPU= 68.512
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000408
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000408
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000408
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000384
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000384
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000384
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000396
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000396
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000396
Zipping up the intermediate file: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_0343_2000050100_123_969_12185987_1_r1360474992_16.zip
Uploading the intermediate file: upload_file_16.zip
08:35:40 STEP 409 H= 409:00 +CPU= 38.021
08:36:20 STEP 410 H= 410:00 +CPU= 40.148
08:37:19 STEP 411 H= 411:00 +CPU= 58.519
08:37:58 STEP 412 H= 412:00 +CPU= 38.627
08:38:37 STEP 413 H= 413:00 +CPU= 39.435
08:39:37 STEP 414 H= 414:00 +CPU= 59.740
08:40:16 STEP 415 H= 415:00 +CPU= 38.602
08:40:57 STEP 416 H= 416:00 +CPU= 40.574
08:41:57 STEP 417 H= 417:00 +CPU= 59.105
08:42:37 STEP 418 H= 418:00 +CPU= 39.969
08:43:16 STEP 419 H= 419:00 +CPU= 38.368
08:44:23 STEP 420 H= 420:00 +CPU= 66.248
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000420
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000420
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000420
08:45:02 STEP 421 H= 421:00 +CPU= 38.595
08:45:43 STEP 422 H= 422:00 +CPU= 40.210
08:46:42 STEP 423 H= 423:00 +CPU= 58.416
08:47:22 STEP 424 H= 424:00 +CPU= 39.192
08:48:02 STEP 425 H= 425:00 +CPU= 39.567
08:49:02 STEP 426 H= 426:00 +CPU= 59.907
08:49:41 STEP 427 H= 427:00 +CPU= 38.449
08:50:22 STEP 428 H= 428:00 +CPU= 40.206
08:51:21 STEP 429 H= 429:00 +CPU= 58.986
08:52:01 STEP 430 H= 430:00 +CPU= 40.015
08:52:43 STEP 431 H= 431:00 +CPU= 41.140
08:53:54 STEP 432 H= 432:00 +CPU= 69.616
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000432
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000432
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000432
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000408
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000408
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000408
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000420
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000420
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000420
Zipping up the intermediate file: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_0343_2000050100_123_969_12185987_1_r1360474992_17.zip
Uploading the intermediate file: upload_file_17.zip
08:54:32 STEP 433 H= 433:00 +CPU= 37.831
08:55:13 STEP 434 H= 434:00 +CPU= 40.503
08:56:11 STEP 435 H= 435:00 +CPU= 57.747
08:56:50 STEP 436 H= 436:00 +CPU= 39.277
08:57:29 STEP 437 H= 437:00 +CPU= 38.760
08:58:29 STEP 438 H= 438:00 +CPU= 59.331
08:59:06 STEP 439 H= 439:00 +CPU= 37.580
08:59:47 STEP 440 H= 440:00 +CPU= 40.044
09:00:45 STEP 441 H= 441:00 +CPU= 58.119
09:01:25 STEP 442 H= 442:00 +CPU= 39.593
09:02:03 STEP 443 H= 443:00 +CPU= 38.533
09:03:10 STEP 444 H= 444:00 +CPU= 65.349
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000444
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000444
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000444
09:03:49 STEP 445 H= 445:00 +CPU= 38.718
09:04:30 STEP 446 H= 446:00 +CPU= 40.602
09:05:31 STEP 447 H= 447:00 +CPU= 59.363
09:06:10 STEP 448 H= 448:00 +CPU= 38.860
09:06:50 STEP 449 H= 449:00 +CPU= 39.630
09:07:50 STEP 450 H= 450:00 +CPU= 60.030
09:08:28 STEP 451 H= 451:00 +CPU= 38.164
09:09:09 STEP 452 H= 452:00 +CPU= 40.585
09:10:09 STEP 453 H= 453:00 +CPU= 59.421
09:10:49 STEP 454 H= 454:00 +CPU= 39.333
09:11:30 STEP 455 H= 455:00 +CPU= 40.635
09:12:41 STEP 456 H= 456:00 +CPU= 70.075
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000456
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000456
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000456
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000432
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000432
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000432
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000444
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000444
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000444
Zipping up the intermediate file: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_0343_2000050100_123_969_12185987_1_r1360474992_18.zip
Uploading the intermediate file: upload_file_18.zip
09:13:21 STEP 457 H= 457:00 +CPU= 38.652
09:14:01 STEP 458 H= 458:00 +CPU= 40.186
09:14:59 STEP 459 H= 459:00 +CPU= 58.296
09:15:39 STEP 460 H= 460:00 +CPU= 39.538
09:16:18 STEP 461 H= 461:00 +CPU= 38.822
09:17:18 STEP 462 H= 462:00 +CPU= 60.019
09:17:57 STEP 463 H= 463:00 +CPU= 37.977
09:18:37 STEP 464 H= 464:00 +CPU= 40.377
09:19:36 STEP 465 H= 465:00 +CPU= 58.527
09:20:14 STEP 466 H= 466:00 +CPU= 38.461
09:20:54 STEP 467 H= 467:00 +CPU= 38.751
09:22:00 STEP 468 H= 468:00 +CPU= 65.411
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000468
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000468
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000468
09:22:39 STEP 469 H= 469:00 +CPU= 38.286
09:23:19 STEP 470 H= 470:00 +CPU= 39.944
09:24:17 STEP 471 H= 471:00 +CPU= 57.525
09:24:56 STEP 472 H= 472:00 +CPU= 38.442
09:25:35 STEP 473 H= 473:00 +CPU= 38.683
09:26:35 STEP 474 H= 474:00 +CPU= 60.235
09:27:13 STEP 475 H= 475:00 +CPU= 38.035
09:27:54 STEP 476 H= 476:00 +CPU= 40.115
09:28:53 STEP 477 H= 477:00 +CPU= 58.774
09:29:32 STEP 478 H= 478:00 +CPU= 38.770
09:30:12 STEP 479 H= 479:00 +CPU= 40.574
09:31:23 STEP 480 H= 480:00 +CPU= 69.763
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000480
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000480
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000480
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000456
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000456
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000456
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000468
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000468
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000468
Zipping up the intermediate file: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_0343_2000050100_123_969_12185987_1_r1360474992_19.zip
Uploading the intermediate file: upload_file_19.zip
Uploading trickle at timestep: 1728000
09:32:02 STEP 481 H= 481:00 +CPU= 38.157
09:32:43 STEP 482 H= 482:00 +CPU= 40.809
09:33:41 STEP 483 H= 483:00 +CPU= 57.295
09:34:20 STEP 484 H= 484:00 +CPU= 39.130
09:34:59 STEP 485 H= 485:00 +CPU= 38.940
09:36:00 STEP 486 H= 486:00 +CPU= 60.653
09:36:39 STEP 487 H= 487:00 +CPU= 38.783
09:37:20 STEP 488 H= 488:00 +CPU= 40.391
09:38:19 STEP 489 H= 489:00 +CPU= 58.863
09:38:58 STEP 490 H= 490:00 +CPU= 39.326
09:39:38 STEP 491 H= 491:00 +CPU= 39.381
09:40:45 STEP 492 H= 492:00 +CPU= 65.845
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000492
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000492
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000492
09:41:24 STEP 493 H= 493:00 +CPU= 38.688
09:42:05 STEP 494 H= 494:00 +CPU= 40.160
09:43:05 STEP 495 H= 495:00 +CPU= 58.899
09:43:45 STEP 496 H= 496:00 +CPU= 38.859
09:44:24 STEP 497 H= 497:00 +CPU= 38.688
09:45:24 STEP 498 H= 498:00 +CPU= 59.777
09:46:03 STEP 499 H= 499:00 +CPU= 38.062
09:46:44 STEP 500 H= 500:00 +CPU= 40.110
09:47:44 STEP 501 H= 501:00 +CPU= 59.725
09:48:23 STEP 502 H= 502:00 +CPU= 38.704
09:49:03 STEP 503 H= 503:00 +CPU= 40.101
09:50:13 STEP 504 H= 504:00 +CPU= 69.071
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000504
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000504
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000504
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000480
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000480
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000480
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000492
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000492
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000492
Zipping up the intermediate file: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_0343_2000050100_123_969_12185987_1_r1360474992_20.zip
Uploading the intermediate file: upload_file_20.zip
09:50:51 STEP 505 H= 505:00 +CPU= 37.127
09:51:31 STEP 506 H= 506:00 +CPU= 39.989
09:52:31 STEP 507 H= 507:00 +CPU= 59.303
09:53:09 STEP 508 H= 508:00 +CPU= 37.970
09:53:49 STEP 509 H= 509:00 +CPU= 39.328
09:54:51 STEP 510 H= 510:00 +CPU= 61.226
09:55:29 STEP 511 H= 511:00 +CPU= 37.601
09:56:10 STEP 512 H= 512:00 +CPU= 40.909
09:57:10 STEP 513 H= 513:00 +CPU= 59.836
09:57:49 STEP 514 H= 514:00 +CPU= 38.779
09:58:29 STEP 515 H= 515:00 +CPU= 39.307
09:59:36 STEP 516 H= 516:00 +CPU= 66.303
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000516
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000516
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000516
10:00:15 STEP 517 H= 517:00 +CPU= 37.840
10:00:56 STEP 518 H= 518:00 +CPU= 40.628
10:01:56 STEP 519 H= 519:00 +CPU= 59.443
10:02:36 STEP 520 H= 520:00 +CPU= 38.935
10:03:16 STEP 521 H= 521:00 +CPU= 39.599
10:04:17 STEP 522 H= 522:00 +CPU= 60.790
10:04:56 STEP 523 H= 523:00 +CPU= 38.010
10:05:37 STEP 524 H= 524:00 +CPU= 41.067
10:06:37 STEP 525 H= 525:00 +CPU= 58.847
10:07:15 STEP 526 H= 526:00 +CPU= 37.956
10:07:56 STEP 527 H= 527:00 +CPU= 40.315
10:09:05 STEP 528 H= 528:00 +CPU= 68.099
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000528
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000528
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000528
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000504
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000504
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000504
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMGGhq0f+000516
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMSHhq0f+000516
Adding to the zip: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_12185987/ICMUAhq0f+000516
Zipping up the intermediate file: /home/tommcfarland/projects/climateprediction.net/oifs_43r3_ps_0343_2000050100_123_969_12185987_1_r1360474992_21.zip
Uploading the intermediate file: upload_file_21.zip
10:09:43 STEP 529 H= 529:00 +CPU= 37.359
10:10:24 STEP 530 H= 530:00 +CPU= 40.105
10:11:23 STEP 531 H= 531:00 +CPU= 58.717
10:12:02 STEP 532 H= 532:00 +CPU= 38.530
10:12:42 STEP 533 H= 533:00 +CPU= 39.476
10:13:43 STEP 534 H= 534:00 +CPU= 60.873
10:14:21 STEP 535 H= 535:00 +CPU= 37.427
10:15:02 STEP 536 H= 536:00 +CPU= 41.103
10:16:03 STEP 537 H= 537:00 +CPU= 60.756
10:16:42 STEP 538 H= 538:00 +CPU= 38.191
10:17:22 STEP 539 H= 539:00 +CPU= 39.556
10:18:29 STEP 540 H= 540:00 +CPU= 67.117
Moving to projects directory: /home/tommcfarland/slots/0/ICMGGhq0f+000540
Moving to projects directory: /home/tommcfarland/slots/0/ICMSHhq0f+000540
Moving to projects directory: /home/tommcfarland/slots/0/ICMUAhq0f+000540
The child process terminated with status: 0
..Failed, model did not complete successfully
</stderr_txt>
]]>
|
