climateprediction.net home page
Task 22440766

Task 22440766

Name oifs_43r3_bl_a2ru_2016092300_20_1017_12285421_1
Workunit 12285421
Created 8 Jun 2024, 9:29:56 UTC
Sent 8 Jun 2024, 9:36:34 UTC
Report deadline 7 Aug 2024, 9:36:34 UTC
Received 9 Jun 2024, 0:51:06 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 0 (0x00000000)
Computer ID 1517479
Run time 12 hours 28 min 19 sec
CPU time 12 hours 21 min 15 sec
Validate state Invalid
Credit 1,976.32
Device peak FLOPS 5.07 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,024.30 MB
Peak swap size 5,980.80 MB
Peak disk usage 1,353.87 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<stderr_txt>
HOOK_TRAPFPE=1
[EC_DRHOOK:amigos:1:1:3395099:3395099] [20240608:125426:1717876466:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:amigos:1:1:3395099:3395099] [20240608:125426:1717876466:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:amigos:1:1:3395099:3395099] [20240608:125426:1717876466:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:amigos:1:1:3395099:3395099] [20240608:125426:1717876466:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:amigos:1:1:3395099:3395099] [20240608:125426:1717876466:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:amigos:1:1:3395099:3395099] [20240608:125426:1717876466:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:amigos:1:1:3395099:3395099] [20240608:125426:1717876466:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:amigos:1:1:3395099:3395099] [20240608:125426:1717876466:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:amigos:1:1:3395099:3395099] [20240608:125426:1717876466:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:amigos:1:1:3395099:3395099] [20240608:125426:1717876466:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:amigos:1:1:3395099:3395099] [20240608:125426:1717876466:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:amigos:1:1:3395099:3395099] [20240608:125426:1717876466:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:amigos:1:1:3395099:3395099] [20240608:125426:1717876466:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:amigos:1:1:3395099:3395099] [20240608:125426:1717876466:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:amigos:1:1:3395099:3395099] [20240608:125426:1717876466:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:amigos:1:1:3395099:3395099] [20240608:125426:1717876466:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:amigos:1:1:3395099:3395099] [20240608:125426:1717876466:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:amigos:1:1:3395099:3395099] [20240608:125426:1717876466:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:amigos:1:1:3395099:3395099] [20240608:125426:1717876466:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:amigos:1:1:3395099:3395099] [20240608:125426:1717876466:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  12:54:54 STEP  864 H= 216:00 +CPU= 25.453
  12:55:20 STEP  865 H= 216:15 +CPU= 25.807
  12:55:46 STEP  866 H= 216:30 +CPU= 25.802
  12:56:12 STEP  867 H= 216:45 +CPU= 25.732
  12:56:37 STEP  868 H= 217:00 +CPU= 25.340
  12:57:03 STEP  869 H= 217:15 +CPU= 25.064
  12:57:28 STEP  870 H= 217:30 +CPU= 24.996
  12:57:53 STEP  871 H= 217:45 +CPU= 24.882
  12:58:18 STEP  872 H= 218:00 +CPU= 25.637
  12:58:43 STEP  873 H= 218:15 +CPU= 24.905
  12:59:10 STEP  874 H= 218:30 +CPU= 26.145
  12:59:35 STEP  875 H= 218:45 +CPU= 25.073
  13:00:05 STEP  876 H= 219:00 +CPU= 30.245
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+000876
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+000876
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+000876
  13:00:33 STEP  877 H= 219:15 +CPU= 27.889
  13:01:02 STEP  878 H= 219:30 +CPU= 28.871
  13:01:31 STEP  879 H= 219:45 +CPU= 28.833
  13:02:01 STEP  880 H= 220:00 +CPU= 29.431
  13:02:31 STEP  881 H= 220:15 +CPU= 29.671
  13:02:59 STEP  882 H= 220:30 +CPU= 28.561
  13:03:29 STEP  883 H= 220:45 +CPU= 29.315
  13:03:58 STEP  884 H= 221:00 +CPU= 29.230
  13:04:26 STEP  885 H= 221:15 +CPU= 28.116
  13:04:56 STEP  886 H= 221:30 +CPU= 29.042
  13:05:25 STEP  887 H= 221:45 +CPU= 28.912
  13:06:01 STEP  888 H= 222:00 +CPU= 36.594
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+000888
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+000888
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+000888
  13:06:30 STEP  889 H= 222:15 +CPU= 28.603
  13:06:59 STEP  890 H= 222:30 +CPU= 28.880
  13:07:28 STEP  891 H= 222:45 +CPU= 28.759
  13:07:57 STEP  892 H= 223:00 +CPU= 29.414
  13:08:26 STEP  893 H= 223:15 +CPU= 28.856
  13:08:55 STEP  894 H= 223:30 +CPU= 28.502
  13:09:24 STEP  895 H= 223:45 +CPU= 28.421
  13:09:54 STEP  896 H= 224:00 +CPU= 29.856
  13:10:23 STEP  897 H= 224:15 +CPU= 28.941
  13:10:52 STEP  898 H= 224:30 +CPU= 29.034
  13:11:20 STEP  899 H= 224:45 +CPU= 28.222
  13:11:56 STEP  900 H= 225:00 +CPU= 36.129
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+000900
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+000900
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+000900
  13:12:25 STEP  901 H= 225:15 +CPU= 28.519
  13:12:54 STEP  902 H= 225:30 +CPU= 28.770
  13:13:23 STEP  903 H= 225:45 +CPU= 28.671
  13:13:52 STEP  904 H= 226:00 +CPU= 28.938
  13:14:20 STEP  905 H= 226:15 +CPU= 28.402
  13:14:50 STEP  906 H= 226:30 +CPU= 29.166
  13:15:18 STEP  907 H= 226:45 +CPU= 28.477
  13:15:48 STEP  908 H= 227:00 +CPU= 29.425
  13:16:16 STEP  909 H= 227:15 +CPU= 28.329
  13:16:46 STEP  910 H= 227:30 +CPU= 29.566
  13:17:15 STEP  911 H= 227:45 +CPU= 28.713
  13:17:54 STEP  912 H= 228:00 +CPU= 38.904
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+000912
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+000912
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+000912
  13:18:23 STEP  913 H= 228:15 +CPU= 28.671
  13:18:52 STEP  914 H= 228:30 +CPU= 28.452
  13:19:21 STEP  915 H= 228:45 +CPU= 29.158
  13:19:50 STEP  916 H= 229:00 +CPU= 28.968
  13:20:19 STEP  917 H= 229:15 +CPU= 28.886
  13:20:48 STEP  918 H= 229:30 +CPU= 28.476
  13:21:18 STEP  919 H= 229:45 +CPU= 29.413
  13:21:47 STEP  920 H= 230:00 +CPU= 29.295
  13:22:16 STEP  921 H= 230:15 +CPU= 29.151
  13:22:45 STEP  922 H= 230:30 +CPU= 29.032
  13:23:15 STEP  923 H= 230:45 +CPU= 29.275
  13:23:51 STEP  924 H= 231:00 +CPU= 35.714
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+000924
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+000924
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+000924
  13:24:20 STEP  925 H= 231:15 +CPU= 28.623
  13:24:48 STEP  926 H= 231:30 +CPU= 28.424
  13:25:17 STEP  927 H= 231:45 +CPU= 28.518
  13:25:46 STEP  928 H= 232:00 +CPU= 29.432
  13:26:15 STEP  929 H= 232:15 +CPU= 28.669
  13:26:44 STEP  930 H= 232:30 +CPU= 28.736
  13:27:14 STEP  931 H= 232:45 +CPU= 29.768
  13:27:44 STEP  932 H= 233:00 +CPU= 29.884
  13:28:13 STEP  933 H= 233:15 +CPU= 28.615
  13:28:42 STEP  934 H= 233:30 +CPU= 28.737
  13:29:11 STEP  935 H= 233:45 +CPU= 29.809
  13:29:48 STEP  936 H= 234:00 +CPU= 36.009
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+000936
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+000936
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+000936
  13:30:17 STEP  937 H= 234:15 +CPU= 28.667
  13:30:45 STEP  938 H= 234:30 +CPU= 28.445
  13:31:14 STEP  939 H= 234:45 +CPU= 28.864
  13:31:44 STEP  940 H= 235:00 +CPU= 29.248
  13:32:12 STEP  941 H= 235:15 +CPU= 28.584
  13:32:41 STEP  942 H= 235:30 +CPU= 28.772
  13:33:10 STEP  943 H= 235:45 +CPU= 28.756
  13:33:40 STEP  944 H= 236:00 +CPU= 29.657
  13:34:09 STEP  945 H= 236:15 +CPU= 28.481
  13:34:38 STEP  946 H= 236:30 +CPU= 28.729
  13:35:07 STEP  947 H= 236:45 +CPU= 29.008
  13:35:43 STEP  948 H= 237:00 +CPU= 35.747
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+000948
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+000948
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+000948
  13:36:12 STEP  949 H= 237:15 +CPU= 28.895
  13:36:39 STEP  950 H= 237:30 +CPU= 27.781
  13:37:08 STEP  951 H= 237:45 +CPU= 28.097
  13:37:38 STEP  952 H= 238:00 +CPU= 29.676
  13:38:06 STEP  953 H= 238:15 +CPU= 28.190
  13:38:35 STEP  954 H= 238:30 +CPU= 28.912
  13:39:03 STEP  955 H= 238:45 +CPU= 27.984
  13:39:33 STEP  956 H= 239:00 +CPU= 29.595
  13:40:01 STEP  957 H= 239:15 +CPU= 27.915
  13:40:30 STEP  958 H= 239:30 +CPU= 28.764
  13:40:59 STEP  959 H= 239:45 +CPU= 28.808
  13:41:39 STEP  960 H= 240:00 +CPU= 39.520
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+000960
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+000960
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+000960
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+000864
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+000864
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+000864
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+000876
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+000876
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+000876
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+000888
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+000888
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+000888
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+000900
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+000900
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+000900
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+000912
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+000912
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+000912
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+000924
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+000924
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+000924
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+000936
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+000936
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+000936
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+000948
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+000948
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+000948
Zipping up the intermediate file: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_a2ru_2016092300_20_1017_12285421_1_r1015525513_9.zip
Uploading the intermediate file: upload_file_9.zip
  13:42:08 STEP  961 H= 240:15 +CPU= 28.692
  13:42:38 STEP  962 H= 240:30 +CPU= 29.638
  13:43:07 STEP  963 H= 240:45 +CPU= 29.138
  13:43:37 STEP  964 H= 241:00 +CPU= 29.427
  13:44:07 STEP  965 H= 241:15 +CPU= 28.956
  13:44:35 STEP  966 H= 241:30 +CPU= 28.531
  13:45:04 STEP  967 H= 241:45 +CPU= 28.583
  13:45:33 STEP  968 H= 242:00 +CPU= 28.695
  13:46:02 STEP  969 H= 242:15 +CPU= 28.808
  13:46:31 STEP  970 H= 242:30 +CPU= 29.516
  13:47:01 STEP  971 H= 242:45 +CPU= 29.113
  13:47:36 STEP  972 H= 243:00 +CPU= 35.652
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+000972
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+000972
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+000972
  13:48:05 STEP  973 H= 243:15 +CPU= 28.441
  13:48:33 STEP  974 H= 243:30 +CPU= 28.056
  13:49:02 STEP  975 H= 243:45 +CPU= 28.715
  13:49:32 STEP  976 H= 244:00 +CPU= 29.793
  13:50:01 STEP  977 H= 244:15 +CPU= 29.034
  13:50:29 STEP  978 H= 244:30 +CPU= 28.018
  13:50:58 STEP  979 H= 244:45 +CPU= 28.738
  13:51:28 STEP  980 H= 245:00 +CPU= 29.899
  13:51:56 STEP  981 H= 245:15 +CPU= 28.096
  13:52:24 STEP  982 H= 245:30 +CPU= 28.155
  13:52:53 STEP  983 H= 245:45 +CPU= 28.591
  13:53:29 STEP  984 H= 246:00 +CPU= 35.805
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+000984
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+000984
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+000984
  13:53:58 STEP  985 H= 246:15 +CPU= 29.118
  13:54:27 STEP  986 H= 246:30 +CPU= 28.379
  13:54:56 STEP  987 H= 246:45 +CPU= 28.705
  13:55:25 STEP  988 H= 247:00 +CPU= 29.428
  13:55:55 STEP  989 H= 247:15 +CPU= 29.455
  13:56:24 STEP  990 H= 247:30 +CPU= 28.957
  13:56:53 STEP  991 H= 247:45 +CPU= 28.470
  13:57:22 STEP  992 H= 248:00 +CPU= 29.295
  13:57:50 STEP  993 H= 248:15 +CPU= 28.261
  13:58:18 STEP  994 H= 248:30 +CPU= 27.787
  13:58:47 STEP  995 H= 248:45 +CPU= 28.125
  13:59:23 STEP  996 H= 249:00 +CPU= 35.823
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+000996
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+000996
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+000996
  13:59:51 STEP  997 H= 249:15 +CPU= 28.419
  14:00:21 STEP  998 H= 249:30 +CPU= 29.572
  14:00:49 STEP  999 H= 249:45 +CPU= 27.839
  14:01:19 STEP 1000 H= 250:00 +CPU= 30.265
  14:01:48 STEP 1001 H= 250:15 +CPU= 28.804
  14:02:17 STEP 1002 H= 250:30 +CPU= 28.671
  14:02:45 STEP 1003 H= 250:45 +CPU= 28.177
  14:03:16 STEP 1004 H= 251:00 +CPU= 30.431
  14:03:45 STEP 1005 H= 251:15 +CPU= 28.940
  14:04:13 STEP 1006 H= 251:30 +CPU= 28.541
  14:04:43 STEP 1007 H= 251:45 +CPU= 29.138
  14:05:22 STEP 1008 H= 252:00 +CPU= 39.200
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001008
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001008
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001008
  14:05:51 STEP 1009 H= 252:15 +CPU= 28.878
  14:06:20 STEP 1010 H= 252:30 +CPU= 28.731
  14:06:50 STEP 1011 H= 252:45 +CPU= 29.165
  14:07:19 STEP 1012 H= 253:00 +CPU= 29.218
  14:07:47 STEP 1013 H= 253:15 +CPU= 28.405
  14:08:17 STEP 1014 H= 253:30 +CPU= 29.293
  14:08:46 STEP 1015 H= 253:45 +CPU= 28.676
  14:09:15 STEP 1016 H= 254:00 +CPU= 28.458
  14:09:43 STEP 1017 H= 254:15 +CPU= 28.388
  14:10:13 STEP 1018 H= 254:30 +CPU= 29.427
  14:10:42 STEP 1019 H= 254:45 +CPU= 28.838
  14:11:18 STEP 1020 H= 255:00 +CPU= 36.254
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001020
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001020
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001020
  14:11:48 STEP 1021 H= 255:15 +CPU= 29.438
  14:12:17 STEP 1022 H= 255:30 +CPU= 29.091
  14:12:46 STEP 1023 H= 255:45 +CPU= 28.836
  14:13:16 STEP 1024 H= 256:00 +CPU= 30.189
  14:13:45 STEP 1025 H= 256:15 +CPU= 28.744
  14:14:13 STEP 1026 H= 256:30 +CPU= 28.128
  14:14:43 STEP 1027 H= 256:45 +CPU= 29.376
  14:15:13 STEP 1028 H= 257:00 +CPU= 29.719
  14:15:42 STEP 1029 H= 257:15 +CPU= 28.976
  14:16:10 STEP 1030 H= 257:30 +CPU= 28.440
  14:16:40 STEP 1031 H= 257:45 +CPU= 29.130
  14:17:17 STEP 1032 H= 258:00 +CPU= 36.712
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001032
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001032
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001032
  14:17:46 STEP 1033 H= 258:15 +CPU= 29.120
  14:18:14 STEP 1034 H= 258:30 +CPU= 28.564
  14:18:44 STEP 1035 H= 258:45 +CPU= 29.929
  14:19:13 STEP 1036 H= 259:00 +CPU= 28.620
  14:19:42 STEP 1037 H= 259:15 +CPU= 29.063
  14:20:11 STEP 1038 H= 259:30 +CPU= 28.391
  14:20:40 STEP 1039 H= 259:45 +CPU= 28.824
  14:21:09 STEP 1040 H= 260:00 +CPU= 29.427
  14:21:38 STEP 1041 H= 260:15 +CPU= 28.621
  14:22:07 STEP 1042 H= 260:30 +CPU= 28.906
  14:22:36 STEP 1043 H= 260:45 +CPU= 28.227
  14:23:11 STEP 1044 H= 261:00 +CPU= 35.538
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001044
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001044
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001044
  14:23:40 STEP 1045 H= 261:15 +CPU= 28.700
  14:24:09 STEP 1046 H= 261:30 +CPU= 28.991
  14:24:38 STEP 1047 H= 261:45 +CPU= 28.437
  14:25:07 STEP 1048 H= 262:00 +CPU= 29.594
  14:25:36 STEP 1049 H= 262:15 +CPU= 28.545
  14:26:05 STEP 1050 H= 262:30 +CPU= 28.682
  14:26:34 STEP 1051 H= 262:45 +CPU= 28.627
  14:27:03 STEP 1052 H= 263:00 +CPU= 29.538
  14:27:33 STEP 1053 H= 263:15 +CPU= 29.211
  14:28:02 STEP 1054 H= 263:30 +CPU= 29.001
  14:28:30 STEP 1055 H= 263:45 +CPU= 28.285
  14:29:11 STEP 1056 H= 264:00 +CPU= 39.557
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001056
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001056
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001056
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+000960
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+000960
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+000960
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+000972
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+000972
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+000972
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+000984
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+000984
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+000984
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+000996
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+000996
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+000996
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001008
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001008
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001008
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001020
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001020
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001020
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001032
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001032
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001032
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001044
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001044
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001044
Zipping up the intermediate file: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_a2ru_2016092300_20_1017_12285421_1_r1015525513_10.zip
Uploading the intermediate file: upload_file_10.zip
  14:29:40 STEP 1057 H= 264:15 +CPU= 28.662
  14:30:09 STEP 1058 H= 264:30 +CPU= 28.741
  14:30:38 STEP 1059 H= 264:45 +CPU= 29.015
  14:31:09 STEP 1060 H= 265:00 +CPU= 29.916
  14:31:36 STEP 1061 H= 265:15 +CPU= 27.781
  14:32:06 STEP 1062 H= 265:30 +CPU= 29.238
  14:32:34 STEP 1063 H= 265:45 +CPU= 28.592
  14:33:05 STEP 1064 H= 266:00 +CPU= 29.940
  14:33:34 STEP 1065 H= 266:15 +CPU= 29.103
  14:34:04 STEP 1066 H= 266:30 +CPU= 29.719
  14:34:32 STEP 1067 H= 266:45 +CPU= 27.942
  14:35:08 STEP 1068 H= 267:00 +CPU= 36.434
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001068
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001068
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001068
  14:35:37 STEP 1069 H= 267:15 +CPU= 28.703
  14:36:06 STEP 1070 H= 267:30 +CPU= 29.293
  14:36:35 STEP 1071 H= 267:45 +CPU= 28.369
  14:37:04 STEP 1072 H= 268:00 +CPU= 28.699
  14:37:32 STEP 1073 H= 268:15 +CPU= 28.143
  14:38:01 STEP 1074 H= 268:30 +CPU= 28.598
  14:38:30 STEP 1075 H= 268:45 +CPU= 29.226
  14:39:00 STEP 1076 H= 269:00 +CPU= 29.810
  14:39:29 STEP 1077 H= 269:15 +CPU= 28.853
  14:39:57 STEP 1078 H= 269:30 +CPU= 28.311
  14:40:27 STEP 1079 H= 269:45 +CPU= 28.971
  14:41:03 STEP 1080 H= 270:00 +CPU= 36.019
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001080
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001080
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001080
  14:41:32 STEP 1081 H= 270:15 +CPU= 28.648
  14:42:00 STEP 1082 H= 270:30 +CPU= 28.111
  14:42:29 STEP 1083 H= 270:45 +CPU= 29.310
  14:42:58 STEP 1084 H= 271:00 +CPU= 28.532
  14:43:26 STEP 1085 H= 271:15 +CPU= 28.302
  14:43:55 STEP 1086 H= 271:30 +CPU= 28.303
  14:44:24 STEP 1087 H= 271:45 +CPU= 29.406
  14:44:55 STEP 1088 H= 272:00 +CPU= 30.380
  14:45:24 STEP 1089 H= 272:15 +CPU= 28.862
  14:45:53 STEP 1090 H= 272:30 +CPU= 28.743
  14:46:22 STEP 1091 H= 272:45 +CPU= 29.159
  14:46:58 STEP 1092 H= 273:00 +CPU= 35.834
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001092
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001092
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001092
  14:47:27 STEP 1093 H= 273:15 +CPU= 28.922
  14:47:56 STEP 1094 H= 273:30 +CPU= 28.930
  14:48:25 STEP 1095 H= 273:45 +CPU= 28.759
  14:48:54 STEP 1096 H= 274:00 +CPU= 29.303
  14:49:22 STEP 1097 H= 274:15 +CPU= 27.948
  14:49:51 STEP 1098 H= 274:30 +CPU= 28.394
  14:50:20 STEP 1099 H= 274:45 +CPU= 28.385
  14:50:48 STEP 1100 H= 275:00 +CPU= 28.580
  14:51:17 STEP 1101 H= 275:15 +CPU= 28.965
  14:51:46 STEP 1102 H= 275:30 +CPU= 28.203
  14:52:14 STEP 1103 H= 275:45 +CPU= 28.375
  14:52:54 STEP 1104 H= 276:00 +CPU= 39.335
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001104
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001104
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001104
  14:53:22 STEP 1105 H= 276:15 +CPU= 28.490
  14:53:52 STEP 1106 H= 276:30 +CPU= 29.244
  14:54:21 STEP 1107 H= 276:45 +CPU= 28.972
  14:54:51 STEP 1108 H= 277:00 +CPU= 30.082
  14:55:20 STEP 1109 H= 277:15 +CPU= 27.869
  14:55:49 STEP 1110 H= 277:30 +CPU= 28.988
  14:56:18 STEP 1111 H= 277:45 +CPU= 29.211
  14:56:47 STEP 1112 H= 278:00 +CPU= 29.173
  14:57:16 STEP 1113 H= 278:15 +CPU= 28.045
  14:57:45 STEP 1114 H= 278:30 +CPU= 29.020
  14:58:14 STEP 1115 H= 278:45 +CPU= 28.585
  14:58:50 STEP 1116 H= 279:00 +CPU= 35.357
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001116
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001116
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001116
  14:59:18 STEP 1117 H= 279:15 +CPU= 28.184
  14:59:47 STEP 1118 H= 279:30 +CPU= 28.980
  15:00:16 STEP 1119 H= 279:45 +CPU= 29.084
  15:00:46 STEP 1120 H= 280:00 +CPU= 29.751
  15:01:14 STEP 1121 H= 280:15 +CPU= 28.415
  15:01:44 STEP 1122 H= 280:30 +CPU= 28.993
  15:02:13 STEP 1123 H= 280:45 +CPU= 28.972
  15:02:43 STEP 1124 H= 281:00 +CPU= 29.894
  15:03:12 STEP 1125 H= 281:15 +CPU= 29.215
  15:03:41 STEP 1126 H= 281:30 +CPU= 28.512
  15:04:10 STEP 1127 H= 281:45 +CPU= 29.608
  15:04:46 STEP 1128 H= 282:00 +CPU= 35.200
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001128
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001128
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001128
  15:05:14 STEP 1129 H= 282:15 +CPU= 28.476
  15:05:43 STEP 1130 H= 282:30 +CPU= 28.101
  15:06:11 STEP 1131 H= 282:45 +CPU= 28.634
  15:06:40 STEP 1132 H= 283:00 +CPU= 28.894
  15:07:09 STEP 1133 H= 283:15 +CPU= 28.861
  15:07:39 STEP 1134 H= 283:30 +CPU= 29.153
  15:08:08 STEP 1135 H= 283:45 +CPU= 28.926
  15:08:38 STEP 1136 H= 284:00 +CPU= 29.831
  15:09:07 STEP 1137 H= 284:15 +CPU= 29.136
  15:09:36 STEP 1138 H= 284:30 +CPU= 28.428
  15:10:04 STEP 1139 H= 284:45 +CPU= 28.556
  15:10:40 STEP 1140 H= 285:00 +CPU= 36.062
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001140
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001140
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001140
  15:11:10 STEP 1141 H= 285:15 +CPU= 29.067
  15:11:38 STEP 1142 H= 285:30 +CPU= 28.549
  15:12:07 STEP 1143 H= 285:45 +CPU= 28.820
  15:12:37 STEP 1144 H= 286:00 +CPU= 29.543
  15:13:05 STEP 1145 H= 286:15 +CPU= 28.393
  15:13:34 STEP 1146 H= 286:30 +CPU= 28.746
  15:14:04 STEP 1147 H= 286:45 +CPU= 29.527
  15:14:35 STEP 1148 H= 287:00 +CPU= 30.639
  15:15:04 STEP 1149 H= 287:15 +CPU= 29.425
  15:15:33 STEP 1150 H= 287:30 +CPU= 28.981
  15:16:03 STEP 1151 H= 287:45 +CPU= 29.294
  15:16:43 STEP 1152 H= 288:00 +CPU= 39.591
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001152
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001152
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001152
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001056
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001056
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001056
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001068
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001068
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001068
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001080
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001080
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001080
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001092
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001092
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001092
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001104
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001104
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001104
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001116
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001116
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001116
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001128
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001128
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001128
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001140
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001140
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001140
Zipping up the intermediate file: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_a2ru_2016092300_20_1017_12285421_1_r1015525513_11.zip
Uploading the intermediate file: upload_file_11.zip
  15:17:12 STEP 1153 H= 288:15 +CPU= 28.793
  15:17:42 STEP 1154 H= 288:30 +CPU= 28.896
  15:18:12 STEP 1155 H= 288:45 +CPU= 29.569
  15:18:42 STEP 1156 H= 289:00 +CPU= 29.908
  15:19:11 STEP 1157 H= 289:15 +CPU= 28.779
  15:19:41 STEP 1158 H= 289:30 +CPU= 29.315
  15:20:10 STEP 1159 H= 289:45 +CPU= 28.578
  15:20:40 STEP 1160 H= 290:00 +CPU= 29.864
  15:21:08 STEP 1161 H= 290:15 +CPU= 28.712
  15:21:37 STEP 1162 H= 290:30 +CPU= 28.298
  15:22:05 STEP 1163 H= 290:45 +CPU= 28.241
  15:22:41 STEP 1164 H= 291:00 +CPU= 35.892
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001164
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001164
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001164
  15:23:10 STEP 1165 H= 291:15 +CPU= 28.921
  15:23:40 STEP 1166 H= 291:30 +CPU= 29.251
  15:24:09 STEP 1167 H= 291:45 +CPU= 29.317
  15:24:39 STEP 1168 H= 292:00 +CPU= 29.374
  15:25:08 STEP 1169 H= 292:15 +CPU= 28.891
  15:25:37 STEP 1170 H= 292:30 +CPU= 29.227
  15:26:07 STEP 1171 H= 292:45 +CPU= 29.287
  15:26:36 STEP 1172 H= 293:00 +CPU= 29.874
  15:27:06 STEP 1173 H= 293:15 +CPU= 29.046
  15:27:35 STEP 1174 H= 293:30 +CPU= 28.913
  15:28:03 STEP 1175 H= 293:45 +CPU= 28.197
  15:28:39 STEP 1176 H= 294:00 +CPU= 36.004
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001176
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001176
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001176
  15:29:08 STEP 1177 H= 294:15 +CPU= 28.394
  15:29:38 STEP 1178 H= 294:30 +CPU= 29.731
  15:30:07 STEP 1179 H= 294:45 +CPU= 29.361
  15:30:36 STEP 1180 H= 295:00 +CPU= 29.222
  15:31:06 STEP 1181 H= 295:15 +CPU= 28.955
  15:31:35 STEP 1182 H= 295:30 +CPU= 29.112
  15:32:05 STEP 1183 H= 295:45 +CPU= 29.910
  15:32:35 STEP 1184 H= 296:00 +CPU= 29.861
  15:33:04 STEP 1185 H= 296:15 +CPU= 28.782
  15:33:32 STEP 1186 H= 296:30 +CPU= 28.047
  15:34:01 STEP 1187 H= 296:45 +CPU= 28.667
  15:34:36 STEP 1188 H= 297:00 +CPU= 35.492
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001188
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001188
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001188
  15:35:06 STEP 1189 H= 297:15 +CPU= 29.262
  15:35:35 STEP 1190 H= 297:30 +CPU= 28.783
  15:36:03 STEP 1191 H= 297:45 +CPU= 28.261
  15:36:32 STEP 1192 H= 298:00 +CPU= 29.093
  15:37:02 STEP 1193 H= 298:15 +CPU= 29.750
  15:37:30 STEP 1194 H= 298:30 +CPU= 28.061
  15:37:59 STEP 1195 H= 298:45 +CPU= 29.050
  15:38:29 STEP 1196 H= 299:00 +CPU= 29.341
  15:38:57 STEP 1197 H= 299:15 +CPU= 28.393
  15:39:26 STEP 1198 H= 299:30 +CPU= 28.776
  15:39:55 STEP 1199 H= 299:45 +CPU= 28.850
  15:40:35 STEP 1200 H= 300:00 +CPU= 39.504
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001200
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001200
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001200
  15:41:04 STEP 1201 H= 300:15 +CPU= 28.978
  15:41:33 STEP 1202 H= 300:30 +CPU= 28.855
  15:42:02 STEP 1203 H= 300:45 +CPU= 28.944
  15:42:32 STEP 1204 H= 301:00 +CPU= 30.165
  15:43:01 STEP 1205 H= 301:15 +CPU= 28.256
  15:43:30 STEP 1206 H= 301:30 +CPU= 29.497
  15:43:59 STEP 1207 H= 301:45 +CPU= 28.689
  15:44:30 STEP 1208 H= 302:00 +CPU= 30.759
  15:44:59 STEP 1209 H= 302:15 +CPU= 29.166
  15:45:29 STEP 1210 H= 302:30 +CPU= 29.340
  15:45:58 STEP 1211 H= 302:45 +CPU= 28.851
  15:46:33 STEP 1212 H= 303:00 +CPU= 35.442
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001212
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001212
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001212
  15:47:03 STEP 1213 H= 303:15 +CPU= 29.158
  15:47:32 STEP 1214 H= 303:30 +CPU= 28.656
  15:48:01 STEP 1215 H= 303:45 +CPU= 28.884
  15:48:31 STEP 1216 H= 304:00 +CPU= 30.138
  15:49:01 STEP 1217 H= 304:15 +CPU= 30.031
  15:49:31 STEP 1218 H= 304:30 +CPU= 29.322
  15:50:00 STEP 1219 H= 304:45 +CPU= 28.742
  15:50:29 STEP 1220 H= 305:00 +CPU= 28.991
  15:50:57 STEP 1221 H= 305:15 +CPU= 28.120
  15:51:27 STEP 1222 H= 305:30 +CPU= 29.371
  15:51:56 STEP 1223 H= 305:45 +CPU= 29.135
  15:52:33 STEP 1224 H= 306:00 +CPU= 37.279
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001224
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001224
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001224
  15:53:02 STEP 1225 H= 306:15 +CPU= 28.472
  15:53:32 STEP 1226 H= 306:30 +CPU= 29.657
  15:54:03 STEP 1227 H= 306:45 +CPU= 30.965
  15:54:35 STEP 1228 H= 307:00 +CPU= 32.286
  15:55:06 STEP 1229 H= 307:15 +CPU= 30.380
  15:55:35 STEP 1230 H= 307:30 +CPU= 29.451
  15:56:05 STEP 1231 H= 307:45 +CPU= 29.702
  15:56:36 STEP 1232 H= 308:00 +CPU= 30.469
  15:57:05 STEP 1233 H= 308:15 +CPU= 29.343
  15:57:36 STEP 1234 H= 308:30 +CPU= 30.141
  15:58:04 STEP 1235 H= 308:45 +CPU= 28.640
  15:58:42 STEP 1236 H= 309:00 +CPU= 37.663
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001236
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001236
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001236
  15:59:13 STEP 1237 H= 309:15 +CPU= 30.337
  15:59:43 STEP 1238 H= 309:30 +CPU= 30.156
  16:00:13 STEP 1239 H= 309:45 +CPU= 29.897
  16:00:44 STEP 1240 H= 310:00 +CPU= 30.490
  16:01:14 STEP 1241 H= 310:15 +CPU= 30.483
  16:01:44 STEP 1242 H= 310:30 +CPU= 29.463
  16:02:14 STEP 1243 H= 310:45 +CPU= 30.031
  16:02:45 STEP 1244 H= 311:00 +CPU= 30.712
  16:03:14 STEP 1245 H= 311:15 +CPU= 29.259
  16:03:44 STEP 1246 H= 311:30 +CPU= 29.605
  16:04:14 STEP 1247 H= 311:45 +CPU= 29.618
  16:04:54 STEP 1248 H= 312:00 +CPU= 39.746
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001248
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001248
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001248
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001152
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001152
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001152
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001164
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001164
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001164
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001176
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001176
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001176
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001188
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001188
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001188
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001200
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001200
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001200
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001212
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001212
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001212
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001224
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001224
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001224
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001236
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001236
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001236
Zipping up the intermediate file: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_a2ru_2016092300_20_1017_12285421_1_r1015525513_12.zip
Uploading the intermediate file: upload_file_12.zip
  16:05:24 STEP 1249 H= 312:15 +CPU= 29.314
  16:05:53 STEP 1250 H= 312:30 +CPU= 28.742
  16:06:21 STEP 1251 H= 312:45 +CPU= 28.313
  16:06:51 STEP 1252 H= 313:00 +CPU= 29.291
  16:07:18 STEP 1253 H= 313:15 +CPU= 27.603
  16:07:47 STEP 1254 H= 313:30 +CPU= 28.408
  16:08:15 STEP 1255 H= 313:45 +CPU= 28.189
  16:08:44 STEP 1256 H= 314:00 +CPU= 28.846
  16:09:14 STEP 1257 H= 314:15 +CPU= 29.464
  16:09:42 STEP 1258 H= 314:30 +CPU= 28.610
  16:10:12 STEP 1259 H= 314:45 +CPU= 29.043
  16:10:48 STEP 1260 H= 315:00 +CPU= 35.859
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001260
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001260
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001260
  16:11:17 STEP 1261 H= 315:15 +CPU= 29.189
  16:11:46 STEP 1262 H= 315:30 +CPU= 29.133
  16:12:15 STEP 1263 H= 315:45 +CPU= 28.461
  16:12:46 STEP 1264 H= 316:00 +CPU= 30.629
  16:13:19 STEP 1265 H= 316:15 +CPU= 33.224
  16:13:49 STEP 1266 H= 316:30 +CPU= 29.715
  16:14:18 STEP 1267 H= 316:45 +CPU= 29.381
  16:14:48 STEP 1268 H= 317:00 +CPU= 29.774
  16:15:19 STEP 1269 H= 317:15 +CPU= 30.378
  16:15:49 STEP 1270 H= 317:30 +CPU= 29.753
  16:16:19 STEP 1271 H= 317:45 +CPU= 30.242
  16:16:56 STEP 1272 H= 318:00 +CPU= 37.188
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001272
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001272
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001272
  16:17:26 STEP 1273 H= 318:15 +CPU= 29.222
  16:17:55 STEP 1274 H= 318:30 +CPU= 29.167
  16:18:25 STEP 1275 H= 318:45 +CPU= 29.510
  16:18:55 STEP 1276 H= 319:00 +CPU= 30.391
  16:19:24 STEP 1277 H= 319:15 +CPU= 29.181
  16:19:53 STEP 1278 H= 319:30 +CPU= 29.012
  16:20:23 STEP 1279 H= 319:45 +CPU= 29.561
  16:20:53 STEP 1280 H= 320:00 +CPU= 29.893
  16:21:23 STEP 1281 H= 320:15 +CPU= 29.392
  16:21:52 STEP 1282 H= 320:30 +CPU= 29.124
  16:22:21 STEP 1283 H= 320:45 +CPU= 29.343
  16:22:58 STEP 1284 H= 321:00 +CPU= 36.217
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001284
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001284
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001284
  16:23:27 STEP 1285 H= 321:15 +CPU= 29.042
  16:23:57 STEP 1286 H= 321:30 +CPU= 29.954
  16:24:26 STEP 1287 H= 321:45 +CPU= 29.243
  16:24:56 STEP 1288 H= 322:00 +CPU= 29.835
  16:25:26 STEP 1289 H= 322:15 +CPU= 29.550
  16:25:55 STEP 1290 H= 322:30 +CPU= 29.305
  16:26:24 STEP 1291 H= 322:45 +CPU= 28.785
  16:26:55 STEP 1292 H= 323:00 +CPU= 30.070
  16:27:25 STEP 1293 H= 323:15 +CPU= 29.752
  16:27:55 STEP 1294 H= 323:30 +CPU= 30.270
  16:28:25 STEP 1295 H= 323:45 +CPU= 30.484
  16:29:06 STEP 1296 H= 324:00 +CPU= 40.430
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001296
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001296
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001296
  16:29:36 STEP 1297 H= 324:15 +CPU= 29.990
  16:30:06 STEP 1298 H= 324:30 +CPU= 29.419
  16:30:35 STEP 1299 H= 324:45 +CPU= 29.597
  16:31:06 STEP 1300 H= 325:00 +CPU= 30.189
  16:31:36 STEP 1301 H= 325:15 +CPU= 29.704
  16:32:06 STEP 1302 H= 325:30 +CPU= 30.309
  16:32:35 STEP 1303 H= 325:45 +CPU= 29.097
  16:33:06 STEP 1304 H= 326:00 +CPU= 29.810
  16:33:35 STEP 1305 H= 326:15 +CPU= 29.191
  16:34:05 STEP 1306 H= 326:30 +CPU= 29.659
  16:34:34 STEP 1307 H= 326:45 +CPU= 29.499
  16:35:12 STEP 1308 H= 327:00 +CPU= 37.206
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001308
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001308
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001308
  16:35:42 STEP 1309 H= 327:15 +CPU= 30.244
  16:36:12 STEP 1310 H= 327:30 +CPU= 29.608
  16:36:42 STEP 1311 H= 327:45 +CPU= 29.771
  16:37:13 STEP 1312 H= 328:00 +CPU= 30.514
  16:37:42 STEP 1313 H= 328:15 +CPU= 29.399
  16:38:11 STEP 1314 H= 328:30 +CPU= 29.025
  16:38:41 STEP 1315 H= 328:45 +CPU= 29.679
  16:39:11 STEP 1316 H= 329:00 +CPU= 29.527
  16:39:40 STEP 1317 H= 329:15 +CPU= 29.537
  16:40:10 STEP 1318 H= 329:30 +CPU= 29.655
  16:40:40 STEP 1319 H= 329:45 +CPU= 29.711
  16:41:18 STEP 1320 H= 330:00 +CPU= 37.611
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001320
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001320
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001320
  16:41:48 STEP 1321 H= 330:15 +CPU= 29.965
  16:42:18 STEP 1322 H= 330:30 +CPU= 30.014
  16:42:48 STEP 1323 H= 330:45 +CPU= 29.407
  16:43:18 STEP 1324 H= 331:00 +CPU= 29.897
  16:43:47 STEP 1325 H= 331:15 +CPU= 29.445
  16:44:17 STEP 1326 H= 331:30 +CPU= 29.342
  16:44:46 STEP 1327 H= 331:45 +CPU= 28.884
  16:45:16 STEP 1328 H= 332:00 +CPU= 29.803
  16:45:45 STEP 1329 H= 332:15 +CPU= 29.099
  16:46:15 STEP 1330 H= 332:30 +CPU= 29.925
  16:46:45 STEP 1331 H= 332:45 +CPU= 29.513
  16:47:21 STEP 1332 H= 333:00 +CPU= 36.428
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001332
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001332
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001332
  16:47:51 STEP 1333 H= 333:15 +CPU= 29.631
  16:48:22 STEP 1334 H= 333:30 +CPU= 30.543
  16:48:51 STEP 1335 H= 333:45 +CPU= 29.751
  16:49:23 STEP 1336 H= 334:00 +CPU= 31.043
  16:49:52 STEP 1337 H= 334:15 +CPU= 29.378
  16:50:21 STEP 1338 H= 334:30 +CPU= 29.038
  16:50:50 STEP 1339 H= 334:45 +CPU= 28.906
  16:51:20 STEP 1340 H= 335:00 +CPU= 29.891
  16:51:51 STEP 1341 H= 335:15 +CPU= 30.202
  16:52:21 STEP 1342 H= 335:30 +CPU= 29.325
  16:52:51 STEP 1343 H= 335:45 +CPU= 29.983
  16:53:31 STEP 1344 H= 336:00 +CPU= 40.235
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001344
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001344
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001344
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001248
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001248
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001248
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001260
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001260
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001260
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001272
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001272
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001272
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001284
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001284
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001284
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001296
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001296
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001296
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001308
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001308
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001308
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001320
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001320
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001320
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001332
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001332
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001332
Zipping up the intermediate file: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_a2ru_2016092300_20_1017_12285421_1_r1015525513_13.zip
Uploading the intermediate file: upload_file_13.zip
  16:54:01 STEP 1345 H= 336:15 +CPU= 29.306
  16:54:31 STEP 1346 H= 336:30 +CPU= 29.389
  16:55:00 STEP 1347 H= 336:45 +CPU= 29.266
  16:55:30 STEP 1348 H= 337:00 +CPU= 30.004
  16:56:00 STEP 1349 H= 337:15 +CPU= 29.883
  16:56:30 STEP 1350 H= 337:30 +CPU= 29.296
  16:57:01 STEP 1351 H= 337:45 +CPU= 30.948
  16:57:31 STEP 1352 H= 338:00 +CPU= 30.545
  16:58:02 STEP 1353 H= 338:15 +CPU= 29.932
  16:58:31 STEP 1354 H= 338:30 +CPU= 29.564
  16:59:01 STEP 1355 H= 338:45 +CPU= 29.705
  16:59:38 STEP 1356 H= 339:00 +CPU= 36.629
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001356
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001356
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001356
  17:00:08 STEP 1357 H= 339:15 +CPU= 30.038
  17:00:37 STEP 1358 H= 339:30 +CPU= 29.075
  17:01:07 STEP 1359 H= 339:45 +CPU= 29.637
  17:01:37 STEP 1360 H= 340:00 +CPU= 30.273
  17:02:07 STEP 1361 H= 340:15 +CPU= 29.852
  17:02:37 STEP 1362 H= 340:30 +CPU= 29.508
  17:03:07 STEP 1363 H= 340:45 +CPU= 30.101
  17:03:37 STEP 1364 H= 341:00 +CPU= 30.008
  17:04:07 STEP 1365 H= 341:15 +CPU= 29.640
  17:04:37 STEP 1366 H= 341:30 +CPU= 29.450
  17:05:06 STEP 1367 H= 341:45 +CPU= 29.197
  17:05:44 STEP 1368 H= 342:00 +CPU= 37.602
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001368
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001368
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001368
  17:06:13 STEP 1369 H= 342:15 +CPU= 29.309
  17:06:43 STEP 1370 H= 342:30 +CPU= 29.948
  17:07:13 STEP 1371 H= 342:45 +CPU= 29.196
  17:07:43 STEP 1372 H= 343:00 +CPU= 30.346
  17:08:13 STEP 1373 H= 343:15 +CPU= 29.648
  17:08:43 STEP 1374 H= 343:30 +CPU= 30.316
  17:09:13 STEP 1375 H= 343:45 +CPU= 29.268
  17:09:43 STEP 1376 H= 344:00 +CPU= 30.525
  17:10:13 STEP 1377 H= 344:15 +CPU= 29.381
  17:10:42 STEP 1378 H= 344:30 +CPU= 29.178
  17:11:12 STEP 1379 H= 344:45 +CPU= 30.243
  17:11:50 STEP 1380 H= 345:00 +CPU= 37.084
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001380
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001380
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001380
  17:12:19 STEP 1381 H= 345:15 +CPU= 29.472
  17:12:49 STEP 1382 H= 345:30 +CPU= 29.256
  17:13:19 STEP 1383 H= 345:45 +CPU= 29.841
  17:13:48 STEP 1384 H= 346:00 +CPU= 29.589
  17:14:19 STEP 1385 H= 346:15 +CPU= 29.973
  17:14:49 STEP 1386 H= 346:30 +CPU= 30.234
  17:15:19 STEP 1387 H= 346:45 +CPU= 29.502
  17:15:49 STEP 1388 H= 347:00 +CPU= 29.741
  17:16:18 STEP 1389 H= 347:15 +CPU= 29.242
  17:16:48 STEP 1390 H= 347:30 +CPU= 29.647
  17:17:17 STEP 1391 H= 347:45 +CPU= 28.922
  17:17:57 STEP 1392 H= 348:00 +CPU= 40.217
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001392
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001392
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001392
  17:18:28 STEP 1393 H= 348:15 +CPU= 30.234
  17:18:57 STEP 1394 H= 348:30 +CPU= 29.454
  17:19:27 STEP 1395 H= 348:45 +CPU= 29.517
  17:19:57 STEP 1396 H= 349:00 +CPU= 29.559
  17:20:26 STEP 1397 H= 349:15 +CPU= 29.538
  17:20:56 STEP 1398 H= 349:30 +CPU= 29.557
  17:21:26 STEP 1399 H= 349:45 +CPU= 29.464
  17:21:57 STEP 1400 H= 350:00 +CPU= 31.130
  17:22:27 STEP 1401 H= 350:15 +CPU= 30.011
  17:22:56 STEP 1402 H= 350:30 +CPU= 29.113
  17:23:26 STEP 1403 H= 350:45 +CPU= 29.587
  17:24:03 STEP 1404 H= 351:00 +CPU= 36.560
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001404
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001404
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001404
  17:24:32 STEP 1405 H= 351:15 +CPU= 29.311
  17:25:02 STEP 1406 H= 351:30 +CPU= 29.341
  17:25:32 STEP 1407 H= 351:45 +CPU= 30.159
  17:26:02 STEP 1408 H= 352:00 +CPU= 29.891
  17:26:32 STEP 1409 H= 352:15 +CPU= 29.393
  17:27:01 STEP 1410 H= 352:30 +CPU= 29.574
  17:27:31 STEP 1411 H= 352:45 +CPU= 29.512
  17:28:01 STEP 1412 H= 353:00 +CPU= 30.209
  17:28:31 STEP 1413 H= 353:15 +CPU= 29.425
  17:29:00 STEP 1414 H= 353:30 +CPU= 29.309
  17:29:29 STEP 1415 H= 353:45 +CPU= 29.012
  17:30:07 STEP 1416 H= 354:00 +CPU= 37.527
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001416
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001416
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001416
  17:30:37 STEP 1417 H= 354:15 +CPU= 29.520
  17:31:06 STEP 1418 H= 354:30 +CPU= 29.196
  17:31:36 STEP 1419 H= 354:45 +CPU= 29.627
  17:32:08 STEP 1420 H= 355:00 +CPU= 31.751
  17:32:39 STEP 1421 H= 355:15 +CPU= 29.816
  17:33:08 STEP 1422 H= 355:30 +CPU= 29.023
  17:33:37 STEP 1423 H= 355:45 +CPU= 29.249
  17:34:07 STEP 1424 H= 356:00 +CPU= 29.581
  17:34:36 STEP 1425 H= 356:15 +CPU= 29.054
  17:35:06 STEP 1426 H= 356:30 +CPU= 29.476
  17:35:35 STEP 1427 H= 356:45 +CPU= 29.382
  17:36:12 STEP 1428 H= 357:00 +CPU= 36.477
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001428
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001428
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001428
  17:36:42 STEP 1429 H= 357:15 +CPU= 29.891
  17:37:12 STEP 1430 H= 357:30 +CPU= 29.792
  17:37:42 STEP 1431 H= 357:45 +CPU= 30.150
  17:38:12 STEP 1432 H= 358:00 +CPU= 29.799
  17:38:42 STEP 1433 H= 358:15 +CPU= 29.928
  17:39:11 STEP 1434 H= 358:30 +CPU= 29.255
  17:39:42 STEP 1435 H= 358:45 +CPU= 30.473
  17:40:12 STEP 1436 H= 359:00 +CPU= 29.460
  17:40:40 STEP 1437 H= 359:15 +CPU= 28.702
  17:41:09 STEP 1438 H= 359:30 +CPU= 28.756
  17:41:38 STEP 1439 H= 359:45 +CPU= 29.053
  17:42:15 STEP 1440 H= 360:00 +CPU= 36.630
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  360.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.1845122821E-20      I       0.1943295788E-06      I
ICloud fractionI      0.0000000000E+00      I       0.2516281586E-02      I
I  Relat. Hum. I      0.1654445057E+00      I       0.1939411015E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.4464576776E+02      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1440 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001440 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001440
 MPI-TASK:   1 -    325226148 BYTES IN      7 RECORDS TRANSFERRED IN    0.0003 SECONDS******** Mbytes/s, TOTAL TIME=    0.0115( 2.6%)
 17:42:15 STEP 1440 H= 360:00 +CPU= 36.630
 END CNT3
 NSTEP =  1440 CNT0   000000000  27.011  27.011  27.120   0.996               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN        17160/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    303    0       46      1044068    17160 ************    29.3     1.2    28.1     1.9    61.6
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********     99.98      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  625    4964.9    4964.9     17.97      0.00
   9 SPCM     - SPECTRAL COMP.                     576     846.5     846.5      2.82      0.00
  10 SCAN2M - PHYSICS                              577   13052.7   13052.7     43.61      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 48     854.8     854.8      0.24      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 25      49.4      49.4      0.01      0.00
  17 GRIDFPOS IN CNT4                               48      29.0      29.0      0.01      0.00
  27 RESTART FILES - WRITING                        12    3419.1    3419.1      0.24      0.00
  28 RESTART FILES - READING                         1    2531.3    2531.3      0.01      0.00
  29 SU4FPOS IN CNT4                                48       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                48    6323.6    6323.6      1.76      0.00
  31 POSDDH IN STEPO                                 3       1.0       1.0      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    577    7023.4    7023.4     23.47      0.00
  39 SU0YOMB                                         1     327.0     327.0      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    577      23.6      23.6      0.08      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              576     351.5     351.5      1.17      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              576      71.1      71.1      0.24      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 25       0.2       0.2      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM     1252     498.6     498.6      3.61      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM      1200     288.8     288.8      2.01      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM       1200     104.9     104.9      0.73      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM      1252     143.8     143.8      1.04      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      29.7      29.7      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION           1252      71.0      71.0      0.52      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION           1200      34.3      34.3      0.24      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION           1252      84.9      84.9      0.62      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION           1200      19.5      19.5      0.14      0.00
 400 GSTATS                                      67705       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 62941       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    17269.5 CPU TIME   17181.0 VECTOR TIME    17181.0


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY       75.1
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMGGh7zg+001440
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMSHh7zg+001440
Moving to projects directory: /mydisks/a/boinc_lib/slots/31/ICMUAh7zg+001440
Adding to the zip: /mydisks/a/boinc_lib/slots/31/NODE.001_01
Adding to the zip: /mydisks/a/boinc_lib/slots/31/ifs.stat
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001344
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001344
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001344
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001356
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001356
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001356
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001368
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001368
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001368
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001380
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001380
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001380
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001392
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001392
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001392
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001404
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001404
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001404
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001416
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001416
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001416
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001428
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001428
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001428
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMGGh7zg+001440
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMSHh7zg+001440
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12285421/ICMUAh7zg+001440
Zipping up the final file: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_a2ru_2016092300_20_1017_12285421_1_r1015525513_14.zip
Uploading the final file: upload_file_14.zip
Uploading trickle at timestep: 1295100
17:45:47 (3395018): called boinc_finish(0)

</stderr_txt>
<message>
upload failure: <file_xfer_error>
  <file_name>oifs_43r3_bl_a2ru_2016092300_20_1017_12285421_1_r1015525513_15.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a2ru_2016092300_20_1017_12285421_1_r1015525513_16.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a2ru_2016092300_20_1017_12285421_1_r1015525513_17.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a2ru_2016092300_20_1017_12285421_1_r1015525513_18.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a2ru_2016092300_20_1017_12285421_1_r1015525513_19.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
</message>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 Jun 2024 00:52:12 1517479 22440766 oifs_43r3_bl_a2ru_2016092300_20_1017_12285421_1 1,295,100 44,475 0.0343


©2024 climateprediction.net