climateprediction.net home page
Task 22434949

Task 22434949

Name oifs_43r3_bl_a16w_2016092300_20_1017_12283371_0
Workunit 12283371
Created 7 Jun 2024, 13:04:51 UTC
Sent 8 Jun 2024, 4:14:38 UTC
Report deadline 7 Aug 2024, 4:14:38 UTC
Received 8 Jun 2024, 18:02:12 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 0 (0x00000000)
Computer ID 1459164
Run time 10 hours 50 min 25 sec
CPU time 10 hours 50 min 25 sec
Validate state Invalid
Credit 1,976.32
Device peak FLOPS 4.86 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,549.71 MB
Peak swap size 5,980.79 MB
Peak disk usage 1,362.73 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<stderr_txt>
10:47 STEP  960 H= 240:00 +CPU= 31.035
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+000960
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+000960
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+000948
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a16w_2016092300_20_1017_12283371_0_r917049641_9.zip
Uploading the intermediate file: upload_file_9.zip
  08:11:11 STEP  961 H= 240:15 +CPU= 24.115
  08:11:36 STEP  962 H= 240:30 +CPU= 25.117
  08:11:59 STEP  963 H= 240:45 +CPU= 23.524
  08:12:25 STEP  964 H= 241:00 +CPU= 25.433
  08:12:47 STEP  965 H= 241:15 +CPU= 22.405
  08:13:11 STEP  966 H= 241:30 +CPU= 24.256
  08:13:38 STEP  967 H= 241:45 +CPU= 26.799
  08:14:01 STEP  968 H= 242:00 +CPU= 23.011
  08:14:26 STEP  969 H= 242:15 +CPU= 24.364
  08:14:51 STEP  970 H= 242:30 +CPU= 25.460
  08:15:17 STEP  971 H= 242:45 +CPU= 25.645
  08:15:45 STEP  972 H= 243:00 +CPU= 27.965
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+000972
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+000972
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+000972
  08:16:16 STEP  973 H= 243:15 +CPU= 31.409
  08:16:43 STEP  974 H= 243:30 +CPU= 27.191
  08:17:10 STEP  975 H= 243:45 +CPU= 26.135
  08:17:34 STEP  976 H= 244:00 +CPU= 24.968
  08:17:59 STEP  977 H= 244:15 +CPU= 24.941
  08:18:28 STEP  978 H= 244:30 +CPU= 28.414
  08:18:52 STEP  979 H= 244:45 +CPU= 24.276
  08:19:18 STEP  980 H= 245:00 +CPU= 25.639
  08:19:45 STEP  981 H= 245:15 +CPU= 27.174
  08:20:13 STEP  982 H= 245:30 +CPU= 27.584
  08:20:37 STEP  983 H= 245:45 +CPU= 24.345
  08:21:06 STEP  984 H= 246:00 +CPU= 28.853
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+000984
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+000984
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+000984
  08:21:33 STEP  985 H= 246:15 +CPU= 27.312
  08:21:57 STEP  986 H= 246:30 +CPU= 24.080
  08:22:26 STEP  987 H= 246:45 +CPU= 28.986
  08:22:53 STEP  988 H= 247:00 +CPU= 27.055
  08:23:18 STEP  989 H= 247:15 +CPU= 24.482
  08:23:43 STEP  990 H= 247:30 +CPU= 24.935
  08:24:14 STEP  991 H= 247:45 +CPU= 30.869
  08:24:45 STEP  992 H= 248:00 +CPU= 31.162
  08:25:13 STEP  993 H= 248:15 +CPU= 28.202
  08:25:42 STEP  994 H= 248:30 +CPU= 28.800
  08:26:06 STEP  995 H= 248:45 +CPU= 24.405
  08:26:42 STEP  996 H= 249:00 +CPU= 35.292
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+000996
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+000996
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+000996
  08:27:14 STEP  997 H= 249:15 +CPU= 31.956
  08:27:41 STEP  998 H= 249:30 +CPU= 26.964
  08:28:07 STEP  999 H= 249:45 +CPU= 25.871
  08:28:37 STEP 1000 H= 250:00 +CPU= 30.778
  08:29:02 STEP 1001 H= 250:15 +CPU= 24.152
  08:29:25 STEP 1002 H= 250:30 +CPU= 23.502
  08:29:47 STEP 1003 H= 250:45 +CPU= 21.884
  08:30:11 STEP 1004 H= 251:00 +CPU= 24.459
  08:30:34 STEP 1005 H= 251:15 +CPU= 22.432
  08:30:56 STEP 1006 H= 251:30 +CPU= 21.817
  08:31:19 STEP 1007 H= 251:45 +CPU= 23.471
  08:31:58 STEP 1008 H= 252:00 +CPU= 38.610
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001008
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001008
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001008
  08:32:21 STEP 1009 H= 252:15 +CPU= 22.853
  08:32:51 STEP 1010 H= 252:30 +CPU= 30.538
  08:33:20 STEP 1011 H= 252:45 +CPU= 29.121
  08:33:43 STEP 1012 H= 253:00 +CPU= 22.738
  08:34:06 STEP 1013 H= 253:15 +CPU= 22.844
  08:34:30 STEP 1014 H= 253:30 +CPU= 24.251
  08:34:53 STEP 1015 H= 253:45 +CPU= 22.601
  08:35:19 STEP 1016 H= 254:00 +CPU= 26.472
  08:35:50 STEP 1017 H= 254:15 +CPU= 30.381
  08:36:18 STEP 1018 H= 254:30 +CPU= 28.394
  08:36:49 STEP 1019 H= 254:45 +CPU= 30.447
  08:37:20 STEP 1020 H= 255:00 +CPU= 31.481
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001020
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001020
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001020
  08:37:47 STEP 1021 H= 255:15 +CPU= 27.268
  08:38:10 STEP 1022 H= 255:30 +CPU= 23.079
  08:38:34 STEP 1023 H= 255:45 +CPU= 23.960
  08:39:00 STEP 1024 H= 256:00 +CPU= 25.638
  08:39:27 STEP 1025 H= 256:15 +CPU= 26.532
  08:39:51 STEP 1026 H= 256:30 +CPU= 24.725
  08:40:20 STEP 1027 H= 256:45 +CPU= 28.689
  08:40:43 STEP 1028 H= 257:00 +CPU= 23.392
  08:41:07 STEP 1029 H= 257:15 +CPU= 24.000
  08:41:32 STEP 1030 H= 257:30 +CPU= 24.684
  08:41:55 STEP 1031 H= 257:45 +CPU= 22.542
  08:42:24 STEP 1032 H= 258:00 +CPU= 29.100
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001032
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001032
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001032
  08:42:47 STEP 1033 H= 258:15 +CPU= 23.030
  08:43:13 STEP 1034 H= 258:30 +CPU= 25.858
  08:43:46 STEP 1035 H= 258:45 +CPU= 33.156
  08:44:16 STEP 1036 H= 259:00 +CPU= 30.431
  08:44:43 STEP 1037 H= 259:15 +CPU= 26.786
  08:45:10 STEP 1038 H= 259:30 +CPU= 26.455
  08:45:38 STEP 1039 H= 259:45 +CPU= 28.303
  08:46:06 STEP 1040 H= 260:00 +CPU= 27.827
  08:46:32 STEP 1041 H= 260:15 +CPU= 25.669
  08:46:58 STEP 1042 H= 260:30 +CPU= 25.973
  08:47:25 STEP 1043 H= 260:45 +CPU= 27.404
  08:48:05 STEP 1044 H= 261:00 +CPU= 39.656
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001044
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001044
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001044
  08:48:29 STEP 1045 H= 261:15 +CPU= 24.496
  08:48:52 STEP 1046 H= 261:30 +CPU= 23.095
  08:49:15 STEP 1047 H= 261:45 +CPU= 22.792
  08:49:38 STEP 1048 H= 262:00 +CPU= 23.106
  08:50:06 STEP 1049 H= 262:15 +CPU= 27.873
  08:50:32 STEP 1050 H= 262:30 +CPU= 26.123
  08:50:54 STEP 1051 H= 262:45 +CPU= 22.183
  08:51:22 STEP 1052 H= 263:00 +CPU= 27.606
  08:51:44 STEP 1053 H= 263:15 +CPU= 22.324
  08:52:09 STEP 1054 H= 263:30 +CPU= 24.121
  08:52:32 STEP 1055 H= 263:45 +CPU= 23.454
  08:53:13 STEP 1056 H= 264:00 +CPU= 40.957
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001056
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001056
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001044
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a16w_2016092300_20_1017_12283371_0_r917049641_10.zip
Uploading the intermediate file: upload_file_10.zip
  08:53:40 STEP 1057 H= 264:15 +CPU= 26.438
  08:54:05 STEP 1058 H= 264:30 +CPU= 25.593
  08:54:32 STEP 1059 H= 264:45 +CPU= 26.495
  08:54:56 STEP 1060 H= 265:00 +CPU= 24.055
  08:55:19 STEP 1061 H= 265:15 +CPU= 22.930
  08:55:41 STEP 1062 H= 265:30 +CPU= 22.661
  08:56:04 STEP 1063 H= 265:45 +CPU= 22.749
  08:56:29 STEP 1064 H= 266:00 +CPU= 24.589
  08:56:57 STEP 1065 H= 266:15 +CPU= 28.006
  08:57:22 STEP 1066 H= 266:30 +CPU= 24.909
  08:57:48 STEP 1067 H= 266:45 +CPU= 25.730
  08:58:22 STEP 1068 H= 267:00 +CPU= 34.404
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001068
  08:58:51 STEP 1069 H= 267:15 +CPU= 29.180
  08:59:18 STEP 1070 H= 267:30 +CPU= 26.802
  08:59:41 STEP 1071 H= 267:45 +CPU= 22.834
  09:00:07 STEP 1072 H= 268:00 +CPU= 25.939
  09:00:31 STEP 1073 H= 268:15 +CPU= 24.585
  09:00:54 STEP 1074 H= 268:30 +CPU= 22.385
  09:01:19 STEP 1075 H= 268:45 +CPU= 24.720
  09:01:43 STEP 1076 H= 269:00 +CPU= 24.054
  09:02:09 STEP 1077 H= 269:15 +CPU= 26.586
  09:02:34 STEP 1078 H= 269:30 +CPU= 24.875
  09:03:05 STEP 1079 H= 269:45 +CPU= 31.106
  09:03:43 STEP 1080 H= 270:00 +CPU= 37.805
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001080
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001080
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001080
  09:04:10 STEP 1081 H= 270:15 +CPU= 27.440
  09:04:41 STEP 1082 H= 270:30 +CPU= 30.747
  09:05:05 STEP 1083 H= 270:45 +CPU= 23.464
  09:05:30 STEP 1084 H= 271:00 +CPU= 25.682
  09:06:00 STEP 1085 H= 271:15 +CPU= 29.446
  09:06:26 STEP 1086 H= 271:30 +CPU= 25.913
  09:06:49 STEP 1087 H= 271:45 +CPU= 22.791
  09:07:13 STEP 1088 H= 272:00 +CPU= 23.871
  09:07:42 STEP 1089 H= 272:15 +CPU= 29.415
  09:08:05 STEP 1090 H= 272:30 +CPU= 23.204
  09:08:32 STEP 1091 H= 272:45 +CPU= 27.107
  09:09:05 STEP 1092 H= 273:00 +CPU= 33.055
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001092
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001092
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001092
  09:09:28 STEP 1093 H= 273:15 +CPU= 22.243
  09:09:50 STEP 1094 H= 273:30 +CPU= 22.232
  09:10:12 STEP 1095 H= 273:45 +CPU= 22.324
  09:10:42 STEP 1096 H= 274:00 +CPU= 29.774
  09:11:04 STEP 1097 H= 274:15 +CPU= 21.797
  09:11:26 STEP 1098 H= 274:30 +CPU= 22.236
  09:11:50 STEP 1099 H= 274:45 +CPU= 23.667
  09:12:12 STEP 1100 H= 275:00 +CPU= 22.600
  09:12:37 STEP 1101 H= 275:15 +CPU= 24.137
  09:13:00 STEP 1102 H= 275:30 +CPU= 23.430
  09:13:29 STEP 1103 H= 275:45 +CPU= 28.734
  09:14:04 STEP 1104 H= 276:00 +CPU= 35.669
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001104
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001104
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001104
  09:14:28 STEP 1105 H= 276:15 +CPU= 23.576
  09:14:52 STEP 1106 H= 276:30 +CPU= 24.179
  09:15:20 STEP 1107 H= 276:45 +CPU= 27.972
  09:15:49 STEP 1108 H= 277:00 +CPU= 28.880
  09:16:12 STEP 1109 H= 277:15 +CPU= 23.412
  09:16:37 STEP 1110 H= 277:30 +CPU= 24.600
  09:16:59 STEP 1111 H= 277:45 +CPU= 22.083
  09:17:23 STEP 1112 H= 278:00 +CPU= 23.778
  09:17:46 STEP 1113 H= 278:15 +CPU= 23.464
  09:18:08 STEP 1114 H= 278:30 +CPU= 22.056
  09:18:32 STEP 1115 H= 278:45 +CPU= 23.150
  09:19:06 STEP 1116 H= 279:00 +CPU= 34.081
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001116
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001116
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001116
  09:19:30 STEP 1117 H= 279:15 +CPU= 23.863
  09:19:55 STEP 1118 H= 279:30 +CPU= 25.542
  09:20:18 STEP 1119 H= 279:45 +CPU= 23.151
  09:20:43 STEP 1120 H= 280:00 +CPU= 24.479
  09:21:05 STEP 1121 H= 280:15 +CPU= 22.113
  09:21:28 STEP 1122 H= 280:30 +CPU= 23.430
  09:21:51 STEP 1123 H= 280:45 +CPU= 22.848
  09:22:15 STEP 1124 H= 281:00 +CPU= 23.834
  09:22:40 STEP 1125 H= 281:15 +CPU= 24.415
  09:23:02 STEP 1126 H= 281:30 +CPU= 22.570
  09:23:25 STEP 1127 H= 281:45 +CPU= 22.657
  09:23:56 STEP 1128 H= 282:00 +CPU= 31.191
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001128
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001128
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001128
  09:24:20 STEP 1129 H= 282:15 +CPU= 23.663
  09:24:43 STEP 1130 H= 282:30 +CPU= 23.125
  09:25:05 STEP 1131 H= 282:45 +CPU= 21.977
  09:25:31 STEP 1132 H= 283:00 +CPU= 26.028
  09:26:03 STEP 1133 H= 283:15 +CPU= 31.840
  09:26:30 STEP 1134 H= 283:30 +CPU= 26.995
  09:26:57 STEP 1135 H= 283:45 +CPU= 27.080
  09:27:20 STEP 1136 H= 284:00 +CPU= 23.118
  09:27:42 STEP 1137 H= 284:15 +CPU= 22.276
  09:28:05 STEP 1138 H= 284:30 +CPU= 22.716
  09:28:33 STEP 1139 H= 284:45 +CPU= 28.328
  09:29:06 STEP 1140 H= 285:00 +CPU= 32.379
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001140
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001140
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001140
  09:29:33 STEP 1141 H= 285:15 +CPU= 27.089
  09:29:55 STEP 1142 H= 285:30 +CPU= 21.853
  09:30:16 STEP 1143 H= 285:45 +CPU= 21.435
  09:30:38 STEP 1144 H= 286:00 +CPU= 21.763
  09:31:04 STEP 1145 H= 286:15 +CPU= 25.944
  09:31:31 STEP 1146 H= 286:30 +CPU= 26.942
  09:31:55 STEP 1147 H= 286:45 +CPU= 24.476
  09:32:17 STEP 1148 H= 287:00 +CPU= 21.899
  09:32:38 STEP 1149 H= 287:15 +CPU= 21.087
  09:33:00 STEP 1150 H= 287:30 +CPU= 21.130
  09:33:24 STEP 1151 H= 287:45 +CPU= 24.312
  09:34:04 STEP 1152 H= 288:00 +CPU= 39.170
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001152
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001152
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001140
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a16w_2016092300_20_1017_12283371_0_r917049641_11.zip
Uploading the intermediate file: upload_file_11.zip
  09:34:35 STEP 1153 H= 288:15 +CPU= 31.347
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a16w
(argv4) batchid: 1017
(argv5) wuid: 12283371
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/6
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/6/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/6/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_2368907c0f9592895e8cde4b12ba5aed to: /var/lib/boinc-client/slots/6/oifs_43r3_bl_a16w_2016092300_20_1017_12283371.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/6/oifs_43r3_bl_a16w_2016092300_20_1017_12283371.zip
ic_ancil_file: ic_ancil_12283371
ifsdata_file: ifsdata_12283371
climate_data_file: clim_data_12283371
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/6/ic_ancil_12283371.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/6/ic_ancil_12283371.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_54241b6ef3bb4c5890e4bcd943aa3b3f to: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12283371.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12283371.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/6/159l_2/clim_data_12283371.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/6/159l_2/clim_data_12283371.zip
Checking for progress XML file: /var/lib/boinc-client/slots/6/progress_file_12283371.xml
Opened progress file ok : /var/lib/boinc-client/slots/6/progress_file_12283371.xml
-- Model is restarting --
Adjusting last_iter, 1153, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/6/progress_file_12283371.xml
last_cpu_time: 31616
upload_file_number: 12
last_iter: 1153
last_upload: 1036800
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a16w_2016092300_20_1017_12283371_0_r917049641
The child process has been launched with process id: 12452
Executing the command: /var/lib/boinc-client/slots/6/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:boinc-03.fullford.com:1:1:12452:12452] [20240608:094950:1717858190:0.001] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  09:50:24 STEP 1152 H= 288:00 +CPU= 27.559
  09:50:48 STEP 1153 H= 288:15 +CPU= 23.269
  09:51:14 STEP 1154 H= 288:30 +CPU= 26.852
  09:51:37 STEP 1155 H= 288:45 +CPU= 22.724
  09:52:00 STEP 1156 H= 289:00 +CPU= 23.197
  09:52:23 STEP 1157 H= 289:15 +CPU= 22.653
  09:52:46 STEP 1158 H= 289:30 +CPU= 22.649
  09:53:08 STEP 1159 H= 289:45 +CPU= 22.589
  09:53:35 STEP 1160 H= 290:00 +CPU= 26.931
  09:54:01 STEP 1161 H= 290:15 +CPU= 25.346
  09:54:27 STEP 1162 H= 290:30 +CPU= 26.042
  09:54:52 STEP 1163 H= 290:45 +CPU= 24.998
  09:55:27 STEP 1164 H= 291:00 +CPU= 35.025
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001164
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001164
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001164
  09:55:52 STEP 1165 H= 291:15 +CPU= 25.041
  09:56:18 STEP 1166 H= 291:30 +CPU= 26.543
  09:56:46 STEP 1167 H= 291:45 +CPU= 27.918
  09:57:12 STEP 1168 H= 292:00 +CPU= 25.694
  09:57:37 STEP 1169 H= 292:15 +CPU= 24.548
  09:58:04 STEP 1170 H= 292:30 +CPU= 26.970
  09:58:26 STEP 1171 H= 292:45 +CPU= 22.968
  09:58:50 STEP 1172 H= 293:00 +CPU= 24.008
  09:59:15 STEP 1173 H= 293:15 +CPU= 24.382
  09:59:37 STEP 1174 H= 293:30 +CPU= 21.818
  09:59:58 STEP 1175 H= 293:45 +CPU= 21.789
  10:00:28 STEP 1176 H= 294:00 +CPU= 29.765
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001176
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001176
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001176
  10:00:52 STEP 1177 H= 294:15 +CPU= 23.549
  10:01:18 STEP 1178 H= 294:30 +CPU= 25.685
  10:01:43 STEP 1179 H= 294:45 +CPU= 25.040
  10:02:08 STEP 1180 H= 295:00 +CPU= 25.731
  10:02:33 STEP 1181 H= 295:15 +CPU= 24.479
  10:02:57 STEP 1182 H= 295:30 +CPU= 23.690
  10:03:20 STEP 1183 H= 295:45 +CPU= 23.026
  10:03:43 STEP 1184 H= 296:00 +CPU= 22.998
  10:04:05 STEP 1185 H= 296:15 +CPU= 22.386
  10:04:27 STEP 1186 H= 296:30 +CPU= 22.453
  10:04:52 STEP 1187 H= 296:45 +CPU= 24.792
  10:05:22 STEP 1188 H= 297:00 +CPU= 29.952
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001188
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001188
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001188
  10:05:47 STEP 1189 H= 297:15 +CPU= 24.425
  10:06:13 STEP 1190 H= 297:30 +CPU= 26.428
  10:06:38 STEP 1191 H= 297:45 +CPU= 25.391
  10:07:02 STEP 1192 H= 298:00 +CPU= 23.301
  10:07:24 STEP 1193 H= 298:15 +CPU= 22.071
  10:07:50 STEP 1194 H= 298:30 +CPU= 26.369
  10:08:15 STEP 1195 H= 298:45 +CPU= 24.267
  10:08:39 STEP 1196 H= 299:00 +CPU= 24.854
  10:09:05 STEP 1197 H= 299:15 +CPU= 25.322
  10:09:28 STEP 1198 H= 299:30 +CPU= 23.131
  10:09:55 STEP 1199 H= 299:45 +CPU= 27.172
  10:10:30 STEP 1200 H= 300:00 +CPU= 34.508
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001200
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001200
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001200
  10:10:56 STEP 1201 H= 300:15 +CPU= 25.929
  10:11:21 STEP 1202 H= 300:30 +CPU= 24.534
  10:11:44 STEP 1203 H= 300:45 +CPU= 23.384
  10:12:08 STEP 1204 H= 301:00 +CPU= 24.257
  10:12:35 STEP 1205 H= 301:15 +CPU= 26.498
  10:13:01 STEP 1206 H= 301:30 +CPU= 25.995
  10:13:23 STEP 1207 H= 301:45 +CPU= 22.650
  10:13:48 STEP 1208 H= 302:00 +CPU= 24.172
  10:14:12 STEP 1209 H= 302:15 +CPU= 24.443
  10:14:36 STEP 1210 H= 302:30 +CPU= 24.067
  10:14:59 STEP 1211 H= 302:45 +CPU= 22.826
  10:15:29 STEP 1212 H= 303:00 +CPU= 30.292
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001212
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001212
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001212
  10:15:58 STEP 1213 H= 303:15 +CPU= 28.831
  10:16:23 STEP 1214 H= 303:30 +CPU= 25.053
  10:16:55 STEP 1215 H= 303:45 +CPU= 31.724
  10:17:21 STEP 1216 H= 304:00 +CPU= 25.796
  10:17:42 STEP 1217 H= 304:15 +CPU= 21.196
  10:18:06 STEP 1218 H= 304:30 +CPU= 23.897
  10:18:31 STEP 1219 H= 304:45 +CPU= 24.781
  10:18:58 STEP 1220 H= 305:00 +CPU= 27.136
  10:19:21 STEP 1221 H= 305:15 +CPU= 23.225
  10:19:44 STEP 1222 H= 305:30 +CPU= 22.475
  10:20:05 STEP 1223 H= 305:45 +CPU= 21.744
  10:20:42 STEP 1224 H= 306:00 +CPU= 36.470
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001224
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001224
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001224
  10:21:08 STEP 1225 H= 306:15 +CPU= 26.321
  10:21:34 STEP 1226 H= 306:30 +CPU= 25.350
  10:22:03 STEP 1227 H= 306:45 +CPU= 29.436
  10:22:29 STEP 1228 H= 307:00 +CPU= 25.679
  10:22:57 STEP 1229 H= 307:15 +CPU= 28.735
  10:23:24 STEP 1230 H= 307:30 +CPU= 26.450
  10:23:45 STEP 1231 H= 307:45 +CPU= 21.339
  10:24:11 STEP 1232 H= 308:00 +CPU= 25.933
  10:24:36 STEP 1233 H= 308:15 +CPU= 24.378
  10:24:59 STEP 1234 H= 308:30 +CPU= 23.501
  10:25:22 STEP 1235 H= 308:45 +CPU= 22.670
  10:25:49 STEP 1236 H= 309:00 +CPU= 27.572
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001236
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001236
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001236
  10:26:13 STEP 1237 H= 309:15 +CPU= 23.576
  10:26:38 STEP 1238 H= 309:30 +CPU= 25.074
  10:27:04 STEP 1239 H= 309:45 +CPU= 25.745
  10:27:28 STEP 1240 H= 310:00 +CPU= 24.486
  10:27:57 STEP 1241 H= 310:15 +CPU= 28.303
  10:28:21 STEP 1242 H= 310:30 +CPU= 24.250
  10:28:45 STEP 1243 H= 310:45 +CPU= 24.448
  10:29:13 STEP 1244 H= 311:00 +CPU= 27.112
  10:29:37 STEP 1245 H= 311:15 +CPU= 24.027
  10:29:59 STEP 1246 H= 311:30 +CPU= 22.712
  10:30:20 STEP 1247 H= 311:45 +CPU= 21.076
  10:30:55 STEP 1248 H= 312:00 +CPU= 34.637
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001248
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001248
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001236
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a16w_2016092300_20_1017_12283371_0_r917049641_12.zip
Uploading the intermediate file: upload_file_12.zip
  10:31:19 STEP 1249 H= 312:15 +CPU= 24.220
  10:31:42 STEP 1250 H= 312:30 +CPU= 22.774
  10:32:07 STEP 1251 H= 312:45 +CPU= 24.381
  10:32:31 STEP 1252 H= 313:00 +CPU= 24.173
  10:32:53 STEP 1253 H= 313:15 +CPU= 21.945
  10:33:15 STEP 1254 H= 313:30 +CPU= 22.162
  10:33:41 STEP 1255 H= 313:45 +CPU= 26.248
  10:34:06 STEP 1256 H= 314:00 +CPU= 24.691
  10:34:33 STEP 1257 H= 314:15 +CPU= 27.416
  10:34:55 STEP 1258 H= 314:30 +CPU= 21.730
  10:35:21 STEP 1259 H= 314:45 +CPU= 26.248
  10:35:52 STEP 1260 H= 315:00 +CPU= 30.539
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001260
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001260
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001260
  10:36:18 STEP 1261 H= 315:15 +CPU= 25.794
  10:36:45 STEP 1262 H= 315:30 +CPU= 27.174
  10:37:11 STEP 1263 H= 315:45 +CPU= 25.989
  10:37:36 STEP 1264 H= 316:00 +CPU= 24.708
  10:37:58 STEP 1265 H= 316:15 +CPU= 22.801
  10:38:29 STEP 1266 H= 316:30 +CPU= 30.523
  10:38:56 STEP 1267 H= 316:45 +CPU= 27.041
  10:39:25 STEP 1268 H= 317:00 +CPU= 28.892
  10:39:55 STEP 1269 H= 317:15 +CPU= 29.881
  10:40:17 STEP 1270 H= 317:30 +CPU= 22.530
  10:40:41 STEP 1271 H= 317:45 +CPU= 24.039
  10:41:15 STEP 1272 H= 318:00 +CPU= 33.676
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001272
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001272
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001272
  10:41:38 STEP 1273 H= 318:15 +CPU= 22.490
  10:42:04 STEP 1274 H= 318:30 +CPU= 26.438
  10:42:28 STEP 1275 H= 318:45 +CPU= 23.424
  10:42:57 STEP 1276 H= 319:00 +CPU= 29.048
  10:43:22 STEP 1277 H= 319:15 +CPU= 25.615
  10:43:47 STEP 1278 H= 319:30 +CPU= 24.429
  10:44:16 STEP 1279 H= 319:45 +CPU= 29.245
  10:44:41 STEP 1280 H= 320:00 +CPU= 24.929
  10:45:03 STEP 1281 H= 320:15 +CPU= 22.547
  10:45:27 STEP 1282 H= 320:30 +CPU= 23.210
  10:45:49 STEP 1283 H= 320:45 +CPU= 22.275
  10:46:21 STEP 1284 H= 321:00 +CPU= 32.500
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001284
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001284
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001284
  10:46:47 STEP 1285 H= 321:15 +CPU= 25.257
  10:47:11 STEP 1286 H= 321:30 +CPU= 23.860
  10:47:33 STEP 1287 H= 321:45 +CPU= 22.031
  10:47:56 STEP 1288 H= 322:00 +CPU= 22.986
  10:48:22 STEP 1289 H= 322:15 +CPU= 25.984
  10:48:48 STEP 1290 H= 322:30 +CPU= 26.255
  10:49:15 STEP 1291 H= 322:45 +CPU= 27.214
  10:49:44 STEP 1292 H= 323:00 +CPU= 29.109
  10:50:11 STEP 1293 H= 323:15 +CPU= 26.544
  10:50:35 STEP 1294 H= 323:30 +CPU= 24.227
  10:50:57 STEP 1295 H= 323:45 +CPU= 22.065
  10:51:31 STEP 1296 H= 324:00 +CPU= 33.811
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001296
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001296
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001296
  10:51:57 STEP 1297 H= 324:15 +CPU= 26.268
  10:52:24 STEP 1298 H= 324:30 +CPU= 27.072
  10:52:47 STEP 1299 H= 324:45 +CPU= 22.165
  10:53:10 STEP 1300 H= 325:00 +CPU= 23.550
  10:53:37 STEP 1301 H= 325:15 +CPU= 26.642
  10:53:59 STEP 1302 H= 325:30 +CPU= 22.654
  10:54:25 STEP 1303 H= 325:45 +CPU= 25.210
  10:54:51 STEP 1304 H= 326:00 +CPU= 26.109
  10:55:13 STEP 1305 H= 326:15 +CPU= 21.967
  10:55:35 STEP 1306 H= 326:30 +CPU= 22.125
  10:55:57 STEP 1307 H= 326:45 +CPU= 22.496
  10:56:25 STEP 1308 H= 327:00 +CPU= 27.703
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001308
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001308
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001308
  10:56:47 STEP 1309 H= 327:15 +CPU= 22.206
  10:57:08 STEP 1310 H= 327:30 +CPU= 20.976
  10:57:33 STEP 1311 H= 327:45 +CPU= 24.428
  10:58:01 STEP 1312 H= 328:00 +CPU= 28.160
  10:58:32 STEP 1313 H= 328:15 +CPU= 31.019
  10:58:55 STEP 1314 H= 328:30 +CPU= 23.329
  10:59:17 STEP 1315 H= 328:45 +CPU= 21.922
  10:59:40 STEP 1316 H= 329:00 +CPU= 23.250
  11:00:02 STEP 1317 H= 329:15 +CPU= 21.470
  11:00:26 STEP 1318 H= 329:30 +CPU= 23.881
  11:00:53 STEP 1319 H= 329:45 +CPU= 27.626
  11:01:27 STEP 1320 H= 330:00 +CPU= 33.389
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001320
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001320
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001320
  11:01:52 STEP 1321 H= 330:15 +CPU= 25.336
  11:02:14 STEP 1322 H= 330:30 +CPU= 21.656
  11:02:38 STEP 1323 H= 330:45 +CPU= 24.006
  11:03:00 STEP 1324 H= 331:00 +CPU= 22.257
  11:03:23 STEP 1325 H= 331:15 +CPU= 23.168
  11:03:46 STEP 1326 H= 331:30 +CPU= 22.740
  11:04:08 STEP 1327 H= 331:45 +CPU= 22.118
  11:04:35 STEP 1328 H= 332:00 +CPU= 26.514
  11:04:59 STEP 1329 H= 332:15 +CPU= 24.713
  11:05:26 STEP 1330 H= 332:30 +CPU= 26.102
  11:05:51 STEP 1331 H= 332:45 +CPU= 24.538
  11:06:18 STEP 1332 H= 333:00 +CPU= 27.044
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001332
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001332
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001332
  11:06:40 STEP 1333 H= 333:15 +CPU= 22.277
  11:07:03 STEP 1334 H= 333:30 +CPU= 22.475
  11:07:27 STEP 1335 H= 333:45 +CPU= 24.261
  11:07:55 STEP 1336 H= 334:00 +CPU= 27.485
  11:08:19 STEP 1337 H= 334:15 +CPU= 24.948
  11:08:45 STEP 1338 H= 334:30 +CPU= 25.903
  11:09:09 STEP 1339 H= 334:45 +CPU= 23.923
  11:09:35 STEP 1340 H= 335:00 +CPU= 25.727
  11:10:01 STEP 1341 H= 335:15 +CPU= 25.686
  11:10:24 STEP 1342 H= 335:30 +CPU= 23.105
  11:10:49 STEP 1343 H= 335:45 +CPU= 24.691
  11:11:23 STEP 1344 H= 336:00 +CPU= 34.543
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001344
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001344
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001332
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a16w_2016092300_20_1017_12283371_0_r917049641_13.zip
Uploading the intermediate file: upload_file_13.zip
  11:11:51 STEP 1345 H= 336:15 +CPU= 27.886
  11:12:23 STEP 1346 H= 336:30 +CPU= 31.410
  11:12:47 STEP 1347 H= 336:45 +CPU= 24.829
  11:13:12 STEP 1348 H= 337:00 +CPU= 24.986
  11:13:42 STEP 1349 H= 337:15 +CPU= 29.238
  11:14:12 STEP 1350 H= 337:30 +CPU= 30.188
  11:14:36 STEP 1351 H= 337:45 +CPU= 23.804
  11:15:01 STEP 1352 H= 338:00 +CPU= 25.616
  11:15:26 STEP 1353 H= 338:15 +CPU= 24.349
  11:15:49 STEP 1354 H= 338:30 +CPU= 23.580
  11:16:12 STEP 1355 H= 338:45 +CPU= 22.823
  11:16:46 STEP 1356 H= 339:00 +CPU= 33.935
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001356
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001356
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001356
  11:17:12 STEP 1357 H= 339:15 +CPU= 26.465
  11:17:38 STEP 1358 H= 339:30 +CPU= 25.959
  11:18:07 STEP 1359 H= 339:45 +CPU= 28.241
  11:18:31 STEP 1360 H= 340:00 +CPU= 24.677
  11:18:54 STEP 1361 H= 340:15 +CPU= 22.678
  11:19:18 STEP 1362 H= 340:30 +CPU= 23.515
  11:19:43 STEP 1363 H= 340:45 +CPU= 25.248
  11:20:11 STEP 1364 H= 341:00 +CPU= 28.450
  11:20:36 STEP 1365 H= 341:15 +CPU= 25.107
  11:21:04 STEP 1366 H= 341:30 +CPU= 27.800
  11:21:29 STEP 1367 H= 341:45 +CPU= 24.770
  11:22:03 STEP 1368 H= 342:00 +CPU= 34.351
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001368
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001368
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001368
  11:22:31 STEP 1369 H= 342:15 +CPU= 28.064
  11:22:56 STEP 1370 H= 342:30 +CPU= 24.218
  11:23:18 STEP 1371 H= 342:45 +CPU= 22.355
  11:23:51 STEP 1372 H= 343:00 +CPU= 33.169
  11:24:20 STEP 1373 H= 343:15 +CPU= 28.509
  11:24:45 STEP 1374 H= 343:30 +CPU= 24.772
  11:25:09 STEP 1375 H= 343:45 +CPU= 24.033
  11:25:33 STEP 1376 H= 344:00 +CPU= 24.486
  11:25:58 STEP 1377 H= 344:15 +CPU= 25.109
  11:26:23 STEP 1378 H= 344:30 +CPU= 24.431
  11:26:49 STEP 1379 H= 344:45 +CPU= 26.229
  11:27:24 STEP 1380 H= 345:00 +CPU= 35.172
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001380
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001380
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001380
  11:27:47 STEP 1381 H= 345:15 +CPU= 22.679
  11:28:14 STEP 1382 H= 345:30 +CPU= 26.803
  11:28:39 STEP 1383 H= 345:45 +CPU= 25.837
  11:29:14 STEP 1384 H= 346:00 +CPU= 34.521
  11:29:43 STEP 1385 H= 346:15 +CPU= 29.233
  11:30:07 STEP 1386 H= 346:30 +CPU= 23.614
  11:30:37 STEP 1387 H= 346:45 +CPU= 29.707
  11:31:09 STEP 1388 H= 347:00 +CPU= 31.968
  11:31:33 STEP 1389 H= 347:15 +CPU= 24.112
  11:31:59 STEP 1390 H= 347:30 +CPU= 26.115
  11:32:24 STEP 1391 H= 347:45 +CPU= 24.936
  11:32:56 STEP 1392 H= 348:00 +CPU= 32.339
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001392
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001392
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001392
  11:33:26 STEP 1393 H= 348:15 +CPU= 29.924
  11:33:49 STEP 1394 H= 348:30 +CPU= 23.310
  11:34:13 STEP 1395 H= 348:45 +CPU= 22.998
  11:34:38 STEP 1396 H= 349:00 +CPU= 25.780
  11:35:01 STEP 1397 H= 349:15 +CPU= 22.547
  11:35:24 STEP 1398 H= 349:30 +CPU= 22.726
  11:35:49 STEP 1399 H= 349:45 +CPU= 24.949
  11:36:14 STEP 1400 H= 350:00 +CPU= 25.735
  11:36:40 STEP 1401 H= 350:15 +CPU= 25.859
  11:37:07 STEP 1402 H= 350:30 +CPU= 26.490
  11:37:32 STEP 1403 H= 350:45 +CPU= 25.376
  11:38:13 STEP 1404 H= 351:00 +CPU= 40.847
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001404
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001404
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001404
  11:38:36 STEP 1405 H= 351:15 +CPU= 22.853
  11:39:00 STEP 1406 H= 351:30 +CPU= 24.671
  11:39:27 STEP 1407 H= 351:45 +CPU= 26.670
  11:39:54 STEP 1408 H= 352:00 +CPU= 26.820
  11:40:19 STEP 1409 H= 352:15 +CPU= 25.186
  11:40:43 STEP 1410 H= 352:30 +CPU= 23.421
  11:41:05 STEP 1411 H= 352:45 +CPU= 22.476
  11:41:31 STEP 1412 H= 353:00 +CPU= 25.911
  11:41:54 STEP 1413 H= 353:15 +CPU= 23.196
  11:42:21 STEP 1414 H= 353:30 +CPU= 27.118
  11:42:47 STEP 1415 H= 353:45 +CPU= 25.610
  11:43:17 STEP 1416 H= 354:00 +CPU= 29.628
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001416
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001416
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001416
  11:43:43 STEP 1417 H= 354:15 +CPU= 26.422
  11:44:10 STEP 1418 H= 354:30 +CPU= 26.881
  11:44:34 STEP 1419 H= 354:45 +CPU= 24.447
  11:45:02 STEP 1420 H= 355:00 +CPU= 27.170
  11:45:26 STEP 1421 H= 355:15 +CPU= 24.056
  11:45:54 STEP 1422 H= 355:30 +CPU= 28.005
  11:46:17 STEP 1423 H= 355:45 +CPU= 23.388
  11:46:42 STEP 1424 H= 356:00 +CPU= 24.952
  11:47:05 STEP 1425 H= 356:15 +CPU= 22.904
  11:47:28 STEP 1426 H= 356:30 +CPU= 23.353
  11:47:51 STEP 1427 H= 356:45 +CPU= 22.834
  11:48:19 STEP 1428 H= 357:00 +CPU= 27.836
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001428
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001428
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001428
  11:48:45 STEP 1429 H= 357:15 +CPU= 26.476
  11:49:14 STEP 1430 H= 357:30 +CPU= 28.239
  11:49:38 STEP 1431 H= 357:45 +CPU= 24.721
  11:50:04 STEP 1432 H= 358:00 +CPU= 25.218
  11:50:27 STEP 1433 H= 358:15 +CPU= 23.769
  11:50:51 STEP 1434 H= 358:30 +CPU= 23.555
  11:51:15 STEP 1435 H= 358:45 +CPU= 23.595
  11:51:38 STEP 1436 H= 359:00 +CPU= 23.020
  11:52:01 STEP 1437 H= 359:15 +CPU= 23.048
  11:52:23 STEP 1438 H= 359:30 +CPU= 22.740
  11:52:48 STEP 1439 H= 359:45 +CPU= 25.090
  11:53:28 STEP 1440 H= 360:00 +CPU= 39.149
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  360.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I      0.2490049237E-20      I       0.4021119848E-05      I
ICloud fractionI      0.0000000000E+00      I       0.4990515781E-01      I
I  Relat. Hum. I      0.5660277944E+00      I       0.5779929442E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.5942899417E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1440 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001440 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001440
 MPI-TASK:   1 -    309559460 BYTES IN      7 RECORDS TRANSFERRED IN    0.0003 SECONDS******** Mbytes/s, TOTAL TIME=    0.0086( 3.5%)
 11:53:28 STEP 1440 H= 360:00 +CPU= 39.149
 END CNT3
 NSTEP =  1440 CNT0   000000000  30.170  30.170  30.209   0.999               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN         8580/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    153    0       46      1044068     8580 ************    15.1     4.5    10.6     5.0    24.9
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 7418315.2 7418315.2    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 7412178.7 7412178.7     99.92      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  313    3327.0    3327.0     14.04      0.00
   9 SPCM     - SPECTRAL COMP.                     288     776.1     776.1      3.01      0.00
  10 SCAN2M - PHYSICS                              289   13703.6   13703.6     53.39      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 24     591.7     591.7      0.19      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 13      31.6      31.6      0.01      0.00
  17 GRIDFPOS IN CNT4                               24      27.8      27.8      0.01      0.00
  27 RESTART FILES - WRITING                         6    2366.5    2366.5      0.19      0.00
  28 RESTART FILES - READING                         1    5690.3    5690.3      0.08      0.00
  29 SU4FPOS IN CNT4                                24       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                24    5505.7    5505.7      1.78      0.00
  31 POSDDH IN STEPO                                 2       2.6       2.6      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    289    4773.8    4773.8     18.60      0.00
  39 SU0YOMB                                         1     304.0     304.0      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    289      15.2      15.2      0.06      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              288     298.5     298.5      1.16      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              288      36.6      36.6      0.14      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 13       0.3       0.3      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      628     369.8     369.8      3.13      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       600     225.5     225.5      1.82      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        600      72.6      72.6      0.59      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       628     136.9     136.9      1.16      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      51.0      51.0      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            628      21.2      21.2      0.18      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            600      15.1      15.1      0.12      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            628      75.0      75.0      0.64      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            600      21.1      21.1      0.17      0.00
 400 GSTATS                                      34412       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 31964       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME     7418.3 CPU TIME    7409.2 VECTOR TIME     7409.2


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      174.8
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001440
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001440
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client/slots/6/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/6/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMSHh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283371/ICMGGh7zg+001344
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a16w_2016092300_20_1017_12283371_0_r917049641_14.zip
Uploading the final file: upload_file_14.zip
Uploading trickle at timestep: 1295100
11:57:00 (12447): called boinc_finish(0)

</stderr_txt>
<message>
upload failure: <file_xfer_error>
  <file_name>oifs_43r3_bl_a16w_2016092300_20_1017_12283371_0_r917049641_15.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a16w_2016092300_20_1017_12283371_0_r917049641_16.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a16w_2016092300_20_1017_12283371_0_r917049641_17.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a16w_2016092300_20_1017_12283371_0_r917049641_18.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a16w_2016092300_20_1017_12283371_0_r917049641_19.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
</message>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
08 Jun 2024 17:01:58 1459164 22434949 oifs_43r3_bl_a16w_2016092300_20_1017_12283371_0 1,295,100 39,025 0.0301


©2024 climateprediction.net