climateprediction.net home page
Task 22310727

Task 22310727

Name oifs_43r3_bl_a2vf_2016092300_15_991_12210510_0
Workunit 12210510
Created 13 Feb 2023, 18:51:55 UTC
Sent 14 Feb 2023, 7:52:35 UTC
Report deadline 15 Apr 2023, 7:52:35 UTC
Received 16 Feb 2023, 18:28:23 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1506016
Run time 10 hours 46 min 35 sec
CPU time 10 hours 46 min 35 sec
Validate state Valid
Credit 1,813.14
Device peak FLOPS 5.52 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.11
x86_64-pc-linux-gnu
Peak working set size 4,974.61 MB
Peak swap size 5,930.79 MB
Peak disk usage 1,458.59 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<stderr_txt>
000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil))
[EC_DRHOOK:naturerick:1:1:14157:14157] [20230215:122921:1676492961:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  12:29:50 STEP  960 H= 240:00 +CPU= 24.170
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+000948
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a2vf_2016092300_15_991_12210510_0_r1547825834_9.zip
Uploading the intermediate file: upload_file_9.zip
  12:30:15 STEP  961 H= 240:15 +CPU= 23.182
  12:30:39 STEP  962 H= 240:30 +CPU= 23.058
  12:31:04 STEP  963 H= 240:45 +CPU= 23.109
  12:31:28 STEP  964 H= 241:00 +CPU= 23.169
  12:31:53 STEP  965 H= 241:15 +CPU= 23.184
  12:32:17 STEP  966 H= 241:30 +CPU= 23.274
  12:32:41 STEP  967 H= 241:45 +CPU= 23.123
  12:33:06 STEP  968 H= 242:00 +CPU= 23.432
  12:33:31 STEP  969 H= 242:15 +CPU= 23.092
  12:33:55 STEP  970 H= 242:30 +CPU= 23.066
  12:34:19 STEP  971 H= 242:45 +CPU= 22.900
  12:34:50 STEP  972 H= 243:00 +CPU= 28.942
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+000972
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+000972
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+000972
  12:35:14 STEP  973 H= 243:15 +CPU= 23.145
  12:35:39 STEP  974 H= 243:30 +CPU= 23.256
  12:36:03 STEP  975 H= 243:45 +CPU= 23.190
  12:36:28 STEP  976 H= 244:00 +CPU= 23.455
  12:36:52 STEP  977 H= 244:15 +CPU= 23.162
  12:37:17 STEP  978 H= 244:30 +CPU= 23.172
  12:37:41 STEP  979 H= 244:45 +CPU= 23.066
  12:38:06 STEP  980 H= 245:00 +CPU= 23.912
  12:38:30 STEP  981 H= 245:15 +CPU= 22.930
  12:38:55 STEP  982 H= 245:30 +CPU= 23.188
  12:39:19 STEP  983 H= 245:45 +CPU= 23.140
  12:39:50 STEP  984 H= 246:00 +CPU= 28.695
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+000984
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+000984
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+000984
  12:40:14 STEP  985 H= 246:15 +CPU= 22.830
  12:40:38 STEP  986 H= 246:30 +CPU= 22.926
  12:41:03 STEP  987 H= 246:45 +CPU= 23.318
  12:41:27 STEP  988 H= 247:00 +CPU= 23.491
  12:41:52 STEP  989 H= 247:15 +CPU= 23.338
  12:42:16 STEP  990 H= 247:30 +CPU= 23.010
  12:42:41 STEP  991 H= 247:45 +CPU= 23.238
  12:43:06 STEP  992 H= 248:00 +CPU= 23.516
  12:43:30 STEP  993 H= 248:15 +CPU= 23.333
  12:43:55 STEP  994 H= 248:30 +CPU= 23.152
  12:44:19 STEP  995 H= 248:45 +CPU= 22.971
  12:44:50 STEP  996 H= 249:00 +CPU= 28.922
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+000996
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+000996
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+000996
  12:45:14 STEP  997 H= 249:15 +CPU= 22.960
  12:45:38 STEP  998 H= 249:30 +CPU= 23.113
  12:46:03 STEP  999 H= 249:45 +CPU= 23.153
  12:46:28 STEP 1000 H= 250:00 +CPU= 23.619
  12:46:52 STEP 1001 H= 250:15 +CPU= 23.095
  12:47:16 STEP 1002 H= 250:30 +CPU= 23.108
  12:47:41 STEP 1003 H= 250:45 +CPU= 23.126
  12:48:06 STEP 1004 H= 251:00 +CPU= 23.642
  12:48:30 STEP 1005 H= 251:15 +CPU= 23.148
  12:48:54 STEP 1006 H= 251:30 +CPU= 23.004
  12:49:19 STEP 1007 H= 251:45 +CPU= 23.324
  12:49:52 STEP 1008 H= 252:00 +CPU= 31.909
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001008
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001008
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001008
  12:50:15 STEP 1009 H= 252:15 +CPU= 21.846
  12:50:43 STEP 1010 H= 252:30 +CPU= 23.926
  12:51:09 STEP 1011 H= 252:45 +CPU= 23.634
  12:51:35 STEP 1012 H= 253:00 +CPU= 24.109
  12:52:04 STEP 1013 H= 253:15 +CPU= 24.887
  12:52:33 STEP 1014 H= 253:30 +CPU= 24.658
  12:53:01 STEP 1015 H= 253:45 +CPU= 24.820
  12:53:29 STEP 1016 H= 254:00 +CPU= 24.964
  12:53:59 STEP 1017 H= 254:15 +CPU= 24.860
  12:54:28 STEP 1018 H= 254:30 +CPU= 24.795
  12:54:58 STEP 1019 H= 254:45 +CPU= 24.785
  12:55:35 STEP 1020 H= 255:00 +CPU= 31.321
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001020
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001020
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001020
  12:56:06 STEP 1021 H= 255:15 +CPU= 24.874
  12:56:36 STEP 1022 H= 255:30 +CPU= 24.800
  12:57:07 STEP 1023 H= 255:45 +CPU= 24.622
  12:57:37 STEP 1024 H= 256:00 +CPU= 25.146
  12:58:06 STEP 1025 H= 256:15 +CPU= 24.794
  12:58:34 STEP 1026 H= 256:30 +CPU= 24.997
  12:59:03 STEP 1027 H= 256:45 +CPU= 24.738
  12:59:33 STEP 1028 H= 257:00 +CPU= 25.303
  13:00:02 STEP 1029 H= 257:15 +CPU= 24.726
  13:00:31 STEP 1030 H= 257:30 +CPU= 25.001
  13:00:59 STEP 1031 H= 257:45 +CPU= 24.638
  13:01:37 STEP 1032 H= 258:00 +CPU= 30.813
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001032
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001032
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001032
  13:02:04 STEP 1033 H= 258:15 +CPU= 24.683
  13:02:32 STEP 1034 H= 258:30 +CPU= 24.478
  13:03:01 STEP 1035 H= 258:45 +CPU= 24.850
  13:03:30 STEP 1036 H= 259:00 +CPU= 25.086
  13:03:59 STEP 1037 H= 259:15 +CPU= 24.705
  13:04:26 STEP 1038 H= 259:30 +CPU= 24.159
  13:04:54 STEP 1039 H= 259:45 +CPU= 24.734
  13:05:21 STEP 1040 H= 260:00 +CPU= 25.209
  13:05:48 STEP 1041 H= 260:15 +CPU= 25.143
  13:06:15 STEP 1042 H= 260:30 +CPU= 24.980
  13:06:42 STEP 1043 H= 260:45 +CPU= 25.018
  13:07:16 STEP 1044 H= 261:00 +CPU= 31.141
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001044
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001044
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001044
  13:07:43 STEP 1045 H= 261:15 +CPU= 24.899
  13:08:10 STEP 1046 H= 261:30 +CPU= 24.885
  13:08:37 STEP 1047 H= 261:45 +CPU= 25.072
  13:09:05 STEP 1048 H= 262:00 +CPU= 25.709
  13:09:31 STEP 1049 H= 262:15 +CPU= 24.726
  13:09:58 STEP 1050 H= 262:30 +CPU= 24.773
  13:10:24 STEP 1051 H= 262:45 +CPU= 24.477
  13:10:52 STEP 1052 H= 263:00 +CPU= 25.143
  13:11:19 STEP 1053 H= 263:15 +CPU= 25.057
  13:11:46 STEP 1054 H= 263:30 +CPU= 24.838
  13:12:11 STEP 1055 H= 263:45 +CPU= 24.130
  13:12:49 STEP 1056 H= 264:00 +CPU= 34.714
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001056
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001056
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001044
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a2vf_2016092300_15_991_12210510_0_r1547825834_10.zip
Uploading the intermediate file: upload_file_10.zip
  13:13:16 STEP 1057 H= 264:15 +CPU= 24.476
  13:13:42 STEP 1058 H= 264:30 +CPU= 24.446
  13:14:09 STEP 1059 H= 264:45 +CPU= 24.548
  13:14:36 STEP 1060 H= 265:00 +CPU= 25.361
  13:15:04 STEP 1061 H= 265:15 +CPU= 24.910
  13:15:31 STEP 1062 H= 265:30 +CPU= 24.328
  13:15:57 STEP 1063 H= 265:45 +CPU= 24.814
  13:16:24 STEP 1064 H= 266:00 +CPU= 24.993
  13:16:50 STEP 1065 H= 266:15 +CPU= 24.520
  13:17:17 STEP 1066 H= 266:30 +CPU= 24.506
  13:17:42 STEP 1067 H= 266:45 +CPU= 24.243
  13:18:16 STEP 1068 H= 267:00 +CPU= 30.839
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001068
  13:18:42 STEP 1069 H= 267:15 +CPU= 24.395
  13:19:08 STEP 1070 H= 267:30 +CPU= 24.333
  13:19:34 STEP 1071 H= 267:45 +CPU= 24.463
  13:20:00 STEP 1072 H= 268:00 +CPU= 25.214
  13:20:27 STEP 1073 H= 268:15 +CPU= 24.840
  13:20:53 STEP 1074 H= 268:30 +CPU= 24.069
  13:21:20 STEP 1075 H= 268:45 +CPU= 24.420
  13:21:48 STEP 1076 H= 269:00 +CPU= 25.307
  13:22:14 STEP 1077 H= 269:15 +CPU= 24.460
  13:22:41 STEP 1078 H= 269:30 +CPU= 24.704
  13:23:07 STEP 1079 H= 269:45 +CPU= 24.241
  13:23:40 STEP 1080 H= 270:00 +CPU= 30.936
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001080
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001080
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001080
  13:24:06 STEP 1081 H= 270:15 +CPU= 24.689
  13:24:32 STEP 1082 H= 270:30 +CPU= 24.602
  13:24:59 STEP 1083 H= 270:45 +CPU= 24.901
  13:25:26 STEP 1084 H= 271:00 +CPU= 25.157
  13:25:52 STEP 1085 H= 271:15 +CPU= 24.199
  13:26:18 STEP 1086 H= 271:30 +CPU= 24.267
  13:26:44 STEP 1087 H= 271:45 +CPU= 24.401
  13:27:11 STEP 1088 H= 272:00 +CPU= 24.975
  13:27:37 STEP 1089 H= 272:15 +CPU= 24.452
  13:28:04 STEP 1090 H= 272:30 +CPU= 24.608
  13:28:29 STEP 1091 H= 272:45 +CPU= 24.085
  13:29:02 STEP 1092 H= 273:00 +CPU= 30.592
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001092
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001092
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001092
  13:29:28 STEP 1093 H= 273:15 +CPU= 24.371
  13:29:54 STEP 1094 H= 273:30 +CPU= 24.552
  13:30:21 STEP 1095 H= 273:45 +CPU= 24.443
  13:30:50 STEP 1096 H= 274:00 +CPU= 25.073
  13:31:19 STEP 1097 H= 274:15 +CPU= 24.318
  13:31:44 STEP 1098 H= 274:30 +CPU= 23.788
  13:32:10 STEP 1099 H= 274:45 +CPU= 24.217
  13:32:37 STEP 1100 H= 275:00 +CPU= 25.045
  13:33:03 STEP 1101 H= 275:15 +CPU= 24.363
  13:33:29 STEP 1102 H= 275:30 +CPU= 24.282
  13:33:55 STEP 1103 H= 275:45 +CPU= 24.248
  13:34:32 STEP 1104 H= 276:00 +CPU= 33.637
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001104
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001104
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001104
  13:34:58 STEP 1105 H= 276:15 +CPU= 24.587
  13:35:26 STEP 1106 H= 276:30 +CPU= 24.955
  13:35:53 STEP 1107 H= 276:45 +CPU= 24.659
  13:36:20 STEP 1108 H= 277:00 +CPU= 24.767
  13:36:48 STEP 1109 H= 277:15 +CPU= 24.539
  13:37:17 STEP 1110 H= 277:30 +CPU= 24.932
  13:37:44 STEP 1111 H= 277:45 +CPU= 24.816
  13:38:11 STEP 1112 H= 278:00 +CPU= 25.518
  13:38:39 STEP 1113 H= 278:15 +CPU= 25.138
  13:39:06 STEP 1114 H= 278:30 +CPU= 24.974
  13:39:32 STEP 1115 H= 278:45 +CPU= 24.514
  13:40:05 STEP 1116 H= 279:00 +CPU= 30.801
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001116
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001116
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001116
  13:40:32 STEP 1117 H= 279:15 +CPU= 24.539
  13:40:59 STEP 1118 H= 279:30 +CPU= 24.837
  13:41:25 STEP 1119 H= 279:45 +CPU= 24.559
  13:41:52 STEP 1120 H= 280:00 +CPU= 25.142
  13:42:20 STEP 1121 H= 280:15 +CPU= 25.192
  13:42:46 STEP 1122 H= 280:30 +CPU= 24.681
  13:43:12 STEP 1123 H= 280:45 +CPU= 24.676
  13:43:40 STEP 1124 H= 281:00 +CPU= 25.287
  13:44:07 STEP 1125 H= 281:15 +CPU= 24.918
  13:44:32 STEP 1126 H= 281:30 +CPU= 24.292
  13:44:59 STEP 1127 H= 281:45 +CPU= 24.506
  13:45:33 STEP 1128 H= 282:00 +CPU= 31.557
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001128
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001128
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001128
  13:46:00 STEP 1129 H= 282:15 +CPU= 24.782
  13:46:26 STEP 1130 H= 282:30 +CPU= 24.894
  13:46:53 STEP 1131 H= 282:45 +CPU= 24.941
  13:47:21 STEP 1132 H= 283:00 +CPU= 25.685
  13:47:47 STEP 1133 H= 283:15 +CPU= 24.609
  13:48:14 STEP 1134 H= 283:30 +CPU= 24.715
  13:48:40 STEP 1135 H= 283:45 +CPU= 24.879
  13:49:08 STEP 1136 H= 284:00 +CPU= 25.317
  13:49:34 STEP 1137 H= 284:15 +CPU= 24.474
  13:50:00 STEP 1138 H= 284:30 +CPU= 24.668
  13:50:27 STEP 1139 H= 284:45 +CPU= 24.973
  13:51:01 STEP 1140 H= 285:00 +CPU= 31.357
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001140
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001140
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001140
  13:51:28 STEP 1141 H= 285:15 +CPU= 25.250
  13:51:58 STEP 1142 H= 285:30 +CPU= 26.307
  13:52:27 STEP 1143 H= 285:45 +CPU= 27.092
  13:52:55 STEP 1144 H= 286:00 +CPU= 25.913
  13:53:21 STEP 1145 H= 286:15 +CPU= 24.754
  13:53:48 STEP 1146 H= 286:30 +CPU= 24.795
  13:54:14 STEP 1147 H= 286:45 +CPU= 24.369
  13:54:41 STEP 1148 H= 287:00 +CPU= 25.518
  13:55:09 STEP 1149 H= 287:15 +CPU= 25.938
  13:55:38 STEP 1150 H= 287:30 +CPU= 26.035
  13:56:06 STEP 1151 H= 287:45 +CPU= 24.629
  13:56:43 STEP 1152 H= 288:00 +CPU= 34.085
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001152
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001152
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001140
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a2vf_2016092300_15_991_12210510_0_r1547825834_11.zip
Uploading the intermediate file: upload_file_11.zip
  13:57:11 STEP 1153 H= 288:15 +CPU= 24.725
  13:57:40 STEP 1154 H= 288:30 +CPU= 26.091
  13:58:07 STEP 1155 H= 288:45 +CPU= 25.001
  13:58:34 STEP 1156 H= 289:00 +CPU= 25.239
  13:59:01 STEP 1157 H= 289:15 +CPU= 24.808
  13:59:27 STEP 1158 H= 289:30 +CPU= 25.155
  13:59:55 STEP 1159 H= 289:45 +CPU= 25.169
  14:00:24 STEP 1160 H= 290:00 +CPU= 26.694
  14:00:52 STEP 1161 H= 290:15 +CPU= 26.635
  14:01:24 STEP 1162 H= 290:30 +CPU= 29.482
  14:01:55 STEP 1163 H= 290:45 +CPU= 27.972
  14:02:33 STEP 1164 H= 291:00 +CPU= 36.033
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001164
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001164
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001164
  14:03:02 STEP 1165 H= 291:15 +CPU= 25.593
  14:03:29 STEP 1166 H= 291:30 +CPU= 24.627
  14:03:56 STEP 1167 H= 291:45 +CPU= 25.049
  14:04:23 STEP 1168 H= 292:00 +CPU= 25.088
  14:04:50 STEP 1169 H= 292:15 +CPU= 24.737
  14:05:17 STEP 1170 H= 292:30 +CPU= 24.729
  14:05:43 STEP 1171 H= 292:45 +CPU= 24.593
  14:06:12 STEP 1172 H= 293:00 +CPU= 25.609
  14:06:40 STEP 1173 H= 293:15 +CPU= 25.262
  14:07:09 STEP 1174 H= 293:30 +CPU= 26.599
  14:07:36 STEP 1175 H= 293:45 +CPU= 24.985
  14:08:09 STEP 1176 H= 294:00 +CPU= 30.859
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001176
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001176
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001176
  14:08:35 STEP 1177 H= 294:15 +CPU= 24.734
  14:09:01 STEP 1178 H= 294:30 +CPU= 24.561
  14:09:28 STEP 1179 H= 294:45 +CPU= 24.946
  14:09:55 STEP 1180 H= 295:00 +CPU= 25.253
  14:10:21 STEP 1181 H= 295:15 +CPU= 24.567
  14:10:51 STEP 1182 H= 295:30 +CPU= 27.838
  14:11:18 STEP 1183 H= 295:45 +CPU= 24.796
  14:11:48 STEP 1184 H= 296:00 +CPU= 28.478
  14:12:16 STEP 1185 H= 296:15 +CPU= 25.809
  14:12:44 STEP 1186 H= 296:30 +CPU= 26.201
  14:13:11 STEP 1187 H= 296:45 +CPU= 24.628
  14:13:47 STEP 1188 H= 297:00 +CPU= 33.223
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001188
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001188
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001188
  14:14:16 STEP 1189 H= 297:15 +CPU= 27.308
  14:14:43 STEP 1190 H= 297:30 +CPU= 24.825
  14:15:10 STEP 1191 H= 297:45 +CPU= 25.506
  14:15:37 STEP 1192 H= 298:00 +CPU= 25.071
  14:16:04 STEP 1193 H= 298:15 +CPU= 24.847
  14:16:32 STEP 1194 H= 298:30 +CPU= 26.104
  14:16:58 STEP 1195 H= 298:45 +CPU= 24.610
  14:17:26 STEP 1196 H= 299:00 +CPU= 26.069
  14:17:54 STEP 1197 H= 299:15 +CPU= 26.186
  14:18:21 STEP 1198 H= 299:30 +CPU= 25.166
  14:18:50 STEP 1199 H= 299:45 +CPU= 26.633
  14:19:29 STEP 1200 H= 300:00 +CPU= 36.045
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001200
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001200
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001200
  14:19:56 STEP 1201 H= 300:15 +CPU= 25.353
  14:20:23 STEP 1202 H= 300:30 +CPU= 25.533
  14:20:50 STEP 1203 H= 300:45 +CPU= 24.990
  14:21:17 STEP 1204 H= 301:00 +CPU= 25.085
  14:21:45 STEP 1205 H= 301:15 +CPU= 24.956
  14:22:13 STEP 1206 H= 301:30 +CPU= 24.823
  14:22:40 STEP 1207 H= 301:45 +CPU= 25.714
  14:23:09 STEP 1208 H= 302:00 +CPU= 25.690
  14:23:38 STEP 1209 H= 302:15 +CPU= 25.589
  14:24:05 STEP 1210 H= 302:30 +CPU= 24.834
  14:24:32 STEP 1211 H= 302:45 +CPU= 24.864
  14:25:09 STEP 1212 H= 303:00 +CPU= 33.691
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001212
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001212
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001212
  14:25:35 STEP 1213 H= 303:15 +CPU= 24.810
  14:26:03 STEP 1214 H= 303:30 +CPU= 25.535
  14:26:30 STEP 1215 H= 303:45 +CPU= 24.736
  14:26:57 STEP 1216 H= 304:00 +CPU= 25.096
  14:27:23 STEP 1217 H= 304:15 +CPU= 24.454
  14:27:50 STEP 1218 H= 304:30 +CPU= 25.439
  14:28:17 STEP 1219 H= 304:45 +CPU= 24.752
  14:28:44 STEP 1220 H= 305:00 +CPU= 25.167
  14:29:11 STEP 1221 H= 305:15 +CPU= 24.532
  14:29:39 STEP 1222 H= 305:30 +CPU= 25.148
  14:30:07 STEP 1223 H= 305:45 +CPU= 25.955
  14:30:43 STEP 1224 H= 306:00 +CPU= 31.886
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001224
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001224
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001224
  14:31:10 STEP 1225 H= 306:15 +CPU= 24.749
  14:31:37 STEP 1226 H= 306:30 +CPU= 25.366
  14:32:06 STEP 1227 H= 306:45 +CPU= 26.513
  14:32:32 STEP 1228 H= 307:00 +CPU= 24.920
  14:33:02 STEP 1229 H= 307:15 +CPU= 27.508
  14:33:31 STEP 1230 H= 307:30 +CPU= 26.936
  14:33:58 STEP 1231 H= 307:45 +CPU= 24.730
  14:34:26 STEP 1232 H= 308:00 +CPU= 27.139
  14:34:54 STEP 1233 H= 308:15 +CPU= 24.995
  14:35:20 STEP 1234 H= 308:30 +CPU= 24.759
  14:35:48 STEP 1235 H= 308:45 +CPU= 25.334
  14:36:20 STEP 1236 H= 309:00 +CPU= 30.456
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001236
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001236
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001236
  14:36:47 STEP 1237 H= 309:15 +CPU= 24.805
  14:37:13 STEP 1238 H= 309:30 +CPU= 24.310
  14:37:41 STEP 1239 H= 309:45 +CPU= 25.725
  14:38:09 STEP 1240 H= 310:00 +CPU= 26.621
  14:38:36 STEP 1241 H= 310:15 +CPU= 24.664
  14:39:02 STEP 1242 H= 310:30 +CPU= 24.609
  14:39:29 STEP 1243 H= 310:45 +CPU= 25.457
  14:39:56 STEP 1244 H= 311:00 +CPU= 25.134
  14:40:23 STEP 1245 H= 311:15 +CPU= 24.673
  14:40:51 STEP 1246 H= 311:30 +CPU= 26.219
  14:41:18 STEP 1247 H= 311:45 +CPU= 24.946
  14:41:55 STEP 1248 H= 312:00 +CPU= 34.858
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001248
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001248
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001236
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a2vf_2016092300_15_991_12210510_0_r1547825834_12.zip
Uploading the intermediate file: upload_file_12.zip
  14:42:24 STEP 1249 H= 312:15 +CPU= 26.483
  14:42:53 STEP 1250 H= 312:30 +CPU= 26.761
  14:43:21 STEP 1251 H= 312:45 +CPU= 25.111
  14:43:49 STEP 1252 H= 313:00 +CPU= 26.238
  14:44:17 STEP 1253 H= 313:15 +CPU= 25.944
  14:44:43 STEP 1254 H= 313:30 +CPU= 24.645
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a2vf
(argv4) batchid: 991
(argv5) wuid: 12210510
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/3
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.11
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/3/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/3/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_c09d8a06f0dd400752fa506157113dc8 to: /var/lib/boinc-client/slots/3/oifs_43r3_bl_a2vf_2016092300_15_991_12210510.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/3/oifs_43r3_bl_a2vf_2016092300_15_991_12210510.zip
ic_ancil_file: ic_ancil_12210510
ifsdata_file: ifsdata_12210510
climate_data_file: clim_data_12210510
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/3/ic_ancil_12210510.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/3/ic_ancil_12210510.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_c7645762ce4ef25e189255e8b4799687 to: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12210510.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12210510.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/3/159l_2/clim_data_12210510.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/3/159l_2/clim_data_12210510.zip
Checking for progress XML file: /var/lib/boinc-client/slots/3/progress_file_12210510.xml
Opened progress file ok : /var/lib/boinc-client/slots/3/progress_file_12210510.xml
-- Model is restarting --
Adjusting last_iter, 1254, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/3/progress_file_12210510.xml
last_cpu_time: 34963
upload_file_number: 13
last_iter: 1249
last_upload: 1123200
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a2vf_2016092300_15_991_12210510_0_r1547825834
The child process has been launched with process id: 5806
Executing the command: /var/lib/boinc-client/slots/3/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil))
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil))
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil))
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil))
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil))
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil))
[EC_DRHOOK:naturerick:1:1:5806:5806] [20230215:231522:1676531722:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  23:15:54 STEP 1248 H= 312:00 +CPU= 24.587
  23:16:18 STEP 1249 H= 312:15 +CPU= 23.626
  23:16:42 STEP 1250 H= 312:30 +CPU= 23.927
  23:17:06 STEP 1251 H= 312:45 +CPU= 23.745
  23:17:30 STEP 1252 H= 313:00 +CPU= 23.889
  23:17:53 STEP 1253 H= 313:15 +CPU= 23.368
  23:18:17 STEP 1254 H= 313:30 +CPU= 23.496
  23:18:41 STEP 1255 H= 313:45 +CPU= 23.500
  23:19:05 STEP 1256 H= 314:00 +CPU= 24.019
  23:19:28 STEP 1257 H= 314:15 +CPU= 23.280
  23:19:52 STEP 1258 H= 314:30 +CPU= 23.442
  23:20:16 STEP 1259 H= 314:45 +CPU= 23.607
  23:20:45 STEP 1260 H= 315:00 +CPU= 29.670
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001260
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001260
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001260
  23:21:09 STEP 1261 H= 315:15 +CPU= 23.914
  23:21:33 STEP 1262 H= 315:30 +CPU= 23.849
  23:21:57 STEP 1263 H= 315:45 +CPU= 23.610
  23:22:22 STEP 1264 H= 316:00 +CPU= 24.199
  23:22:45 STEP 1265 H= 316:15 +CPU= 23.533
  23:23:09 STEP 1266 H= 316:30 +CPU= 23.350
  23:23:33 STEP 1267 H= 316:45 +CPU= 23.643
  23:23:57 STEP 1268 H= 317:00 +CPU= 24.342
  23:24:21 STEP 1269 H= 317:15 +CPU= 23.817
  23:24:45 STEP 1270 H= 317:30 +CPU= 24.069
  23:25:09 STEP 1271 H= 317:45 +CPU= 23.471
  23:25:39 STEP 1272 H= 318:00 +CPU= 29.602
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001272
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001272
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001272
  23:26:02 STEP 1273 H= 318:15 +CPU= 23.440
  23:26:26 STEP 1274 H= 318:30 +CPU= 24.145
  23:26:50 STEP 1275 H= 318:45 +CPU= 23.618
  23:27:14 STEP 1276 H= 319:00 +CPU= 24.112
  23:27:39 STEP 1277 H= 319:15 +CPU= 24.015
  23:28:02 STEP 1278 H= 319:30 +CPU= 23.610
  23:28:26 STEP 1279 H= 319:45 +CPU= 23.618
  23:28:51 STEP 1280 H= 320:00 +CPU= 24.332
  23:29:14 STEP 1281 H= 320:15 +CPU= 23.737
  23:29:38 STEP 1282 H= 320:30 +CPU= 23.302
  23:30:02 STEP 1283 H= 320:45 +CPU= 23.755
  23:30:31 STEP 1284 H= 321:00 +CPU= 29.403
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001284
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001284
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001284
  23:30:55 STEP 1285 H= 321:15 +CPU= 23.472
  23:31:19 STEP 1286 H= 321:30 +CPU= 23.760
  23:31:42 STEP 1287 H= 321:45 +CPU= 23.349
  23:32:07 STEP 1288 H= 322:00 +CPU= 24.204
  23:32:30 STEP 1289 H= 322:15 +CPU= 23.493
  23:32:54 STEP 1290 H= 322:30 +CPU= 23.726
  23:33:18 STEP 1291 H= 322:45 +CPU= 23.912
  23:33:42 STEP 1292 H= 323:00 +CPU= 23.875
  23:34:06 STEP 1293 H= 323:15 +CPU= 23.937
  23:34:30 STEP 1294 H= 323:30 +CPU= 23.636
  23:34:54 STEP 1295 H= 323:45 +CPU= 23.246
  23:35:27 STEP 1296 H= 324:00 +CPU= 33.311
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001296
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001296
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001296
  23:35:49 STEP 1297 H= 324:15 +CPU= 22.072
  23:36:09 STEP 1298 H= 324:30 +CPU= 19.675
  23:36:29 STEP 1299 H= 324:45 +CPU= 19.952
  23:36:51 STEP 1300 H= 325:00 +CPU= 21.793
  23:37:11 STEP 1301 H= 325:15 +CPU= 20.200
  23:37:31 STEP 1302 H= 325:30 +CPU= 20.056
  23:37:51 STEP 1303 H= 325:45 +CPU= 19.921
  23:38:12 STEP 1304 H= 326:00 +CPU= 20.065
  23:38:32 STEP 1305 H= 326:15 +CPU= 20.701
  23:38:53 STEP 1306 H= 326:30 +CPU= 20.033
  23:39:12 STEP 1307 H= 326:45 +CPU= 19.771
  23:39:37 STEP 1308 H= 327:00 +CPU= 24.811
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001308
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001308
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001308
  23:39:57 STEP 1309 H= 327:15 +CPU= 19.983
  23:40:17 STEP 1310 H= 327:30 +CPU= 19.025
  23:40:36 STEP 1311 H= 327:45 +CPU= 19.527
  23:40:56 STEP 1312 H= 328:00 +CPU= 20.118
  23:41:17 STEP 1313 H= 328:15 +CPU= 20.248
  23:41:37 STEP 1314 H= 328:30 +CPU= 19.644
  23:41:57 STEP 1315 H= 328:45 +CPU= 20.048
  23:42:18 STEP 1316 H= 329:00 +CPU= 20.690
  23:42:38 STEP 1317 H= 329:15 +CPU= 19.927
  23:42:57 STEP 1318 H= 329:30 +CPU= 19.342
  23:43:17 STEP 1319 H= 329:45 +CPU= 19.762
  23:43:42 STEP 1320 H= 330:00 +CPU= 24.552
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001320
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001320
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001320
  23:44:01 STEP 1321 H= 330:15 +CPU= 19.777
  23:44:22 STEP 1322 H= 330:30 +CPU= 20.019
  23:44:42 STEP 1323 H= 330:45 +CPU= 20.426
  23:45:03 STEP 1324 H= 331:00 +CPU= 20.430
  23:45:23 STEP 1325 H= 331:15 +CPU= 20.117
  23:45:43 STEP 1326 H= 331:30 +CPU= 20.253
  23:46:03 STEP 1327 H= 331:45 +CPU= 19.339
  23:46:23 STEP 1328 H= 332:00 +CPU= 20.296
  23:46:44 STEP 1329 H= 332:15 +CPU= 20.468
  23:47:04 STEP 1330 H= 332:30 +CPU= 19.914
  23:47:23 STEP 1331 H= 332:45 +CPU= 19.397
  23:47:48 STEP 1332 H= 333:00 +CPU= 25.121
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001332
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001332
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001332
  23:48:08 STEP 1333 H= 333:15 +CPU= 19.322
  23:48:28 STEP 1334 H= 333:30 +CPU= 20.227
  23:48:48 STEP 1335 H= 333:45 +CPU= 20.105
  23:49:09 STEP 1336 H= 334:00 +CPU= 20.567
  23:49:29 STEP 1337 H= 334:15 +CPU= 19.933
  23:49:49 STEP 1338 H= 334:30 +CPU= 20.116
  23:50:09 STEP 1339 H= 334:45 +CPU= 19.419
  23:50:27 STEP 1340 H= 335:00 +CPU= 18.055
  23:50:45 STEP 1341 H= 335:15 +CPU= 17.781
  23:51:03 STEP 1342 H= 335:30 +CPU= 17.805
  23:51:20 STEP 1343 H= 335:45 +CPU= 17.199
  23:51:45 STEP 1344 H= 336:00 +CPU= 24.990
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001344
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001344
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001332
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a2vf_2016092300_15_991_12210510_0_r1547825834_13.zip
Uploading the intermediate file: upload_file_13.zip
  23:52:04 STEP 1345 H= 336:15 +CPU= 18.714
  23:52:21 STEP 1346 H= 336:30 +CPU= 17.186
  23:52:38 STEP 1347 H= 336:45 +CPU= 17.016
  23:52:56 STEP 1348 H= 337:00 +CPU= 17.939
  23:53:13 STEP 1349 H= 337:15 +CPU= 17.013
  23:53:30 STEP 1350 H= 337:30 +CPU= 16.849
  23:53:47 STEP 1351 H= 337:45 +CPU= 16.712
  23:54:04 STEP 1352 H= 338:00 +CPU= 16.569
  23:54:22 STEP 1353 H= 338:15 +CPU= 17.960
  23:54:39 STEP 1354 H= 338:30 +CPU= 17.173
  23:54:56 STEP 1355 H= 338:45 +CPU= 16.526
  23:55:18 STEP 1356 H= 339:00 +CPU= 22.212
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001356
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001356
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001356
  23:55:35 STEP 1357 H= 339:15 +CPU= 16.798
  23:55:52 STEP 1358 H= 339:30 +CPU= 16.653
  23:56:08 STEP 1359 H= 339:45 +CPU= 16.514
  23:56:24 STEP 1360 H= 340:00 +CPU= 16.089
  23:56:41 STEP 1361 H= 340:15 +CPU= 17.011
  23:56:58 STEP 1362 H= 340:30 +CPU= 16.586
  23:57:15 STEP 1363 H= 340:45 +CPU= 16.757
  23:57:32 STEP 1364 H= 341:00 +CPU= 16.909
  23:57:48 STEP 1365 H= 341:15 +CPU= 16.309
  23:58:05 STEP 1366 H= 341:30 +CPU= 16.193
  23:58:21 STEP 1367 H= 341:45 +CPU= 15.942
  23:58:42 STEP 1368 H= 342:00 +CPU= 21.660
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001368
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001368
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001368
  23:58:59 STEP 1369 H= 342:15 +CPU= 16.915
  23:59:17 STEP 1370 H= 342:30 +CPU= 17.589
  23:59:34 STEP 1371 H= 342:45 +CPU= 16.686
  23:59:52 STEP 1372 H= 343:00 +CPU= 18.338
  00:00:09 STEP 1373 H= 343:15 +CPU= 16.925
  00:00:25 STEP 1374 H= 343:30 +CPU= 15.851
  00:00:42 STEP 1375 H= 343:45 +CPU= 17.257
  00:01:00 STEP 1376 H= 344:00 +CPU= 17.741
  00:01:17 STEP 1377 H= 344:15 +CPU= 16.618
  00:01:33 STEP 1378 H= 344:30 +CPU= 16.316
  00:01:49 STEP 1379 H= 344:45 +CPU= 16.064
  00:02:11 STEP 1380 H= 345:00 +CPU= 20.972
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001380
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001380
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001380
  00:02:27 STEP 1381 H= 345:15 +CPU= 16.483
  00:02:43 STEP 1382 H= 345:30 +CPU= 15.947
  00:03:00 STEP 1383 H= 345:45 +CPU= 17.119
  00:03:18 STEP 1384 H= 346:00 +CPU= 17.570
  00:03:35 STEP 1385 H= 346:15 +CPU= 16.592
  00:03:52 STEP 1386 H= 346:30 +CPU= 17.651
  00:04:10 STEP 1387 H= 346:45 +CPU= 17.627
  00:04:27 STEP 1388 H= 347:00 +CPU= 16.431
  00:04:43 STEP 1389 H= 347:15 +CPU= 16.219
  00:04:59 STEP 1390 H= 347:30 +CPU= 16.283
  00:05:15 STEP 1391 H= 347:45 +CPU= 16.037
  00:05:39 STEP 1392 H= 348:00 +CPU= 23.929
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001392
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001392
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001392
  00:05:56 STEP 1393 H= 348:15 +CPU= 16.460
  00:06:13 STEP 1394 H= 348:30 +CPU= 16.593
  00:06:29 STEP 1395 H= 348:45 +CPU= 16.098
  00:06:46 STEP 1396 H= 349:00 +CPU= 17.537
  00:07:04 STEP 1397 H= 349:15 +CPU= 17.079
  00:07:20 STEP 1398 H= 349:30 +CPU= 16.620
  00:07:37 STEP 1399 H= 349:45 +CPU= 16.802
  00:07:54 STEP 1400 H= 350:00 +CPU= 16.959
  00:08:11 STEP 1401 H= 350:15 +CPU= 16.676
  00:08:28 STEP 1402 H= 350:30 +CPU= 16.906
  00:08:45 STEP 1403 H= 350:45 +CPU= 17.213
  00:09:06 STEP 1404 H= 351:00 +CPU= 20.881
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001404
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001404
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001404
  00:09:23 STEP 1405 H= 351:15 +CPU= 16.846
  00:09:40 STEP 1406 H= 351:30 +CPU= 17.128
  00:09:57 STEP 1407 H= 351:45 +CPU= 16.686
  00:10:16 STEP 1408 H= 352:00 +CPU= 18.613
  00:10:34 STEP 1409 H= 352:15 +CPU= 17.826
  00:10:51 STEP 1410 H= 352:30 +CPU= 17.729
  00:11:08 STEP 1411 H= 352:45 +CPU= 16.591
  00:11:25 STEP 1412 H= 353:00 +CPU= 16.430
  00:11:42 STEP 1413 H= 353:15 +CPU= 17.243
  00:11:58 STEP 1414 H= 353:30 +CPU= 16.397
  00:12:16 STEP 1415 H= 353:45 +CPU= 17.132
  00:12:37 STEP 1416 H= 354:00 +CPU= 21.427
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001416
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001416
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001416
  00:12:54 STEP 1417 H= 354:15 +CPU= 16.654
  00:13:10 STEP 1418 H= 354:30 +CPU= 16.513
  00:13:28 STEP 1419 H= 354:45 +CPU= 17.431
  00:13:46 STEP 1420 H= 355:00 +CPU= 17.487
  00:14:02 STEP 1421 H= 355:15 +CPU= 16.621
  00:14:19 STEP 1422 H= 355:30 +CPU= 16.370
  00:14:36 STEP 1423 H= 355:45 +CPU= 16.885
  00:14:54 STEP 1424 H= 356:00 +CPU= 18.174
  00:15:11 STEP 1425 H= 356:15 +CPU= 17.304
  00:15:29 STEP 1426 H= 356:30 +CPU= 17.160
  00:15:46 STEP 1427 H= 356:45 +CPU= 17.009
  00:16:06 STEP 1428 H= 357:00 +CPU= 20.707
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001428
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001428
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001428
  00:16:23 STEP 1429 H= 357:15 +CPU= 16.417
  00:16:40 STEP 1430 H= 357:30 +CPU= 17.374
  00:16:57 STEP 1431 H= 357:45 +CPU= 16.699
  00:17:15 STEP 1432 H= 358:00 +CPU= 17.436
  00:17:33 STEP 1433 H= 358:15 +CPU= 18.322
  00:17:50 STEP 1434 H= 358:30 +CPU= 16.872
  00:18:06 STEP 1435 H= 358:45 +CPU= 16.330
  00:18:23 STEP 1436 H= 359:00 +CPU= 16.358
  00:18:40 STEP 1437 H= 359:15 +CPU= 17.069
  00:18:58 STEP 1438 H= 359:30 +CPU= 17.959
  00:19:15 STEP 1439 H= 359:45 +CPU= 16.602
  00:19:39 STEP 1440 H= 360:00 +CPU= 23.779
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  360.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.1737240410E-21      I       0.7384509706E-06      I
ICloud fractionI      0.0000000000E+00      I       0.4268719213E-02      I
I  Relat. Hum. I      0.4332067525E+00      I       0.4423482210E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.6850924443E+02      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1440 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001440 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001440
 MPI-TASK:   1 -    311637697 BYTES IN      7 RECORDS TRANSFERRED IN    0.0004 SECONDS779094.3 Mbytes/s, TOTAL TIME=    0.0061( 6.6%)
 00:19:39 STEP 1440 H= 360:00 +CPU= 23.779
 END CNT3
 NSTEP =  1440 CNT0   000000000  16.416  16.416  16.531   0.993               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           34/  1069125708 WRITTEN         5672/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    103    0       34      1044068     5672 ************    13.4     5.4     8.0     6.2    15.8
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 3856609.2 3856609.2    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 3849318.7 3849318.7     99.81      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  209    3408.7    3408.7     18.47      0.00
   9 SPCM     - SPECTRAL COMP.                     192     579.6     579.6      2.89      0.00
  10 SCAN2M - PHYSICS                              193    8618.7    8618.7     43.13      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 16     717.1     717.1      0.30      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                  9      25.5      25.5      0.01      0.00
  17 GRIDFPOS IN CNT4                               16      16.9      16.9      0.01      0.00
  27 RESTART FILES - WRITING                         4    2868.3    2868.3      0.30      0.00
  28 RESTART FILES - READING                         1    6756.7    6756.7      0.18      0.00
  29 SU4FPOS IN CNT4                                16       0.0       0.0      0.00      0.00
  30 DYNFPOS IN CNT4                                16    4460.2    4460.2      1.85      0.00
  31 POSDDH IN STEPO                                 1       0.6       0.6      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    193    4499.1    4499.1     22.52      0.00
  39 SU0YOMB                                         1     342.9     342.9      0.01      0.00
  51 SCAN2M   - SL COMM. PART 1                    193      20.8      20.8      0.10      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              192     320.1     320.1      1.59      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              192      65.2      65.2      0.32      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                  9       0.2       0.2      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      420     328.3     328.3      3.57      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       400     194.3     194.3      2.02      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        400      72.1      72.1      0.75      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       420     104.2     104.2      1.14      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      39.9      39.9      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            420      55.1      55.1      0.60      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            400      27.0      27.0      0.28      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            420      32.5      32.5      0.35      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            400      16.5      16.5      0.17      0.00
 400 GSTATS                                      23227       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 21563       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME     3856.6 CPU TIME    3832.9 VECTOR TIME     3832.9


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      336.7
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001440
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001440
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client/slots/3/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/3/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMUAh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMGGh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12210510/ICMSHh7zg+001356
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a2vf_2016092300_15_991_12210510_0_r1547825834_14.zip
Uploading the final file: upload_file_14.zip
Uploading trickle at timestep: 1295100
00:23:08 (5789): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:51:52 1506016 22310727 oifs_43r3_bl_a2vf_2016092300_15_991_12210510_0 1,295,100 38,795 0.0300


©2024 climateprediction.net