Name | oifs_43r3_bl_a4sz_2016092300_20_1018_12294554_1 |
Workunit | 12294554 |
Created | 15 Jun 2024, 12:35:51 UTC |
Sent | 15 Jun 2024, 12:36:00 UTC |
Report deadline | 14 Aug 2024, 12:36:00 UTC |
Received | 16 Jun 2024, 0:12:42 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 1 (0x00000001) Unknown error code |
Computer ID | 1551095 |
Run time | 9 hours 13 min 31 sec |
CPU time | 9 hours 9 min 9 sec |
Validate state | Invalid |
Credit | 1,318.46 |
Device peak FLOPS | 1.00 GFLOPS |
Application version | OpenIFS 43r3 Baroclinic Lifecycle v1.13 x86_64-pc-linux-gnu |
Peak working set size | 5,025.35 MB |
Peak swap size | 5,981.07 MB |
Peak disk usage | 1,362.20 MB |
Stderr | <core_client_version>8.0.2</core_client_version> <![CDATA[ <message> process exited with code 1 (0x1, -255)</message> <stderr_txt> 00732 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+000732 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+000744 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+000744 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+000744 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+000756 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+000756 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+000756 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4sz_2016092300_20_1018_12294554_1_r1423833186_7.zip Uploading the intermediate file: upload_file_7.zip 19:32:17 STEP 769 H= 192:15 +CPU= 31.138 19:32:46 STEP 770 H= 192:30 +CPU= 28.176 19:33:14 STEP 771 H= 192:45 +CPU= 28.225 19:33:43 STEP 772 H= 193:00 +CPU= 28.461 19:34:12 STEP 773 H= 193:15 +CPU= 29.066 19:34:40 STEP 774 H= 193:30 +CPU= 27.828 19:35:08 STEP 775 H= 193:45 +CPU= 27.849 19:35:38 STEP 776 H= 194:00 +CPU= 29.701 19:36:06 STEP 777 H= 194:15 +CPU= 28.133 19:36:34 STEP 778 H= 194:30 +CPU= 27.890 19:37:02 STEP 779 H= 194:45 +CPU= 27.883 19:37:38 STEP 780 H= 195:00 +CPU= 35.884 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000780 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000780 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000780 19:38:06 STEP 781 H= 195:15 +CPU= 28.232 19:38:34 STEP 782 H= 195:30 +CPU= 27.734 19:39:02 STEP 783 H= 195:45 +CPU= 28.069 19:39:32 STEP 784 H= 196:00 +CPU= 29.408 19:40:00 STEP 785 H= 196:15 +CPU= 27.692 19:40:29 STEP 786 H= 196:30 +CPU= 29.530 Suspend request received from the BOINC client, suspending the child process Resuming the child process 19:41:18 STEP 787 H= 196:45 +CPU= 27.186 19:41:48 STEP 788 H= 197:00 +CPU= 29.533 19:42:16 STEP 789 H= 197:15 +CPU= 28.073 19:42:44 STEP 790 H= 197:30 +CPU= 28.399 19:43:13 STEP 791 H= 197:45 +CPU= 28.388 19:43:49 STEP 792 H= 198:00 +CPU= 35.679 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000792 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000792 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000792 19:44:18 STEP 793 H= 198:15 +CPU= 28.830 19:44:46 STEP 794 H= 198:30 +CPU= 28.052 19:45:15 STEP 795 H= 198:45 +CPU= 28.922 19:45:44 STEP 796 H= 199:00 +CPU= 28.558 19:46:12 STEP 797 H= 199:15 +CPU= 28.239 19:46:41 STEP 798 H= 199:30 +CPU= 28.938 19:47:10 STEP 799 H= 199:45 +CPU= 28.420 19:47:40 STEP 800 H= 200:00 +CPU= 29.995 19:48:10 STEP 801 H= 200:15 +CPU= 29.548 19:48:39 STEP 802 H= 200:30 +CPU= 28.648 19:49:07 STEP 803 H= 200:45 +CPU= 28.535 19:49:45 STEP 804 H= 201:00 +CPU= 37.621 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000804 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000804 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000804 19:50:16 STEP 805 H= 201:15 +CPU= 30.244 19:50:45 STEP 806 H= 201:30 +CPU= 29.072 19:51:14 STEP 807 H= 201:45 +CPU= 28.958 19:51:44 STEP 808 H= 202:00 +CPU= 29.797 19:52:12 STEP 809 H= 202:15 +CPU= 27.723 19:52:41 STEP 810 H= 202:30 +CPU= 28.886 19:53:10 STEP 811 H= 202:45 +CPU= 29.045 19:53:42 STEP 812 H= 203:00 +CPU= 30.923 19:54:11 STEP 813 H= 203:15 +CPU= 29.572 19:54:40 STEP 814 H= 203:30 +CPU= 28.735 19:55:10 STEP 815 H= 203:45 +CPU= 29.738 19:55:51 STEP 816 H= 204:00 +CPU= 41.190 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000816 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000816 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000816 19:56:21 STEP 817 H= 204:15 +CPU= 29.645 19:56:51 STEP 818 H= 204:30 +CPU= 30.039 19:57:19 STEP 819 H= 204:45 +CPU= 28.004 19:57:50 STEP 820 H= 205:00 +CPU= 30.156 19:58:18 STEP 821 H= 205:15 +CPU= 28.462 19:58:47 STEP 822 H= 205:30 +CPU= 28.588 19:59:17 STEP 823 H= 205:45 +CPU= 30.095 19:59:48 STEP 824 H= 206:00 +CPU= 30.599 20:00:18 STEP 825 H= 206:15 +CPU= 29.605 20:00:47 STEP 826 H= 206:30 +CPU= 28.855 20:01:16 STEP 827 H= 206:45 +CPU= 28.353 20:01:53 STEP 828 H= 207:00 +CPU= 37.056 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000828 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000828 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000828 20:02:23 STEP 829 H= 207:15 +CPU= 29.409 20:02:51 STEP 830 H= 207:30 +CPU= 28.003 20:03:21 STEP 831 H= 207:45 +CPU= 30.249 20:03:52 STEP 832 H= 208:00 +CPU= 30.338 20:04:21 STEP 833 H= 208:15 +CPU= 28.888 20:04:51 STEP 834 H= 208:30 +CPU= 29.890 20:05:21 STEP 835 H= 208:45 +CPU= 29.401 Suspend request received from the BOINC client, suspending the child process Resuming the child process 20:06:12 STEP 836 H= 209:00 +CPU= 29.515 20:06:40 STEP 837 H= 209:15 +CPU= 28.523 Suspend request received from the BOINC client, suspending the child process Resuming the child process 20:07:30 STEP 838 H= 209:30 +CPU= 28.686 20:08:00 STEP 839 H= 209:45 +CPU= 29.455 20:08:38 STEP 840 H= 210:00 +CPU= 37.980 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000840 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000840 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000840 20:09:09 STEP 841 H= 210:15 +CPU= 30.341 Suspend request received from the BOINC client, suspending the child process Resuming the child process 20:09:59 STEP 842 H= 210:30 +CPU= 28.810 20:10:29 STEP 843 H= 210:45 +CPU= 30.144 20:11:00 STEP 844 H= 211:00 +CPU= 30.589 20:11:28 STEP 845 H= 211:15 +CPU= 28.762 20:11:58 STEP 846 H= 211:30 +CPU= 29.440 20:12:28 STEP 847 H= 211:45 +CPU= 29.457 20:12:58 STEP 848 H= 212:00 +CPU= 29.709 20:13:28 STEP 849 H= 212:15 +CPU= 30.027 20:13:57 STEP 850 H= 212:30 +CPU= 28.910 20:14:26 STEP 851 H= 212:45 +CPU= 28.861 20:15:03 STEP 852 H= 213:00 +CPU= 37.027 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000852 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000852 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000852 20:15:33 STEP 853 H= 213:15 +CPU= 29.656 20:16:04 STEP 854 H= 213:30 +CPU= 30.176 20:16:33 STEP 855 H= 213:45 +CPU= 29.279 20:17:04 STEP 856 H= 214:00 +CPU= 30.546 20:17:33 STEP 857 H= 214:15 +CPU= 29.398 20:18:03 STEP 858 H= 214:30 +CPU= 29.453 20:18:32 STEP 859 H= 214:45 +CPU= 28.853 20:19:02 STEP 860 H= 215:00 +CPU= 29.490 20:19:32 STEP 861 H= 215:15 +CPU= 29.898 20:20:00 STEP 862 H= 215:30 +CPU= 28.104 20:20:29 STEP 863 H= 215:45 +CPU= 29.264 20:21:12 STEP 864 H= 216:00 +CPU= 42.105 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000864 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000864 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000864 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+000768 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+000768 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+000768 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+000780 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+000780 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+000780 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+000792 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+000792 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+000792 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+000804 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+000804 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+000804 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+000816 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+000816 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+000816 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+000828 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+000828 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+000828 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+000840 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+000840 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+000840 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+000852 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+000852 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+000852 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4sz_2016092300_20_1018_12294554_1_r1423833186_8.zip Uploading the intermediate file: upload_file_8.zip 20:21:43 STEP 865 H= 216:15 +CPU= 30.943 20:22:13 STEP 866 H= 216:30 +CPU= 29.738 20:22:43 STEP 867 H= 216:45 +CPU= 29.523 20:23:13 STEP 868 H= 217:00 +CPU= 30.247 20:23:45 STEP 869 H= 217:15 +CPU= 30.903 20:24:15 STEP 870 H= 217:30 +CPU= 30.083 20:24:45 STEP 871 H= 217:45 +CPU= 29.732 20:25:16 STEP 872 H= 218:00 +CPU= 31.084 20:25:45 STEP 873 H= 218:15 +CPU= 28.929 20:26:16 STEP 874 H= 218:30 +CPU= 30.572 20:26:45 STEP 875 H= 218:45 +CPU= 28.953 Suspend request received from the BOINC client, suspending the child process Resuming the child process 20:27:42 STEP 876 H= 219:00 +CPU= 36.168 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000876 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000876 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000876 20:28:12 STEP 877 H= 219:15 +CPU= 29.703 20:28:42 STEP 878 H= 219:30 +CPU= 29.361 20:29:13 STEP 879 H= 219:45 +CPU= 30.674 20:29:43 STEP 880 H= 220:00 +CPU= 29.922 20:30:13 STEP 881 H= 220:15 +CPU= 30.202 20:30:41 STEP 882 H= 220:30 +CPU= 28.180 20:31:11 STEP 883 H= 220:45 +CPU= 29.593 20:31:42 STEP 884 H= 221:00 +CPU= 30.658 20:32:12 STEP 885 H= 221:15 +CPU= 30.563 20:32:41 STEP 886 H= 221:30 +CPU= 28.902 20:33:09 STEP 887 H= 221:45 +CPU= 28.078 20:33:47 STEP 888 H= 222:00 +CPU= 37.461 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000888 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000888 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000888 20:34:18 STEP 889 H= 222:15 +CPU= 30.180 20:34:46 STEP 890 H= 222:30 +CPU= 28.460 20:35:16 STEP 891 H= 222:45 +CPU= 29.521 Suspend request received from the BOINC client, suspending the child process Resuming the child process 20:36:07 STEP 892 H= 223:00 +CPU= 29.539 20:36:35 STEP 893 H= 223:15 +CPU= 28.653 20:37:05 STEP 894 H= 223:30 +CPU= 28.918 20:37:34 STEP 895 H= 223:45 +CPU= 28.959 20:38:04 STEP 896 H= 224:00 +CPU= 30.423 20:38:34 STEP 897 H= 224:15 +CPU= 29.671 20:39:04 STEP 898 H= 224:30 +CPU= 29.346 20:39:32 STEP 899 H= 224:45 +CPU= 28.473 20:40:10 STEP 900 H= 225:00 +CPU= 37.310 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000900 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000900 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000900 20:40:40 STEP 901 H= 225:15 +CPU= 30.060 20:41:10 STEP 902 H= 225:30 +CPU= 29.479 20:41:38 STEP 903 H= 225:45 +CPU= 28.231 20:42:09 STEP 904 H= 226:00 +CPU= 30.588 20:42:37 STEP 905 H= 226:15 +CPU= 28.260 20:43:07 STEP 906 H= 226:30 +CPU= 29.337 20:43:36 STEP 907 H= 226:45 +CPU= 28.970 20:44:06 STEP 908 H= 227:00 +CPU= 30.008 20:44:35 STEP 909 H= 227:15 +CPU= 28.987 Suspend request received from the BOINC client, suspending the child process Resuming the child process 20:45:26 STEP 910 H= 227:30 +CPU= 28.908 20:45:55 STEP 911 H= 227:45 +CPU= 28.855 20:46:36 STEP 912 H= 228:00 +CPU= 40.724 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000912 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000912 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000912 20:47:05 STEP 913 H= 228:15 +CPU= 29.496 20:47:34 STEP 914 H= 228:30 +CPU= 28.904 20:48:03 STEP 915 H= 228:45 +CPU= 28.471 20:48:33 STEP 916 H= 229:00 +CPU= 30.118 20:49:02 STEP 917 H= 229:15 +CPU= 28.755 20:49:30 STEP 918 H= 229:30 +CPU= 28.079 20:50:00 STEP 919 H= 229:45 +CPU= 29.520 20:50:30 STEP 920 H= 230:00 +CPU= 30.246 20:51:00 STEP 921 H= 230:15 +CPU= 28.932 20:51:29 STEP 922 H= 230:30 +CPU= 29.247 20:51:59 STEP 923 H= 230:45 +CPU= 29.364 20:52:35 STEP 924 H= 231:00 +CPU= 36.437 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000924 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000924 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000924 20:53:04 STEP 925 H= 231:15 +CPU= 28.546 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 21:13:47 STEP 926 H= 231:30 +CPU= 29.493 21:14:16 STEP 927 H= 231:45 +CPU= 28.903 21:14:46 STEP 928 H= 232:00 +CPU= 29.307 21:15:16 STEP 929 H= 232:15 +CPU= 29.930 21:15:44 STEP 930 H= 232:30 +CPU= 28.030 21:16:14 STEP 931 H= 232:45 +CPU= 29.165 21:16:45 STEP 932 H= 233:00 +CPU= 30.717 21:17:16 STEP 933 H= 233:15 +CPU= 31.230 21:17:45 STEP 934 H= 233:30 +CPU= 28.600 21:18:14 STEP 935 H= 233:45 +CPU= 28.569 21:18:51 STEP 936 H= 234:00 +CPU= 37.461 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000936 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000936 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000936 21:19:20 STEP 937 H= 234:15 +CPU= 28.547 21:19:49 STEP 938 H= 234:30 +CPU= 28.492 21:20:18 STEP 939 H= 234:45 +CPU= 28.761 21:20:48 STEP 940 H= 235:00 +CPU= 29.896 21:21:18 STEP 941 H= 235:15 +CPU= 29.348 21:21:46 STEP 942 H= 235:30 +CPU= 27.901 21:22:15 STEP 943 H= 235:45 +CPU= 28.809 21:22:45 STEP 944 H= 236:00 +CPU= 29.332 21:23:14 STEP 945 H= 236:15 +CPU= 29.171 21:23:43 STEP 946 H= 236:30 +CPU= 29.281 21:24:13 STEP 947 H= 236:45 +CPU= 29.813 21:24:50 STEP 948 H= 237:00 +CPU= 36.512 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000948 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000948 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000948 21:25:19 STEP 949 H= 237:15 +CPU= 28.648 21:25:48 STEP 950 H= 237:30 +CPU= 29.042 21:26:19 STEP 951 H= 237:45 +CPU= 30.587 21:26:49 STEP 952 H= 238:00 +CPU= 29.533 21:27:19 STEP 953 H= 238:15 +CPU= 29.355 Suspend request received from the BOINC client, suspending the child process Resuming the child process 21:28:08 STEP 954 H= 238:30 +CPU= 28.192 21:28:37 STEP 955 H= 238:45 +CPU= 28.791 21:29:07 STEP 956 H= 239:00 +CPU= 29.808 21:29:36 STEP 957 H= 239:15 +CPU= 28.720 21:30:05 STEP 958 H= 239:30 +CPU= 29.373 21:30:35 STEP 959 H= 239:45 +CPU= 29.448 21:31:16 STEP 960 H= 240:00 +CPU= 41.169 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000960 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000960 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000960 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+000864 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+000864 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+000864 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+000876 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+000876 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+000876 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+000888 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+000888 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+000888 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+000900 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+000900 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+000900 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+000912 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+000912 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+000912 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+000924 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+000924 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+000924 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+000936 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+000936 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+000936 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+000948 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+000948 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+000948 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4sz_2016092300_20_1018_12294554_1_r1423833186_9.zip Uploading the intermediate file: upload_file_9.zip 21:31:48 STEP 961 H= 240:15 +CPU= 30.956 Uploading trickle at timestep: 864000 21:32:18 STEP 962 H= 240:30 +CPU= 30.097 21:32:47 STEP 963 H= 240:45 +CPU= 28.663 21:33:17 STEP 964 H= 241:00 +CPU= 30.064 21:33:47 STEP 965 H= 241:15 +CPU= 28.964 21:34:16 STEP 966 H= 241:30 +CPU= 28.933 21:34:45 STEP 967 H= 241:45 +CPU= 28.343 21:35:14 STEP 968 H= 242:00 +CPU= 29.331 21:35:44 STEP 969 H= 242:15 +CPU= 29.800 21:36:13 STEP 970 H= 242:30 +CPU= 29.006 21:36:42 STEP 971 H= 242:45 +CPU= 28.563 21:37:19 STEP 972 H= 243:00 +CPU= 37.310 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000972 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000972 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000972 21:37:50 STEP 973 H= 243:15 +CPU= 30.069 21:38:19 STEP 974 H= 243:30 +CPU= 28.838 21:38:47 STEP 975 H= 243:45 +CPU= 28.129 21:39:17 STEP 976 H= 244:00 +CPU= 29.520 Suspend request received from the BOINC client, suspending the child process Resuming the child process 21:40:06 STEP 977 H= 244:15 +CPU= 28.030 21:40:36 STEP 978 H= 244:30 +CPU= 29.457 21:41:04 STEP 979 H= 244:45 +CPU= 28.582 21:41:33 STEP 980 H= 245:00 +CPU= 28.920 21:42:02 STEP 981 H= 245:15 +CPU= 28.523 21:42:31 STEP 982 H= 245:30 +CPU= 28.817 21:43:01 STEP 983 H= 245:45 +CPU= 29.378 21:43:38 STEP 984 H= 246:00 +CPU= 36.440 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000984 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000984 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000984 21:44:07 STEP 985 H= 246:15 +CPU= 29.053 21:44:37 STEP 986 H= 246:30 +CPU= 29.383 21:45:06 STEP 987 H= 246:45 +CPU= 28.842 21:45:36 STEP 988 H= 247:00 +CPU= 29.143 21:46:05 STEP 989 H= 247:15 +CPU= 28.843 21:46:34 STEP 990 H= 247:30 +CPU= 28.584 21:47:03 STEP 991 H= 247:45 +CPU= 28.000 21:47:32 STEP 992 H= 248:00 +CPU= 29.473 21:48:01 STEP 993 H= 248:15 +CPU= 28.286 21:48:29 STEP 994 H= 248:30 +CPU= 28.017 21:48:58 STEP 995 H= 248:45 +CPU= 28.825 21:49:36 STEP 996 H= 249:00 +CPU= 36.906 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000996 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000996 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000996 21:50:04 STEP 997 H= 249:15 +CPU= 28.295 21:50:34 STEP 998 H= 249:30 +CPU= 29.283 21:51:03 STEP 999 H= 249:45 +CPU= 29.148 Suspend request received from the BOINC client, suspending the child process Resuming the child process 21:51:54 STEP 1000 H= 250:00 +CPU= 29.213 21:52:23 STEP 1001 H= 250:15 +CPU= 29.161 21:52:52 STEP 1002 H= 250:30 +CPU= 29.122 21:53:22 STEP 1003 H= 250:45 +CPU= 29.329 21:53:53 STEP 1004 H= 251:00 +CPU= 30.226 21:54:23 STEP 1005 H= 251:15 +CPU= 30.434 21:54:53 STEP 1006 H= 251:30 +CPU= 29.705 21:55:24 STEP 1007 H= 251:45 +CPU= 30.324 21:56:06 STEP 1008 H= 252:00 +CPU= 41.589 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001008 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001008 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001008 21:56:36 STEP 1009 H= 252:15 +CPU= 29.770 21:57:04 STEP 1010 H= 252:30 +CPU= 27.979 21:57:33 STEP 1011 H= 252:45 +CPU= 29.156 21:58:03 STEP 1012 H= 253:00 +CPU= 29.527 21:58:32 STEP 1013 H= 253:15 +CPU= 29.058 21:59:02 STEP 1014 H= 253:30 +CPU= 29.410 21:59:31 STEP 1015 H= 253:45 +CPU= 28.334 22:00:03 STEP 1016 H= 254:00 +CPU= 32.015 22:00:37 STEP 1017 H= 254:15 +CPU= 33.413 22:01:07 STEP 1018 H= 254:30 +CPU= 29.644 22:01:38 STEP 1019 H= 254:45 +CPU= 29.754 22:02:15 STEP 1020 H= 255:00 +CPU= 36.956 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001020 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001020 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001020 22:02:44 STEP 1021 H= 255:15 +CPU= 28.419 22:03:12 STEP 1022 H= 255:30 +CPU= 28.260 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:04:03 STEP 1023 H= 255:45 +CPU= 29.280 22:04:34 STEP 1024 H= 256:00 +CPU= 31.110 22:05:04 STEP 1025 H= 256:15 +CPU= 30.103 22:05:35 STEP 1026 H= 256:30 +CPU= 31.098 22:06:06 STEP 1027 H= 256:45 +CPU= 30.481 22:06:37 STEP 1028 H= 257:00 +CPU= 30.746 22:07:08 STEP 1029 H= 257:15 +CPU= 31.025 22:07:40 STEP 1030 H= 257:30 +CPU= 31.022 22:08:11 STEP 1031 H= 257:45 +CPU= 31.643 22:08:52 STEP 1032 H= 258:00 +CPU= 39.515 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001032 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001032 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001032 22:09:22 STEP 1033 H= 258:15 +CPU= 30.562 22:09:54 STEP 1034 H= 258:30 +CPU= 31.104 22:10:24 STEP 1035 H= 258:45 +CPU= 30.225 22:10:54 STEP 1036 H= 259:00 +CPU= 29.089 22:11:24 STEP 1037 H= 259:15 +CPU= 30.020 22:11:54 STEP 1038 H= 259:30 +CPU= 29.822 22:12:24 STEP 1039 H= 259:45 +CPU= 29.589 22:12:55 STEP 1040 H= 260:00 +CPU= 31.232 22:13:26 STEP 1041 H= 260:15 +CPU= 30.632 22:13:58 STEP 1042 H= 260:30 +CPU= 31.187 22:14:28 STEP 1043 H= 260:45 +CPU= 30.583 22:15:08 STEP 1044 H= 261:00 +CPU= 39.258 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001044 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001044 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001044 22:15:38 STEP 1045 H= 261:15 +CPU= 30.624 22:16:09 STEP 1046 H= 261:30 +CPU= 30.717 22:16:40 STEP 1047 H= 261:45 +CPU= 31.064 22:17:14 STEP 1048 H= 262:00 +CPU= 33.158 22:17:45 STEP 1049 H= 262:15 +CPU= 31.105 22:18:15 STEP 1050 H= 262:30 +CPU= 29.676 22:18:45 STEP 1051 H= 262:45 +CPU= 29.942 22:19:17 STEP 1052 H= 263:00 +CPU= 30.988 22:19:47 STEP 1053 H= 263:15 +CPU= 29.614 22:20:18 STEP 1054 H= 263:30 +CPU= 30.957 22:20:48 STEP 1055 H= 263:45 +CPU= 30.387 22:21:31 STEP 1056 H= 264:00 +CPU= 42.549 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001056 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001056 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001056 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+000960 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+000960 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+000960 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+000972 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+000972 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+000972 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+000984 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+000984 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+000984 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+000996 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+000996 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+000996 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+001008 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+001008 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+001008 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+001020 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+001020 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+001020 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+001032 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+001032 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+001032 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+001044 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+001044 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+001044 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4sz_2016092300_20_1018_12294554_1_r1423833186_10.zip Uploading the intermediate file: upload_file_10.zip 22:22:03 STEP 1057 H= 264:15 +CPU= 31.389 22:22:33 STEP 1058 H= 264:30 +CPU= 29.603 22:23:03 STEP 1059 H= 264:45 +CPU= 29.627 22:23:33 STEP 1060 H= 265:00 +CPU= 30.131 22:24:04 STEP 1061 H= 265:15 +CPU= 29.643 22:24:34 STEP 1062 H= 265:30 +CPU= 30.119 22:25:04 STEP 1063 H= 265:45 +CPU= 29.865 22:25:35 STEP 1064 H= 266:00 +CPU= 30.670 22:26:05 STEP 1065 H= 266:15 +CPU= 29.837 22:26:34 STEP 1066 H= 266:30 +CPU= 28.894 22:27:03 STEP 1067 H= 266:45 +CPU= 28.972 22:27:40 STEP 1068 H= 267:00 +CPU= 36.674 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001068 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001068 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001068 22:28:09 STEP 1069 H= 267:15 +CPU= 29.434 22:28:40 STEP 1070 H= 267:30 +CPU= 30.208 22:29:09 STEP 1071 H= 267:45 +CPU= 28.982 22:29:39 STEP 1072 H= 268:00 +CPU= 29.361 22:30:08 STEP 1073 H= 268:15 +CPU= 29.440 22:30:38 STEP 1074 H= 268:30 +CPU= 29.375 22:31:07 STEP 1075 H= 268:45 +CPU= 29.118 22:31:38 STEP 1076 H= 269:00 +CPU= 29.718 22:32:06 STEP 1077 H= 269:15 +CPU= 28.333 22:32:36 STEP 1078 H= 269:30 +CPU= 28.777 22:33:05 STEP 1079 H= 269:45 +CPU= 29.065 22:33:42 STEP 1080 H= 270:00 +CPU= 37.069 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001080 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001080 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001080 22:34:11 STEP 1081 H= 270:15 +CPU= 28.174 22:34:40 STEP 1082 H= 270:30 +CPU= 29.191 22:35:10 STEP 1083 H= 270:45 +CPU= 29.726 22:35:41 STEP 1084 H= 271:00 +CPU= 30.254 22:36:10 STEP 1085 H= 271:15 +CPU= 28.909 22:36:39 STEP 1086 H= 271:30 +CPU= 28.529 22:37:07 STEP 1087 H= 271:45 +CPU= 28.540 22:37:37 STEP 1088 H= 272:00 +CPU= 29.477 22:38:07 STEP 1089 H= 272:15 +CPU= 29.155 22:38:36 STEP 1090 H= 272:30 +CPU= 28.739 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:39:26 STEP 1091 H= 272:45 +CPU= 28.577 22:40:02 STEP 1092 H= 273:00 +CPU= 35.919 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001092 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001092 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001092 22:40:31 STEP 1093 H= 273:15 +CPU= 28.863 22:41:01 STEP 1094 H= 273:30 +CPU= 29.300 22:41:31 STEP 1095 H= 273:45 +CPU= 29.832 22:42:03 STEP 1096 H= 274:00 +CPU= 31.366 22:42:32 STEP 1097 H= 274:15 +CPU= 29.136 22:43:01 STEP 1098 H= 274:30 +CPU= 28.425 22:43:30 STEP 1099 H= 274:45 +CPU= 28.657 22:43:59 STEP 1100 H= 275:00 +CPU= 29.621 22:44:29 STEP 1101 H= 275:15 +CPU= 29.566 22:44:58 STEP 1102 H= 275:30 +CPU= 28.560 22:45:28 STEP 1103 H= 275:45 +CPU= 29.136 22:46:08 STEP 1104 H= 276:00 +CPU= 40.279 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001104 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001104 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001104 22:46:37 STEP 1105 H= 276:15 +CPU= 28.797 22:47:06 STEP 1106 H= 276:30 +CPU= 28.784 22:47:35 STEP 1107 H= 276:45 +CPU= 28.130 22:48:05 STEP 1108 H= 277:00 +CPU= 29.817 22:48:34 STEP 1109 H= 277:15 +CPU= 29.027 22:49:03 STEP 1110 H= 277:30 +CPU= 28.887 22:49:32 STEP 1111 H= 277:45 +CPU= 28.865 22:50:02 STEP 1112 H= 278:00 +CPU= 29.640 22:50:31 STEP 1113 H= 278:15 +CPU= 28.685 22:51:00 STEP 1114 H= 278:30 +CPU= 28.704 22:51:29 STEP 1115 H= 278:45 +CPU= 29.328 22:52:06 STEP 1116 H= 279:00 +CPU= 36.062 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001116 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001116 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001116 22:52:34 STEP 1117 H= 279:15 +CPU= 28.480 22:53:03 STEP 1118 H= 279:30 +CPU= 28.425 22:53:32 STEP 1119 H= 279:45 +CPU= 28.900 22:54:03 STEP 1120 H= 280:00 +CPU= 30.562 22:54:31 STEP 1121 H= 280:15 +CPU= 28.517 22:55:00 STEP 1122 H= 280:30 +CPU= 28.361 22:55:28 STEP 1123 H= 280:45 +CPU= 28.338 22:55:57 STEP 1124 H= 281:00 +CPU= 28.859 22:56:26 STEP 1125 H= 281:15 +CPU= 28.568 22:56:55 STEP 1126 H= 281:30 +CPU= 28.943 22:57:24 STEP 1127 H= 281:45 +CPU= 28.564 22:58:01 STEP 1128 H= 282:00 +CPU= 36.389 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001128 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001128 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001128 22:58:29 STEP 1129 H= 282:15 +CPU= 28.248 22:58:59 STEP 1130 H= 282:30 +CPU= 29.201 22:59:28 STEP 1131 H= 282:45 +CPU= 29.160 22:59:58 STEP 1132 H= 283:00 +CPU= 29.746 23:00:28 STEP 1133 H= 283:15 +CPU= 29.961 23:00:58 STEP 1134 H= 283:30 +CPU= 28.936 23:01:27 STEP 1135 H= 283:45 +CPU= 28.878 23:01:57 STEP 1136 H= 284:00 +CPU= 29.850 23:02:25 STEP 1137 H= 284:15 +CPU= 28.019 23:02:55 STEP 1138 H= 284:30 +CPU= 28.784 23:03:25 STEP 1139 H= 284:45 +CPU= 30.162 23:04:02 STEP 1140 H= 285:00 +CPU= 36.543 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001140 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001140 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001140 23:04:32 STEP 1141 H= 285:15 +CPU= 29.525 23:05:02 STEP 1142 H= 285:30 +CPU= 30.106 23:05:30 STEP 1143 H= 285:45 +CPU= 28.249 23:06:00 STEP 1144 H= 286:00 +CPU= 29.514 23:06:29 STEP 1145 H= 286:15 +CPU= 28.911 23:06:58 STEP 1146 H= 286:30 +CPU= 28.474 23:07:28 STEP 1147 H= 286:45 +CPU= 29.707 23:07:58 STEP 1148 H= 287:00 +CPU= 29.675 23:08:28 STEP 1149 H= 287:15 +CPU= 29.143 23:08:57 STEP 1150 H= 287:30 +CPU= 29.097 23:09:27 STEP 1151 H= 287:45 +CPU= 29.742 23:10:08 STEP 1152 H= 288:00 +CPU= 40.598 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001152 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001152 Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001152 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+001056 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+001056 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+001056 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+001068 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+001068 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+001068 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+001080 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+001080 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+001080 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+001092 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+001092 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+001092 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+001104 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+001104 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+001104 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+001116 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+001116 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+001116 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+001128 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+001128 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+001128 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMGGh7zg+001140 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMSHh7zg+001140 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294554/ICMUAh7zg+001140 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4sz_2016092300_20_1018_12294554_1_r1423833186_11.zip Uploading the intermediate file: upload_file_11.zip 23:10:40 STEP 1153 H= 288:15 +CPU= 31.164 23:11:09 STEP 1154 H= 288:30 +CPU= 28.999 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231130:1718493090:38103.390] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1538] Received signal#8 (SIGFPE) :: 4412MB (heap), 5025MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231130:1718493090:38103.390] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231130:1718493090:38103.390] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x820890 (old at (nil)) [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231130:1718493090:38103.390] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1617] Signal#8 was caused by floating-point overflow [memaddr=0x1cc6692], nsigs = 1 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231130:1718493090:38103.390] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#8, nsigs = 1 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231130:1718493090:38103.390] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3843] 4412 MB (maxheap), 5025 MB (maxrss), 0 MB (maxstack) [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231130:1718493090:38103.390] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : MASTER [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231130:1718493090:38103.390] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT0 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231130:1718493090:38103.390] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT1 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231130:1718493090:38103.390] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT2 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231130:1718493090:38103.390] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT3 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231130:1718493090:38103.390] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT4 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231130:1718493090:38103.390] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : STEPO [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231130:1718493090:38103.390] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : SCAN2M [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231130:1718493090:38103.390] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231130:1718493090:38103.390] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : GP_MODEL [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231130:1718493090:38103.390] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : EC_PHYS_DRV [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231130:1718493090:38103.390] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : >OMP-PHYSICS (1001) [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231130:1718493090:38103.390] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : EC_PHYS [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231130:1718493090:38103.390] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CALLPAR [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231130:1718493090:38103.390] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : TURBULENCE_LAYER [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231130:1718493090:38103.390] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : VDFOUTER [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231130:1718493090:38103.390] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : VDFMAIN [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc-client/slots/7/oifs_43r3_model.exe' : sigcontextptr=0x7ffee6b08950 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 24) with addr2line-cmd [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:302] /usr/bin/addr2line -fs -e '/var/lib/boinc-client/slots/7/oifs_43r3_model.exe' 0x83a8f3 0x83aef9 0x1dd5490 0x1cc6692 0x121fd8b 0x1207e29 0x11fdcb2 0xfac003 0xf5fa6b 0xf4dbc4 0xbd7131 0x12c5002 0xbacbae 0x688153 0x4cd1dc 0x4b5d85 0x4b4f52 0x49d638 0x406fbd 0x401e7f 0x401e12 0x1dcc34a 0x1dcdbe7 0x401cd5 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [0]: [0x83a8f3] : ??() at ??:0 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [1]: [0x83aef9] : ??() at ??:0 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [2]: [0x1dd5490] : ??() at ??:0 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [3]: [0x1cc6692] : ??() at ??:0 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [4]: [0x121fd8b] : ??() at ??:0 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [5]: [0x1207e29] : ??() at ??:0 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [6]: [0x11fdcb2] : ??() at ??:0 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [7]: [0xfac003] : ??() at ??:0 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [8]: [0xf5fa6b] : ??() at ??:0 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [9]: [0xf4dbc4] : ??() at ??:0 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [10]: [0xbd7131] : ??() at ??:0 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [11]: [0x12c5002] : ??() at ??:0 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [12]: [0xbacbae] : ??() at ??:0 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [13]: [0x688153] : ??() at ??:0 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [14]: [0x4cd1dc] : ??() at ??:0 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [15]: [0x4b5d85] : ??() at ??:0 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [16]: [0x4b4f52] : ??() at ??:0 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [17]: [0x49d638] : ??() at ??:0 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [18]: [0x406fbd] : ??() at ??:0 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [19]: [0x401e7f] : ??() at ??:0 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [20]: [0x401e12] : ??() at ??:0 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [21]: [0x1dcc34a] : ??() at ??:0 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [22]: [0x1dcdbe7] : ??() at ??:0 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [23]: [0x401cd5] : ??() at ??:0 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.391] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s) Process 1 thread 1 calling linux_trbk from intel_trbk() [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc-client/slots/7/oifs_43r3_model.exe' : sigcontextptr=0x7ffee6b08860 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 25) with addr2line-cmd [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:302] /usr/bin/addr2line -fs -e '/var/lib/boinc-client/slots/7/oifs_43r3_model.exe' 0x83a8f3 0x83aef9 0x822540 0x1dd5490 0x1cc6692 0x121fd8b 0x1207e29 0x11fdcb2 0xfac003 0xf5fa6b 0xf4dbc4 0xbd7131 0x12c5002 0xbacbae 0x688153 0x4cd1dc 0x4b5d85 0x4b4f52 0x49d638 0x406fbd 0x401e7f 0x401e12 0x1dcc34a 0x1dcdbe7 0x401cd5 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [0]: [0x83a8f3] : ??() at ??:0 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [1]: [0x83aef9] : ??() at ??:0 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [2]: [0x822540] : ??() at ??:0 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [3]: [0x1dd5490] : ??() at ??:0 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [4]: [0x1cc6692] : ??() at ??:0 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [5]: [0x121fd8b] : ??() at ??:0 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [6]: [0x1207e29] : ??() at ??:0 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [7]: [0x11fdcb2] : ??() at ??:0 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [8]: [0xfac003] : ??() at ??:0 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [9]: [0xf5fa6b] : ??() at ??:0 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [10]: [0xf4dbc4] : ??() at ??:0 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [11]: [0xbd7131] : ??() at ??:0 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [12]: [0x12c5002] : ??() at ??:0 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [13]: [0xbacbae] : ??() at ??:0 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [14]: [0x688153] : ??() at ??:0 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [15]: [0x4cd1dc] : ??() at ??:0 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [16]: [0x4b5d85] : ??() at ??:0 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [17]: [0x4b4f52] : ??() at ??:0 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [18]: [0x49d638] : ??() at ??:0 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [19]: [0x406fbd] : ??() at ??:0 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [20]: [0x401e7f] : ??() at ??:0 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [21]: [0x401e12] : ??() at ??:0 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [22]: [0x1dcc34a] : ??() at ??:0 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [23]: [0x1dcdbe7] : ??() at ??:0 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [24]: [0x401cd5] : ??() at ??:0 [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s) [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.406] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#8, nsigs = 1 [EC_DRHOOK:ubuntu:1:1:3239568:3239568] [20240615:231131:1718493091:38104.406] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1cf8da0 for signal#8, nsigs = 1 forrtl: error (72): floating overflow Image PC Routine Line Source oifs_43r3_model.e 0000000001CF935B Unknown Unknown Unknown oifs_43r3_model.e 00000000008214C1 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DD5490 Unknown Unknown Unknown oifs_43r3_model.e 0000000001CC6692 Unknown Unknown Unknown oifs_43r3_model.e 000000000121FD8B vdfmain_ 874 vdfmain.F90 oifs_43r3_model.e 0000000001207E29 vdfouter_ 739 vdfouter.F90 oifs_43r3_model.e 00000000011FDCB2 turbulence_layer_ 158 turbulence_layer.F90 oifs_43r3_model.e 0000000000FAC003 callpar_ 869 callpar.F90 oifs_43r3_model.e 0000000000F5FA6B ec_phys_ 670 ec_phys.F90 oifs_43r3_model.e 0000000000F4DBC4 ec_phys_drv_ 599 ec_phys_drv.F90 oifs_43r3_model.e 0000000000BD7131 gp_model_ 613 gp_model.F90 oifs_43r3_model.e 00000000012C5002 gp_model_heap_ 74 gp_model_heap.F90 oifs_43r3_model.e 0000000000BACBAE scan2m_ 535 scan2m.F90 oifs_43r3_model.e 0000000000688153 stepo_ 327 stepo.F90 oifs_43r3_model.e 00000000004CD1DC cnt4_ 1133 cnt4.F90 oifs_43r3_model.e 00000000004B5D85 cnt3_ 267 cnt3.F90 oifs_43r3_model.e 00000000004B4F52 cnt2_ 88 cnt2.F90 oifs_43r3_model.e 000000000049D638 cnt1_ 92 cnt1.F90 oifs_43r3_model.e 0000000000406FBD cnt0_ 146 cnt0.F90 oifs_43r3_model.e 0000000000401E7F MAIN__ 96 master.F90 oifs_43r3_model.e 0000000000401E12 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DCC34A Unknown Unknown Unknown oifs_43r3_model.e 0000000001DCDBE7 Unknown Unknown Unknown oifs_43r3_model.e 0000000000401CD5 Unknown Unknown Unknown ..The child process has been killed with signal: 6 >>> Printing last 70 lines from file: NODE.001_01 GPNORM CLOUD FRACTION AVERAGE MINIMUM MAXIMUM AVE 0.234398488330141E+00 0.000000000000000E+00 0.100000000000000E+01 GPNORM CLW AVERAGE MINIMUM MAXIMUM AVE 0.354333504306688E-05 0.000000000000000E+00 0.850193317378906E-02 GPNORM CIW AVERAGE MINIMUM MAXIMUM AVE 0.262908651152596E-04 0.000000000000000E+00 0.122578261430972E-01 GPNORM CC AVERAGE MINIMUM MAXIMUM AVE 0.234125122981385E+00 0.000000000000000E+00 0.100000000000000E+01 GPNORM RFL AVERAGE MINIMUM MAXIMUM AVE 0.252846541440154E-04 0.000000000000000E+00 0.256617756543740E-01 GPNORM SFL AVERAGE MINIMUM MAXIMUM AVE 0.177075515467531E-04 0.000000000000000E+00 0.642793496776320E-02 GPNORM CRFL AVERAGE MINIMUM MAXIMUM AVE 0.579023029707175E-05 0.000000000000000E+00 0.205567062165160E-01 GPNORM CSFL AVERAGE MINIMUM MAXIMUM AVE 0.177782461529425E-05 0.000000000000000E+00 0.115399420929837E-01 GPNORM PFRC AVERAGE MINIMUM MAXIMUM AVE 0.198955430319888E+00 0.000000000000000E+00 0.100000000000000E+01 NSTEP = 1152 STEPO F00000000 CREATE RESTART FILE ./srf00120000.0001 IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ./srf00120000.0001 MODE=w IO-STREAM CLOSED - ./srf00120000.0001 MPI-TASK: 1 - ************ BYTES IN 46 RECORDS TRANSFERRED IN 2.7823 SECONDS******** Mbytes/s, TOTAL TIME= 3.4942(79.6%) CREATE RESTART CONTROL FILE ./rcf WRRESF: DELETED PREVIOUS RESTART FILE OK: ./srf00111200.0001 MAXGPFV : MAX. VALUE = 0.000000000000000E+000 MAXGPFV : MAX. VALUE = 0.000000000000000E+000 NSTEP = 1152 SCAN2M_HPOS P IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ICMGGh7zg+001152 MODE=w IO-STREAM CLOSED - ICMGGh7zg+001152 MPI-TASK: 1 - 316357972 BYTES IN 6 RECORDS TRANSFERRED IN 0.0002 SECONDS******** Mbytes/s, TOTAL TIME= 0.0060( 3.3%) NSTEP = 1152 STEPO 0AA000000 IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ./ICMSHh7zg+001152 MODE=w IO-STREAM CLOSED - ./ICMSHh7zg+001152 MPI-TASK: 1 - 707399250 BYTES IN 202 RECORDS TRANSFERRED IN 0.0070 SECONDS101057.0 Mbytes/s, TOTAL TIME= 0.2954( 2.4%) IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ./ICMUAh7zg+001152 MODE=w IO-STREAM CLOSED - ./ICMUAh7zg+001152 MPI-TASK: 1 - 5061727552 BYTES IN 125 RECORDS TRANSFERRED IN 0.0048 SECONDS******** Mbytes/s, TOTAL TIME= 0.2243( 2.1%) STEPO_FPOS WAS : B00000F00E IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ./ICMUAh7zg+001152 MODE=a IO-STREAM CLOSED - ./ICMUAh7zg+001152 MPI-TASK: 1 - 10 BYTES IN 2 RECORDS TRANSFERRED IN 0.0003 SECONDS 0.0 Mbytes/s, TOTAL TIME= 0.0030(10.0%) STEPO_FPOS WAS : V00000000Y IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ./ICMUAh7zg+001152 MODE=a IO-STREAM CLOSED - ./ICMUAh7zg+001152 MPI-TASK: 1 - 307778243 BYTES IN 4 RECORDS TRANSFERRED IN 0.0002 SECONDS******** Mbytes/s, TOTAL TIME= 0.0043( 4.7%) STEPO_FPOS WAS : T00000000M NSTEP = 1152 STEPO 0AAA00AAA MAXGPFV : MAX. VALUE = 0.000000000000000E+000 MAXGPFV : MAX. VALUE = 0.000000000000000E+000 NSTEP = 1152 SCAN2M_HPOS P IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ICMGGh7zg+001152 MODE=a IO-STREAM CLOSED - ICMGGh7zg+001152 MPI-TASK: 1 - 316357972 BYTES IN 7 RECORDS TRANSFERRED IN 0.0004 SECONDS790894.9 Mbytes/s, TOTAL TIME= 0.0069( 5.8%) 23:10:08 STEP 1152 H= 288:00 +CPU= 40.598 NSTEP = 1153 STEPO 0AAA00AAA 23:10:40 STEP 1153 H= 288:15 +CPU= 31.164 NSTEP = 1154 STEPO 0AAA00AAA 23:11:09 STEP 1154 H= 288:30 +CPU= 28.999 ISTEP= 1155 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482 ISTEP= 1155 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00 NSTEP = 1155 STEPO 0AAA00AAA ------------------------------------------------ CNT0 not found; string returned was: 'STEPO' >>> Printing last 8 lines from file: ifs.stat 23:09:33 FULLPOS-B DYNFPOS 1152 1.423 1.423 1.427 547:13 633:06 0.37558479549783E-04 2GB 0MB 23:09:37 FULLPOS-V DYNFPOS 1152 3.727 3.727 4.000 547:17 633:10 0.37558479549783E-04 2GB 0MB 23:09:38 FULLPOS-T DYNFPOS 1152 0.787 0.787 0.789 547:17 633:11 0.37558479549783E-04 2GB 0MB 23:09:38 0AAA00AAA STEPO 1152 0.784 0.784 0.787 547:18 633:11 0.37558479549783E-04 2GB 0MB 23:10:08 0AAA00AAA STEPO 1153 29.722 29.722 29.839 547:48 633:41 0.37558479549783E-04 2GB 0MB 23:10:40 0AAA00AAA STEPO 1154 31.164 31.164 31.580 548:19 634:13 0.37558479549783E-04 2GB 0MB 23:11:09 0AAA00AAA STEPO 1155 28.999 28.999 29.213 548:48 634:42 0.37558479549783E-04 2GB 0MB ------------------------------------------------ >>> Printing last 11 lines from file: rcf CSTEP = " 1152", CTIME = "00120000 ", NSTEPLPP = 201, 202, 49, 228226, 228227, 2*0, 1140, 2*0, IPRGPNSRES = 1, IPRGPEWRES = 1, IPRTRWRES = 1, IPRTRVRES = 1, GMASS0 = 0.000000000000000E+000, GMASSI = 0.000000000000000E+000 / ------------------------------------------------ >>> Printing last 8 lines from file: /var/lib/boinc-client/slots/7/progress_file_12294554.xml <running_values> <last_cpu_time>32944.690000</last_cpu_time> <upload_file_number>12</upload_file_number> <last_iter>1155</last_iter> <last_upload>1036800</last_upload> <model_completed>0</model_completed> </running_values> ------------------------------------------------ ..Failed, model did not complete successfully </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
15 Jun 2024 21:32:31 | 1551095 | 22449120 | oifs_43r3_bl_a4sz_2016092300_20_1018_12294554_1 | 864,000 | 27,074 | 0.0313 |
©2024 cpdn.org