climateprediction.net home page

Download error


Advanced search

Message boards : Number crunching : Download error

AuthorMessage
Melvyn Bobo Slacke
Send message
Joined: Aug 16 04
Posts: 128
Credit: 4,403,088
RAC: 1,967
Message 43219 - Posted 15 Oct 2011 3:52:59 UTC

    Last modified: 15 Oct 2011 4:43:54 UTC

    There are some new hadam3p_eu out but I can't get two files: "15-Oct-2011 05:03:12 [climateprediction.net] [error] File so2dms_N96_2004_12_2006_02f_1900rescale.gz has wrong size: expected 2122795, got 0" and o3_N96_pers_1959_1998_2011.gz

    Edit: 15-Oct-2011 06:03:52 [climateprediction.net] task hadam3p_eu_6fhk_2004_1_007487341_0 aborted by user

    Edit2: same crap on another box "File so2dms_N96_2001_12_2003_02f_1900rescale.gz has wrong size: expected 2122796, got 0" and File o3_N96_pers_1959_1998_2011.gz

    Les Bayliss
    Forum moderator
    Send message
    Joined: Sep 5 04
    Posts: 5424
    Credit: 9,074,925
    RAC: 2,357
    Message 43220 - Posted 15 Oct 2011 6:09:27 UTC

      Download errors for those 2 files is a server problem.


      ____________
      Backups: Here

      skgiven
      Avatar
      Send message
      Joined: Jun 5 06
      Posts: 27
      Credit: 1,516,223
      RAC: 692
      Message 43221 - Posted 15 Oct 2011 10:18:49 UTC - in response to Message 43220.

        Last modified: 15 Oct 2011 10:57:40 UTC

        I have download problems, from yesterday evening (UK time, 12h or so):
        6 tasks are still in the process of downloading.
        6 Files are still downloading (retried many times)

        03_N96_pers_1959_1998_2011.gz
        so2dns_N96_2002_12_2004_02f_1900rescale.gz
        so2dns_N96_2004_12_2006_02f_1900rescale.gz
        so2dns_N96_2000_12_2002_02f_1900rescale.gz
        so2dns_N96_2001_12_2003_02f_1900rescale.gz
        so2dns_N96_2003_12_2005_02f_1900rescale.gz

        There appears to be no server problems on the server status page.

        I can download tasks from other projects.

        15/10/2011 11:44:47 climateprediction.net Started download of so2dms_N96_2004_12_2006_02f_1900rescale.gz
        15/10/2011 11:44:49 climateprediction.net Temporarily failed download of so2dms_N96_2004_12_2006_02f_1900rescale.gz: HTTP error
        15/10/2011 11:44:49 climateprediction.net Backing off 1 hr 16 min 14 sec on download of so2dms_N96_2004_12_2006_02f_1900rescale.gz


        Tried the cc_config.xml option, <http_1_0>1</http_1_0>, but no success,

        15/10/2011 11:50:52 climateprediction.net Started download of so2dms_N96_2004_12_2006_02f_1900rescale.gz
        15/10/2011 11:50:54 climateprediction.net Temporarily failed download of so2dms_N96_2004_12_2006_02f_1900rescale.gz: HTTP error
        15/10/2011 11:50:54 climateprediction.net Backing off 54 min 44 sec on download of so2dms_N96_2004_12_2006_02f_1900rescale.gz
        15/10/2011 11:51:02 climateprediction.net Started download of so2dms_N96_2000_12_2002_02f_1900rescale.gz
        15/10/2011 11:51:03 climateprediction.net Temporarily failed download of so2dms_N96_2000_12_2002_02f_1900rescale.gz: HTTP error
        15/10/2011 11:51:03 climateprediction.net Backing off 1 hr 34 min 47 sec on download of so2dms_N96_2000_12_2002_02f_1900rescale.gz

        Tried a project reset, but the servers want Boinc to backoff for an hour (must be as sick of me I as I of them).
        Going back to HTTP 1.1
        ____________

        Richard Haselgrove
        Send message
        Joined: Jan 1 07
        Posts: 300
        Credit: 6,878,929
        RAC: 1
        Message 43222 - Posted 15 Oct 2011 11:30:08 UTC - in response to Message 43221.

          I have download problems, from yesterday evening (UK time, 12h or so):
          6 tasks are still in the process of downloading.
          6 Files are still downloading (retried many times)

          Tried the cc_config.xml option, <http_1_0>1</http_1_0>, but no success,
          ...

          You're a moderator/tester on other BOINC projects, you know how these things work.

          Why not try <http_debug>1</http_debug> first, to find out what's wrong, before guessing at a solution?

          15/10/2011 12:18:28 | | [http] [ID#5103] Info: About to connect() to cpdn-downloads.oerc.ox.ac.uk port 80 (#0)
          15/10/2011 12:18:28 | | [http] [ID#5103] Info: Trying 129.67.195.71...
          15/10/2011 12:18:28 | | [http] [ID#5103] Info: Connected to cpdn-downloads.oerc.ox.ac.uk (129.67.195.71) port 80 (#0)
          15/10/2011 12:18:28 | | [http] [ID#5103] Received header from server: HTTP/1.1 302 Found
          15/10/2011 12:18:28 | | [http] [ID#5103] Received header from server:
          15/10/2011 12:18:28 | | [http] [ID#5103] Info: Ignoring the response-body
          15/10/2011 12:18:28 | | [http] [ID#5103] Info: Connection #0 to host cpdn-downloads.oerc.ox.ac.uk left intact
          15/10/2011 12:18:28 | | [http] [ID#5103] Info: Issue another request to this URL: 'http://cpdn-downloads.oerc.ox.ac.uk/download/hadam3p/ancil/o3_N96_pers_1959_1998_2011.gz'
          15/10/2011 12:18:28 | | [http] [ID#5103] Info: Re-using existing connection! (#0) with host cpdn-downloads.oerc.ox.ac.uk
          15/10/2011 12:18:28 | | [http] [ID#5103] Info: Connected to cpdn-downloads.oerc.ox.ac.uk (129.67.195.71) port 80 (#0)
          15/10/2011 12:18:28 | | [http] [ID#5103] Sent header to server: GET /download/hadam3p/ancil/o3_N96_pers_1959_1998_2011.gz HTTP/1.1
          15/10/2011 12:18:29 | | [http] [ID#5103] Received header from server: HTTP/1.1 403 Forbidden
          15/10/2011 12:18:29 | climateprediction.net | Temporarily failed download of o3_N96_pers_1959_1998_2011.gz: HTTP error

          (several lines omitted for brevity)

          I think you'll find that is a server problem, as Les said: quite possibly a security setting gone awry on cpdn-downloads.oerc.ox.ac.uk/download/hadam3p/ancil/

          Profile Byron Leigh Hatch @ team Carl Sagan
          Send message
          Joined: Aug 17 04
          Posts: 177
          Credit: 4,259,436
          RAC: 3,572
          Message 43223 - Posted 15 Oct 2011 11:31:21 UTC

            yep me too :( _ reporting the following:

            Temporarily failed download of so2dms_N96_2009_12_2011_02f_1900rescale.gz: HTTP error
            Temporarily failed download of so2dms_N96_2005_12_2007_02f_1900rescale.gz: HTTP error
            Temporarily failed download of so2dms_N96_2008_12_2010_02f_1900rescale.gz: HTTP error
            Temporarily failed download of so2dms_N96_2006_12_2008_02f_1900rescale.gz: HTTP error
            Temporarily failed download of so2dms_N96_2002_12_2004_02f_1900rescale.gz: HTTP error
            Temporarily failed download of so2dms_N96_2003_12_2005_02f_1900rescale.gz: HTTP error
            Temporarily failed download of o3_N96_pers_1959_1998_2011.gz: HTTP error

            skgiven
            Avatar
            Send message
            Joined: Jun 5 06
            Posts: 27
            Credit: 1,516,223
            RAC: 692
            Message 43224 - Posted 15 Oct 2011 15:02:59 UTC - in response to Message 43222.

              Thought it might be reasonable to try <http_1_0>1</http_1_0> as that potential solution worked on the 13th, http://climateapps2.oerc.ox.ac.uk/cpdnboinc/forum_thread.php?id=7308.
              It didn't work so I wanted to report what I could (time since this problem started), might help someone fix the problem. Don't know if this is different form the OP or not, but slightly different error messages.

              Still seeing the HTTP Errors and 403 Forbidden, but at 12.53 (UK Time) 4 tasks succeeded in downloading and are now running.

              The files that still have not downloaded are,
              so2dns_N96_2001_12_2003_02f_1900rescale.gz
              03_N96_pers_1959_1998_2011.gz
              ____________

              Profile JIM
              Send message
              Joined: Dec 31 07
              Posts: 682
              Credit: 4,215,401
              RAC: 2,853
              Message 43226 - Posted 15 Oct 2011 15:28:18 UTC

                I have the same problem here. Two WU?s stuck in download.

                Apprently, about an hour and a half ago I started WU hadam3p_eu_1965_1_007485611_2 started and then failed after 13 minutes.

                <core_client_version>6.10.58</core_client_version>
                <![CDATA[
                <stderr_txt>

                Model crashed: 
                Leaving CPDN_Main::Monitor...
                Called boinc_finish

                </stderr_txt>
                <message>
                <file_xfer_error>


                The server then attempted the download of 2 new WUs that are now stuck.

                10/15/2011 9:55:11 AM climateprediction.net Scheduler request completed: got 2 new tasks
                10/15/2011 9:55:13 AM climateprediction.net Started download of hadam3p_eu_6rpv_2004_1_007503192.zip
                10/15/2011 9:55:13 AM climateprediction.net Started download of o3_N96_pers_1959_1998_2011.gz
                10/15/2011 9:55:16 AM climateprediction.net Finished download of hadam3p_eu_6rpv_2004_1_007503192.zip
                10/15/2011 9:55:16 AM climateprediction.net Temporarily failed download of o3_N96_pers_1959_1998_2011.gz: HTTP error
                10/15/2011 9:55:16 AM climateprediction.net Backing off 1 min 0 sec on download of o3_N96_pers_1959_1998_2011.gz
                10/15/2011 9:55:16 AM climateprediction.net Started download of ic19610814_11_N96.gz
                10/15/2011 9:55:16 AM climateprediction.net Started download of xaclfa.start.0000.gz
                10/15/2011 9:55:19 AM climateprediction.net Finished download of ic19610814_11_N96.gz
                10/15/2011 9:55:19 AM climateprediction.net Started download of so2dms_N96_2004_12_2006_02f_1900rescale.gz
                10/15/2011 9:55:20 AM climateprediction.net Temporarily failed download of so2dms_N96_2004_12_2006_02f_1900rescale.gz: HTTP error
                10/15/2011 9:55:20 AM climateprediction.net Backing off 1 min 0 sec on download of so2dms_N96_2004_12_2006_02f_1900rescale.gz
                10/15/2011 9:55:20 AM climateprediction.net Started download of dchapa.start.EU.b.new.0000.gz
                10/15/2011 9:55:28 AM climateprediction.net Finished download of dchapa.start.EU.b.new.0000.gz
                10/15/2011 9:55:28 AM climateprediction.net Started download of HadISST_SI_N96_2004_12_2006_01f_1900.gz
                10/15/2011 9:55:30 AM climateprediction.net Finished download of HadISST_SI_N96_2004_12_2006_01f_1900.gz
                10/15/2011 9:55:30 AM climateprediction.net Started download of HadISST_SST_N96_2004_12_2006_01f_1900.gz
                10/15/2011 9:55:36 AM climateprediction.net Finished download of HadISST_SST_N96_2004_12_2006_01f_1900.gz
                10/15/2011 9:55:36 AM climateprediction.net Started download of hadam3p_eu_6rqb_2009_1_007503208.zip
                10/15/2011 9:55:37 AM climateprediction.net Finished download of hadam3p_eu_6rqb_2009_1_007503208.zip
                10/15/2011 9:55:37 AM climateprediction.net Started download of ic19610513_14_N96.gz
                10/15/2011 9:55:40 AM climateprediction.net Finished download of ic19610513_14_N96.gz
                10/15/2011 9:55:40 AM climateprediction.net Started download of so2dms_N96_2009_12_2011_02f_1900rescale.gz
                10/15/2011 9:55:42 AM climateprediction.net Temporarily failed download of so2dms_N96_2009_12_2011_02f_1900rescale.gz: HTTP error
                10/15/2011 9:55:42 AM climateprediction.net Backing off 1 min 0 sec on download of so2dms_N96_2009_12_2011_02f_1900rescale.gz
                10/15/2011 9:55:42 AM climateprediction.net Started download of HadISST_SI_N96_2009_12_2011_01f_1900.gz
                10/15/2011 9:55:43 AM climateprediction.net Finished download of xaclfa.start.0000.gz
                10/15/2011 9:55:43 AM climateprediction.net Started download of HadISST_SST_N96_2009_12_2011_01f_1900.gz
                10/15/2011 9:55:44 AM climateprediction.net Finished download of HadISST_SI_N96_2009_12_2011_01f_1900.gz
                10/15/2011 9:55:50 AM climateprediction.net Finished download of HadISST_SST_N96_2009_12_2011_01f_1900.gz
                10/15/2011 9:56:16 AM climateprediction.net Started download of o3_N96_pers_1959_1998_2011.gz
                10/15/2011 9:56:18 AM climateprediction.net Temporarily failed download of o3_N96_pers_1959_1998_2011.gz: HTTP error
                10/15/2011 9:56:18 AM climateprediction.net Backing off 1 min 0 sec on download of o3_N96_pers_1959_1998_2011.gz
                10/15/2011 9:56:20 AM climateprediction.net Started download of so2dms_N96_2004_12_2006_02f_1900rescale.gz
                10/15/2011 9:56:21 AM climateprediction.net Temporarily failed download of so2dms_N96_2004_12_2006_02f_1900rescale.gz: HTTP error
                10/15/2011 9:56:21 AM climateprediction.net Backing off 1 min 0 sec on download of so2dms_N96_2004_12_2006_02f_1900rescale.gz
                10/15/2011 9:56:43 AM climateprediction.net Started download of so2dms_N96_2009_12_2011_02f_1900rescale.gz
                10/15/2011 9:56:44 AM climateprediction.net Temporarily failed download of so2dms_N96_2009_12_2011_02f_1900rescale.gz: HTTP error
                10/15/2011 9:56:44 AM climateprediction.net Backing off 1 min 0 sec on download of so2dms_N96_2009_12_2011_02f_1900rescale.gz
                10/15/2011 9:57:44 AM climateprediction.net Started download of o3_N96_pers_1959_1998_2011.gz
                10/15/2011 9:57:44 AM climateprediction.net Started download of so2dms_N96_2004_12_2006_02f_1900rescale.gz
                10/15/2011 9:57:46 AM climateprediction.net Temporarily failed download of o3_N96_pers_1959_1998_2011.gz: HTTP error
                10/15/2011 9:57:46 AM climateprediction.net Backing off 1 min 0 sec on download of o3_N96_pers_1959_1998_2011.gz
                10/15/2011 9:57:46 AM climateprediction.net Temporarily failed download of so2dms_N96_2004_12_2006_02f_1900rescale.gz: HTTP error
                10/15/2011 9:57:46 AM climateprediction.net Backing off 1 min 0 sec on download of so2dms_N96_2004_12_2006_02f_1900rescale.gz
                10/15/2011 9:59:11 AM climateprediction.net Started download of o3_N96_pers_1959_1998_2011.gz
                10/15/2011 9:59:11 AM climateprediction.net Started download of so2dms_N96_2004_12_2006_02f_1900rescale.gz
                10/15/2011 9:59:12 AM climateprediction.net Temporarily failed download of o3_N96_pers_1959_1998_2011.gz: HTTP error
                10/15/2011 9:59:12 AM climateprediction.net Backing off 1 min 0 sec on download of o3_N96_pers_1959_1998_2011.gz


                ____________

                Eirik Redd
                Send message
                Joined: Aug 31 04
                Posts: 262
                Credit: 28,540,901
                RAC: 26,537
                Message 43229 - Posted 16 Oct 2011 6:59:27 UTC

                  Last modified: 16 Oct 2011 7:10:41 UTC

                  So which is better to do?

                  1- cancel download and hope for a wu that can download?
                  2- just wait -- knowing that these type problems usually get fixed soonish.

                  I'm doing option 2 -- better ideas?

                  Is it just some fraction of current downloadable wu - 14000 or so that have the problem or all of them?

                  Eager for CPDN work

                  Edit -- sometimes it seems that the failed downloads are blocking other downloads for BOINC CPND stuff -- this is a BOINC problem for sure

                  Eirikr
                  ____________

                  Les Bayliss
                  Forum moderator
                  Send message
                  Joined: Sep 5 04
                  Posts: 5424
                  Credit: 9,074,925
                  RAC: 2,357
                  Message 43230 - Posted 16 Oct 2011 7:18:27 UTC - in response to Message 43229.

                    Have you read my news item about this?

                    ____________
                    Backups: Here

                    Profile Thyme Lawn
                    Forum moderator
                    Send message
                    Joined: Aug 5 04
                    Posts: 1232
                    Credit: 10,445,655
                    RAC: 1,218
                    Message 43231 - Posted 16 Oct 2011 8:07:15 UTC - in response to Message 43229.

                      So which is better to do?

                      1- cancel download and hope for a wu that can download?
                      2- just wait -- knowing that these type problems usually get fixed soonish.

                      I'm doing option 2 -- better ideas?

                      That's the best approach. The downloads will succeed as soon as the permissions problem on the server has been fixed.

                      Edit -- sometimes it seems that the failed downloads are blocking other downloads for BOINC CPND stuff -- this is a BOINC problem for sure

                      Indeed it is, and there are plans to change the behaviour.

                      Currently BOINC has dynamic upload and download backoff periods for the project and each queued file. When the project backoff expires BOINC tries to upload or download files which have been in the queue longest and aren't backed off. This can result in blocking when a project uses multiple servers and one is down (e.g. CPDN has 8 upload servers and uploads from a task can currently be directed to up 4 different servers) or there are file specific problems (e.g. the current permissions problem with so2dms_* and o3_* downloads).

                      The mechanism is being changed to try to send/receive files which haven't been attempted before doing any retries. I don't know if this will be in a future BOINC 6.12.* or have to wait until 6.14.*.

                      There is, however, a perverse upside to the current problem. The project has an ongoing problem with misconfigured and incompatible hosts which immediately crash every task sent to them (blocking them until the owner sorts out the problem is also ongoing). A lot of these hosts have yet to be identified and are crashing tasks until they hit their daily quota. The download blockage means they can't crash tasks so reliable hosts will have a better chance of getting work than usual ...
                      ____________
                      "The ultimate test of a moral society is the kind of world that it leaves to its children." - Dietrich Bonhoeffer

                      MarkJ
                      Avatar
                      Send message
                      Joined: Mar 28 09
                      Posts: 104
                      Credit: 5,123,022
                      RAC: 813
                      Message 43232 - Posted 16 Oct 2011 12:17:55 UTC - in response to Message 43231.

                        Last modified: 16 Oct 2011 12:18:39 UTC

                        The mechanism is being changed to try to send/receive files which haven't been attempted before doing any retries. I don't know if this will be in a future BOINC 6.12.* or have to wait until 6.14.*.

                        <Rest cut for brevity>


                        Its in 6.13.7.

                        Profile Iain Inglis
                        Forum moderator
                        Send message
                        Joined: Jan 16 10
                        Posts: 522
                        Credit: 9,532
                        RAC: 0
                        Message 43233 - Posted 16 Oct 2011 12:34:31 UTC

                          The permissions should now have been fixed. If anyone gets a new occurence of this problem then please report it here ...

                          skgiven
                          Avatar
                          Send message
                          Joined: Jun 5 06
                          Posts: 27
                          Credit: 1,516,223
                          RAC: 692
                          Message 43235 - Posted 16 Oct 2011 13:38:21 UTC - in response to Message 43233.

                            Both files that were previously not downloading on my system have now downloaded.
                            Thanks,
                            ____________

                            Profile Byron Leigh Hatch @ team Carl Sagan
                            Send message
                            Joined: Aug 17 04
                            Posts: 177
                            Credit: 4,259,436
                            RAC: 3,572
                            Message 43236 - Posted 16 Oct 2011 13:42:10 UTC - in response to Message 43235.

                              Both files that were previously not downloading on my system have now downloaded.
                              Thanks,

                              Yes, I can confirm the same on my computer. Thank you to who ever fix this :-)

                              Darmok
                              Avatar
                              Send message
                              Joined: Dec 29 09
                              Posts: 27
                              Credit: 2,611,432
                              RAC: 516
                              Message 43261 - Posted 18 Oct 2011 23:59:39 UTC

                                Got this new model:

                                10/18/2011 6:20:31 PM climateprediction.net Scheduler request completed: got 1 new tasks
                                10/18/2011 6:20:33 PM climateprediction.net Started download of hadcm3n_0025_1900_40_007486274.zip
                                10/18/2011 6:20:34 PM climateprediction.net Finished download of hadcm3n_0025_1900_40_007486274.zip

                                and that was it for the download with the atmos and ocean files missing.Files also absent from the HD. Nothing else was downloaded on the next hour.

                                Darmok
                                Avatar
                                Send message
                                Joined: Dec 29 09
                                Posts: 27
                                Credit: 2,611,432
                                RAC: 516
                                Message 43275 - Posted 25 Oct 2011 21:37:28 UTC

                                  Seeing this on My Account page:

                                  Warning: mysql_num_rows() expects parameter 1 to be resource, boolean given in /var/www/boinc/projects/cpdnboinc/html/inc/user.inc on line 152

                                  Warning: mysql_fetch_assoc() expects parameter 1 to be resource, boolean given in /var/www/boinc/projects/cpdnboinc/html/inc/user.inc on line 166

                                  Les Bayliss
                                  Forum moderator
                                  Send message
                                  Joined: Sep 5 04
                                  Posts: 5424
                                  Credit: 9,074,925
                                  RAC: 2,357
                                  Message 43281 - Posted 26 Oct 2011 1:09:19 UTC - in response to Message 43275.

                                    PHP board

                                    Post to thread

                                    Message boards : Number crunching : Download error




                                    Copyright © 2002-2014 climateprediction.net