climateprediction.net home page

The world's largest climate forecasting experiment for the 21st century.

Trouble uploading work


Advanced search

Message boards : Number crunching : Trouble uploading work

AuthorMessage
Profile Steve Dodd
Send message
Joined: Oct 28 11
Posts: 3
Credit: 1,527,419
RAC: 556
Message 43426 - Posted 19 Nov 2011 3:10:48 UTC

    I've had a problem uploading work (non-trickle) since the outage.

    11/18/2011 7:05:08 PM | climateprediction.net | Started upload of hadam3p_eu_677v_2006_1_007509982_2_13.zip
    11/18/2011 7:05:10 PM | | Internet access OK - project servers may be temporarily down.
    11/18/2011 7:05:10 PM | climateprediction.net | Temporarily failed upload of hadam3p_eu_677v_2006_1_007509982_2_13.zip: can't resolve hostname
    11/18/2011 7:05:10 PM | climateprediction.net | Backing off 7 hr 44 min 3 sec on upload of hadam3p_eu_677v_2006_1_007509982_2_13.zip

    I've rebooted numerous times. Anyone have any ideas? Do I need to reset my router to clear the cache?

    MarkJ
    Avatar
    Send message
    Joined: Mar 28 09
    Posts: 101
    Credit: 4,001,654
    RAC: 29,360
    Message 43427 - Posted 19 Nov 2011 4:28:22 UTC

      See this message in the News and Announcements thread.

      Its always best to check there first to see if there is a known problem.
      ____________
      BOINC blog

      Profile Steve Dodd
      Send message
      Joined: Oct 28 11
      Posts: 3
      Credit: 1,527,419
      RAC: 556
      Message 43430 - Posted 19 Nov 2011 10:52:21 UTC

        Thank you, Mark. I only did a little cursory search before posting. Sorry. That does explain the situation.

        Belfry
        Send message
        Joined: Apr 19 08
        Posts: 170
        Credit: 3,311,495
        RAC: 1,298
        Message 43431 - Posted 19 Nov 2011 14:29:13 UTC

          Last modified: 19 Nov 2011 14:41:29 UTC

          A workaround is possible for the adventurous. Note for Windows users: never use the included Notepad to edit BOINC files. Use a third party file editor compatible with Unix text. (I have used this utility in the past http://www.gena01.com/win32pad/).

          1) Stop all BOINC processes. From BOINC Manager: advanced view / advanced / shut down connected client.
          2) Backup the BOINC data directory in case bad things happen.
          3) Open client_state.xml using a text editor and search for instances of _13.zip. The lines you want to edit will have <num_retries> a few lines below it. Replace:

          <url>http://climateapps1.oerc.ox.ac.uk/cgi-bin/file_upload_handler</url>

          with:

          <url>http://uploader.oerc.ox.ac.uk/cgi-bin/file_upload_handler</url>

          4) Save and restart BOINC.

          edit: 4 comes after 3, not 5

          Belfry
          Send message
          Joined: Apr 19 08
          Posts: 170
          Credit: 3,311,495
          RAC: 1,298
          Message 43433 - Posted 19 Nov 2011 15:53:49 UTC

            I should qualify that my workaround is only for 100% complete HADAM3P's that need to upload the final *_13.zip file. From the server status page one can see boinc1.coas.oregonstate is down, so intermediate uploads destined for it will fail, and no amount of client file editing will fix that.

            Also if your computer becomes radioactive and molten, please don't hold me responsible.

            Profile astroWX
            Forum moderator
            Send message
            Joined: Aug 5 04
            Posts: 1250
            Credit: 35,045,397
            RAC: 23,438
            Message 43434 - Posted 19 Nov 2011 17:55:41 UTC

              Belfry,
              That edit will upload restart dumps but drop them into the wrong pot. I have no idea how much extra work that will create for the staff but it's not something they need now.

              What is the issue with Notepad? (I've used it to edit client_state ever since dumping Linux in favor of Windows, years, without incident.)

              Cheers.
              ____________
              "We have met the enemy and he is us." -- Pogo
              Greetings from coastal Washington state, the scenic US Pacific Northwest.

              Belfry
              Send message
              Joined: Apr 19 08
              Posts: 170
              Credit: 3,311,495
              RAC: 1,298
              Message 43435 - Posted 19 Nov 2011 18:51:46 UTC

                This morning all DNS records for the Oxford servers were resolving to the same address, but only uploader.oerc.ox.ac.uk wasn't producing a canonical name. Something has changed in the last couple hours and now climateapps1.oerc.ox.ac.uk is resolving to something else, so maybe someone fixed it.

                Climateapps1.oucs.ox.ac.uk is the restart dump name so I don't think my edit will cause any problems. If I am wrong please delete my post and accept my apologies.

                I guess if you don't mind the rigors of viewing xml without formatting, there's nothing dangerous in using Notepad.

                Belfry
                Send message
                Joined: Apr 19 08
                Posts: 170
                Credit: 3,311,495
                RAC: 1,298
                Message 43436 - Posted 19 Nov 2011 19:16:00 UTC

                  Last modified: 19 Nov 2011 19:33:27 UTC

                  Um, seeing some spurious DNS results right now for climateapps1.oerc.ox.ac.uk:

                  OpenDNS (my usual) is pointing to 67.215.65.132, San Francisco

                  Frontier is pointing to 8.15.7.110, Los Angeles.

                  Comcast is unresolvable.


                  Edit: The other names are pointing as they should to Oxford through all of the above servers. Attempting an upload to a DNS poisoner will not be a problem because the client will not receive a a valid signature. Right?

                  Belfry
                  Send message
                  Joined: Apr 19 08
                  Posts: 170
                  Credit: 3,311,495
                  RAC: 1,298
                  Message 43437 - Posted 19 Nov 2011 20:34:36 UTC

                    Last modified: 19 Nov 2011 20:45:01 UTC

                    8.15.7.110 and 67.215.65.132 are both running web servers. No spam yet, just 404 not found.

                    Did CPDN relenquish climateapps1.oerc.ox.ac.uk? I can't risk a shooting war from my current location, but on Monday I could run Nessus (a vulnerability scan) against them.

                    Edit: ... in order to learn if they're trying to mimic a BOINC server.

                    Belfry
                    Send message
                    Joined: Apr 19 08
                    Posts: 170
                    Credit: 3,311,495
                    RAC: 1,298
                    Message 43438 - Posted 19 Nov 2011 20:54:20 UTC

                      Last modified: 19 Nov 2011 20:57:44 UTC

                      Looks like Microsoft has produced an XML version of Notepad: http://www.microsoft.com/download/en/details.aspx?id=7973

                      I'd still use my first link though, as no one can ever gage Microsoft's intentions.

                      Profile geophi
                      Forum moderator
                      Send message
                      Joined: Aug 7 04
                      Posts: 1447
                      Credit: 22,205,470
                      RAC: 11,289
                      Message 43439 - Posted 19 Nov 2011 23:50:35 UTC

                        Belfry,

                        We think your guidance will work alright with the location of the files. Milo, former cpdn admin, says that the methods of accessing completed result files doesn't depend on a specific server name. It should work as long as the upload server notifies the results database that the result files are there.

                        Belfry
                        Send message
                        Joined: Apr 19 08
                        Posts: 170
                        Credit: 3,311,495
                        RAC: 1,298
                        Message 43440 - Posted 20 Nov 2011 0:57:01 UTC

                          Last modified: 20 Nov 2011 0:58:45 UTC

                          I'm glad to hear that geophi. In my zeal to offer a solution I assumed if a server could accept an upload then it would be authorized to insert into the database. But things aren't always configured ideally; my solution definitely could have left a big headache for someone on Monday. Next time I will pm an admin before posting alternate methods for accessing the servers. Good to know Milo is still offering CPDN advice.

                          Profile astroWX
                          Forum moderator
                          Send message
                          Joined: Aug 5 04
                          Posts: 1250
                          Credit: 35,045,397
                          RAC: 23,438
                          Message 43441 - Posted 20 Nov 2011 5:03:05 UTC

                            Belfry,
                            It seems I was overly cautious. Mea culpa! (Milo to the rescue again!) So, I used your suggestion and cleared my field of grounded birds.

                            ____________
                            "We have met the enemy and he is us." -- Pogo
                            Greetings from coastal Washington state, the scenic US Pacific Northwest.

                            Profile JIM
                            Send message
                            Joined: Dec 31 07
                            Posts: 609
                            Credit: 3,353,165
                            RAC: 4,862
                            Message 43442 - Posted 20 Nov 2011 6:04:32 UTC

                              I hate to rain on the parade, but, I made the edit to the client_state file and it didn?t work. It seems to have allowed me to upload the 12.zip file that was stuck in the transfer tab, but, not the two 13.zip files that are still stuck there.

                              ____________

                              Profile JIM
                              Send message
                              Joined: Dec 31 07
                              Posts: 609
                              Credit: 3,353,165
                              RAC: 4,862
                              Message 43443 - Posted 20 Nov 2011 6:46:56 UTC

                                Please disregard the above post. It seems that I edited the wrong place. Have now edited the right place and the 13.zip files have uploaded successfully.

                                Belfry
                                Send message
                                Joined: Apr 19 08
                                Posts: 170
                                Credit: 3,311,495
                                RAC: 1,298
                                Message 43446 - Posted 21 Nov 2011 13:26:09 UTC

                                  Jim, glad to hear it worked out, but ditch Notepad for xml editing (I'm guessing that's what led to the error).

                                  This morning all the DNS issues appear to be fixed. Comcast, Frontier, OpenDNS, and Google DNS servers are resolving climateapps1.oerc.ox.ac.uk correctly. This should obiviate the need for any more climate_state.xml editing :)

                                  Les Bayliss
                                  Forum moderator
                                  Send message
                                  Joined: Sep 5 04
                                  Posts: 5131
                                  Credit: 8,474,017
                                  RAC: 6,823
                                  Message 43450 - Posted 21 Nov 2011 21:51:36 UTC

                                    Would everyone please read my post here in the Windows section.

                                    Especially the last line. :)


                                    ____________
                                    Backups: Here

                                    Belfry
                                    Send message
                                    Joined: Apr 19 08
                                    Posts: 170
                                    Credit: 3,311,495
                                    RAC: 1,298
                                    Message 43451 - Posted 21 Nov 2011 22:27:47 UTC

                                      Did something go pear/cabbage/mobius shaped?

                                      Les Bayliss
                                      Forum moderator
                                      Send message
                                      Joined: Sep 5 04
                                      Posts: 5131
                                      Credit: 8,474,017
                                      RAC: 6,823
                                      Message 43452 - Posted 21 Nov 2011 23:57:08 UTC

                                        Different data is expected to be on certain servers.
                                        This can be worked around, but may in the future prove to be a problem, if it's then accessed differently.
                                        In which case, it may just be lost and allowed to stay that way. Which, in effect, will mean that the crunching was in vain.


                                        ____________
                                        Backups: Here

                                        Belfry
                                        Send message
                                        Joined: Apr 19 08
                                        Posts: 170
                                        Credit: 3,311,495
                                        RAC: 1,298
                                        Message 43453 - Posted 22 Nov 2011 0:04:38 UTC

                                          Sorry.

                                          Post to thread

                                          Message boards : Number crunching : Trouble uploading work




                                          Copyright © 2002-2014 climateprediction.net