climateprediction.net home page
Trouble uploading work
Trouble uploading work
log in

Advanced search

Message boards : Number crunching : Trouble uploading work

Author Message
Profile Steve Dodd
Send message
Joined: 28 Oct 11
Posts: 7
Credit: 2,536,716
RAC: 519
Message 43426 - Posted: 19 Nov 2011, 3:10:48 UTC

I've had a problem uploading work (non-trickle) since the outage.

11/18/2011 7:05:08 PM | climateprediction.net | Started upload of hadam3p_eu_677v_2006_1_007509982_2_13.zip
11/18/2011 7:05:10 PM | | Internet access OK - project servers may be temporarily down.
11/18/2011 7:05:10 PM | climateprediction.net | Temporarily failed upload of hadam3p_eu_677v_2006_1_007509982_2_13.zip: can't resolve hostname
11/18/2011 7:05:10 PM | climateprediction.net | Backing off 7 hr 44 min 3 sec on upload of hadam3p_eu_677v_2006_1_007509982_2_13.zip

I've rebooted numerous times. Anyone have any ideas? Do I need to reset my router to clear the cache?

MarkJ
Avatar
Send message
Joined: 28 Mar 09
Posts: 114
Credit: 9,825,980
RAC: 267
Message 43427 - Posted: 19 Nov 2011, 4:28:22 UTC

See this message in the News and Announcements thread.

Its always best to check there first to see if there is a known problem.
____________
BOINC blog

Profile Steve Dodd
Send message
Joined: 28 Oct 11
Posts: 7
Credit: 2,536,716
RAC: 519
Message 43430 - Posted: 19 Nov 2011, 10:52:21 UTC

Thank you, Mark. I only did a little cursory search before posting. Sorry. That does explain the situation.

Belfry
Send message
Joined: 19 Apr 08
Posts: 179
Credit: 4,306,992
RAC: 100
Message 43431 - Posted: 19 Nov 2011, 14:29:13 UTC
Last modified: 19 Nov 2011, 14:41:29 UTC

A workaround is possible for the adventurous. Note for Windows users: never use the included Notepad to edit BOINC files. Use a third party file editor compatible with Unix text. (I have used this utility in the past http://www.gena01.com/win32pad/).

1) Stop all BOINC processes. From BOINC Manager: advanced view / advanced / shut down connected client.
2) Backup the BOINC data directory in case bad things happen.
3) Open client_state.xml using a text editor and search for instances of _13.zip. The lines you want to edit will have <num_retries> a few lines below it. Replace:

<url>http://climateapps1.oerc.ox.ac.uk/cgi-bin/file_upload_handler</url>

with:

<url>http://uploader.oerc.ox.ac.uk/cgi-bin/file_upload_handler</url>

4) Save and restart BOINC.

edit: 4 comes after 3, not 5

Belfry
Send message
Joined: 19 Apr 08
Posts: 179
Credit: 4,306,992
RAC: 100
Message 43433 - Posted: 19 Nov 2011, 15:53:49 UTC

I should qualify that my workaround is only for 100% complete HADAM3P's that need to upload the final *_13.zip file. From the server status page one can see boinc1.coas.oregonstate is down, so intermediate uploads destined for it will fail, and no amount of client file editing will fix that.

Also if your computer becomes radioactive and molten, please don't hold me responsible.

Profile astroWX
Volunteer moderator
Send message
Joined: 5 Aug 04
Posts: 1426
Credit: 61,828,043
RAC: 11,478
Message 43434 - Posted: 19 Nov 2011, 17:55:41 UTC

Belfry,
That edit will upload restart dumps but drop them into the wrong pot. I have no idea how much extra work that will create for the staff but it's not something they need now.

What is the issue with Notepad? (I've used it to edit client_state ever since dumping Linux in favor of Windows, years, without incident.)

Cheers.
____________
"We have met the enemy and he is us." -- Pogo
Greetings from coastal Washington state, the scenic US Pacific Northwest.

Belfry
Send message
Joined: 19 Apr 08
Posts: 179
Credit: 4,306,992
RAC: 100
Message 43435 - Posted: 19 Nov 2011, 18:51:46 UTC

This morning all DNS records for the Oxford servers were resolving to the same address, but only uploader.oerc.ox.ac.uk wasn't producing a canonical name. Something has changed in the last couple hours and now climateapps1.oerc.ox.ac.uk is resolving to something else, so maybe someone fixed it.

Climateapps1.oucs.ox.ac.uk is the restart dump name so I don't think my edit will cause any problems. If I am wrong please delete my post and accept my apologies.

I guess if you don't mind the rigors of viewing xml without formatting, there's nothing dangerous in using Notepad.

Belfry
Send message
Joined: 19 Apr 08
Posts: 179
Credit: 4,306,992
RAC: 100
Message 43436 - Posted: 19 Nov 2011, 19:16:00 UTC
Last modified: 19 Nov 2011, 19:33:27 UTC

Um, seeing some spurious DNS results right now for climateapps1.oerc.ox.ac.uk:

OpenDNS (my usual) is pointing to 67.215.65.132, San Francisco

Frontier is pointing to 8.15.7.110, Los Angeles.

Comcast is unresolvable.


Edit: The other names are pointing as they should to Oxford through all of the above servers. Attempting an upload to a DNS poisoner will not be a problem because the client will not receive a a valid signature. Right?

Belfry
Send message
Joined: 19 Apr 08
Posts: 179
Credit: 4,306,992
RAC: 100
Message 43437 - Posted: 19 Nov 2011, 20:34:36 UTC
Last modified: 19 Nov 2011, 20:45:01 UTC

8.15.7.110 and 67.215.65.132 are both running web servers. No spam yet, just 404 not found.

Did CPDN relenquish climateapps1.oerc.ox.ac.uk? I can't risk a shooting war from my current location, but on Monday I could run Nessus (a vulnerability scan) against them.

Edit: ... in order to learn if they're trying to mimic a BOINC server.

Belfry
Send message
Joined: 19 Apr 08
Posts: 179
Credit: 4,306,992
RAC: 100
Message 43438 - Posted: 19 Nov 2011, 20:54:20 UTC
Last modified: 19 Nov 2011, 20:57:44 UTC

Looks like Microsoft has produced an XML version of Notepad: http://www.microsoft.com/download/en/details.aspx?id=7973

I'd still use my first link though, as no one can ever gage Microsoft's intentions.

Profile geophi
Volunteer moderator
Send message
Joined: 7 Aug 04
Posts: 1601
Credit: 28,752,041
RAC: 288
Message 43439 - Posted: 19 Nov 2011, 23:50:35 UTC

Belfry,

We think your guidance will work alright with the location of the files. Milo, former cpdn admin, says that the methods of accessing completed result files doesn't depend on a specific server name. It should work as long as the upload server notifies the results database that the result files are there.

Belfry
Send message
Joined: 19 Apr 08
Posts: 179
Credit: 4,306,992
RAC: 100
Message 43440 - Posted: 20 Nov 2011, 0:57:01 UTC
Last modified: 20 Nov 2011, 0:58:45 UTC

I'm glad to hear that geophi. In my zeal to offer a solution I assumed if a server could accept an upload then it would be authorized to insert into the database. But things aren't always configured ideally; my solution definitely could have left a big headache for someone on Monday. Next time I will pm an admin before posting alternate methods for accessing the servers. Good to know Milo is still offering CPDN advice.

Profile astroWX
Volunteer moderator
Send message
Joined: 5 Aug 04
Posts: 1426
Credit: 61,828,043
RAC: 11,478
Message 43441 - Posted: 20 Nov 2011, 5:03:05 UTC

Belfry,
It seems I was overly cautious. Mea culpa! (Milo to the rescue again!) So, I used your suggestion and cleared my field of grounded birds.

____________
"We have met the enemy and he is us." -- Pogo
Greetings from coastal Washington state, the scenic US Pacific Northwest.

Profile JIM
Send message
Joined: 31 Dec 07
Posts: 902
Credit: 8,511,072
RAC: 11,708
Message 43442 - Posted: 20 Nov 2011, 6:04:32 UTC

I hate to rain on the parade, but, I made the edit to the client_state file and it didn?t work. It seems to have allowed me to upload the 12.zip file that was stuck in the transfer tab, but, not the two 13.zip files that are still stuck there.

____________

Profile JIM
Send message
Joined: 31 Dec 07
Posts: 902
Credit: 8,511,072
RAC: 11,708
Message 43443 - Posted: 20 Nov 2011, 6:46:56 UTC

Please disregard the above post. It seems that I edited the wrong place. Have now edited the right place and the 13.zip files have uploaded successfully.

Belfry
Send message
Joined: 19 Apr 08
Posts: 179
Credit: 4,306,992
RAC: 100
Message 43446 - Posted: 21 Nov 2011, 13:26:09 UTC

Jim, glad to hear it worked out, but ditch Notepad for xml editing (I'm guessing that's what led to the error).

This morning all the DNS issues appear to be fixed. Comcast, Frontier, OpenDNS, and Google DNS servers are resolving climateapps1.oerc.ox.ac.uk correctly. This should obiviate the need for any more climate_state.xml editing :)

Les Bayliss
Volunteer moderator
Send message
Joined: 5 Sep 04
Posts: 6234
Credit: 14,607,204
RAC: 543
Message 43450 - Posted: 21 Nov 2011, 21:51:36 UTC

Would everyone please read my post here in the Windows section.

Especially the last line. :)


____________
Backups: Here

Belfry
Send message
Joined: 19 Apr 08
Posts: 179
Credit: 4,306,992
RAC: 100
Message 43451 - Posted: 21 Nov 2011, 22:27:47 UTC

Did something go pear/cabbage/mobius shaped?

Les Bayliss
Volunteer moderator
Send message
Joined: 5 Sep 04
Posts: 6234
Credit: 14,607,204
RAC: 543
Message 43452 - Posted: 21 Nov 2011, 23:57:08 UTC

Different data is expected to be on certain servers.
This can be worked around, but may in the future prove to be a problem, if it's then accessed differently.
In which case, it may just be lost and allowed to stay that way. Which, in effect, will mean that the crunching was in vain.


____________
Backups: Here

Belfry
Send message
Joined: 19 Apr 08
Posts: 179
Credit: 4,306,992
RAC: 100
Message 43453 - Posted: 22 Nov 2011, 0:04:38 UTC

Sorry.

Message boards : Number crunching : Trouble uploading work


Main page · Your account · Message boards


Copyright © 2016 climateprediction.net