climateprediction.net home page
Uploads not working

Uploads not working

Message boards : Number crunching : Uploads not working
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4

AuthorMessage
Profile Byron Leigh Hatch @ team Carl ...
Avatar

Send message
Joined: 17 Aug 04
Posts: 289
Credit: 44,103,664
RAC: 0
Message 44927 - Posted: 29 Sep 2012, 21:56:49 UTC

Just reporting some good news. zip files seem to be uploading.
ID: 44927 · Report as offensive     Reply Quote
Profile [B@H] Ray
Avatar

Send message
Joined: 19 Aug 05
Posts: 104
Credit: 1,866,495
RAC: 0
Message 44928 - Posted: 30 Sep 2012, 1:06:16 UTC

For my units the PNW units are uploading good, the EU units have been just setting here. One system is working on it's last model, hope there is new work this week.
Keep on crunching Pizza@Home
ID: 44928 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 44929 - Posted: 30 Sep 2012, 4:10:10 UTC - in response to Message 44928.  

PNW goes directly to Uni of Oregon, USA, so don't count.

New work won't even be considered until ALL of the server problems are sorted, which may be another week yet.

Michaelmas Term starts in a weeks time. or thereabouts, so Long Vacation will finish in a few days, and all of the IT people who scarpered as soon as it started should be back soon, and dealing with problems in their various areas.


Backups: Here
ID: 44929 · Report as offensive     Reply Quote
Profile [AF>Le_Pommier] Jerome_C2005

Send message
Joined: 21 Oct 10
Posts: 53
Credit: 2,101,753
RAC: 3,985
Message 44931 - Posted: 30 Sep 2012, 10:39:14 UTC
Last modified: 30 Sep 2012, 10:52:44 UTC

Just when I was about to write "it's stuck with me too" it started to upload again :)

edit : well then it gets stuck again, then it restarts again... so I guess we'll have to wait for the return of the Jedi...
ID: 44931 · Report as offensive     Reply Quote
Profile JIM

Send message
Joined: 31 Dec 07
Posts: 1152
Credit: 22,053,321
RAC: 4,417
Message 44934 - Posted: 30 Sep 2012, 16:17:31 UTC

I don�t believe that there is much that you can do to speed this up. The only real solution is to wait for the server problems to be fixed. You might suspend network activity so that the stuck zip file doesn�t keep trying to upload. If you are running other types of WU�s or other Boinc projects you can reenable network activity about once a day to let other types to upload and then resuspend.

ID: 44934 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 44935 - Posted: 30 Sep 2012, 20:11:10 UTC

To: J. Patrick Malone

I've hidden your post to stop spammers from getting your email address.

As for mailing results back to the project, this isn't how BOINC projects work.
You'll just have to wait patiently like all of us.
If you read back through this thread, you'll find one of my earlier posts, where I listed the only steps that can be taken.


Backups: Here
ID: 44935 · Report as offensive     Reply Quote
Profile Patrick

Send message
Joined: 8 Sep 10
Posts: 6
Credit: 1,475,984
RAC: 0
Message 44995 - Posted: 2 Oct 2012, 19:15:49 UTC

FWIW, my long queue of EU model uploads has decreased and some of the uploads are now getting through.
ID: 44995 · Report as offensive     Reply Quote
pioneer1

Send message
Joined: 16 May 07
Posts: 10
Credit: 2,368,487
RAC: 0
Message 44999 - Posted: 3 Oct 2012, 9:09:11 UTC

Here we go again :(

03/10/2012 12:01:08 | climateprediction.net | [error] Error reported by file upload server: can't write file /storage/incoming/uploader//hadam3p_eu_2r82_1972_1_008189180_0_7.zip: No space left on server
03/10/2012 12:01:08 | climateprediction.net | Temporarily failed upload of hadam3p_eu_2r82_1972_1_008189180_0_7.zip: transient upload error
03/10/2012 12:01:08 | climateprediction.net | Backing off 9 hr 1 min 29 sec on upload of hadam3p_eu_2r82_1972_1_008189180_0_7.zip

ID: 44999 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 45000 - Posted: 3 Oct 2012, 9:36:56 UTC - in response to Message 44999.  

It's more a matter of "still" rather than "again".

Have you read the News thread?
It could be next week before the bulk of the uploads get through.


Backups: Here
ID: 45000 · Report as offensive     Reply Quote
pioneer1

Send message
Joined: 16 May 07
Posts: 10
Credit: 2,368,487
RAC: 0
Message 45001 - Posted: 3 Oct 2012, 9:49:47 UTC - in response to Message 45000.  


But all my uploads & of others went through so I thought the problem(s) were fixed that's why the "again". Never mind.

And, of course, I've read both the News and Announcements plus the other threads (Uploads not working, Server out of disk space,...) not to mention my own topic "Permanent HTTP Error".
ID: 45001 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4309
Credit: 16,356,117
RAC: 5,224
Message 45002 - Posted: 3 Oct 2012, 10:12:29 UTC - in response to Message 45001.  

If everyone were to pick a day of the week and a time to enable internet activity, it would reduce the load on the server after outages. Even if quite a few people chose the same day, it would reduce the hammering when first back on line. Perhaps the information where people sign up should suggest this? I know it is nice to look at stats and see how you are doing but I am sure most of us could cope with getting our fix once a week rather than several times a day?...........
ID: 45002 · Report as offensive     Reply Quote
nedsram-cdl

Send message
Joined: 14 Apr 05
Posts: 31
Credit: 16,491,691
RAC: 0
Message 45003 - Posted: 3 Oct 2012, 12:16:49 UTC - in response to Message 45002.  

Possibly, but the point is that this issue has been present for some time. I currently have 20 eu zip files unable to upload.

Hopefully we will be told when this has been resolved, on the news thread - although they did say "next week" about a week ago...
Brian
ID: 45003 · Report as offensive     Reply Quote
Profile Byron Leigh Hatch @ team Carl ...
Avatar

Send message
Joined: 17 Aug 04
Posts: 289
Credit: 44,103,664
RAC: 0
Message 45005 - Posted: 3 Oct 2012, 12:38:21 UTC

Just a few minuets ago while reading this thread, my 24 eu zip files are starting to upload. Yay!
ID: 45005 · Report as offensive     Reply Quote
Profile Byron Leigh Hatch @ team Carl ...
Avatar

Send message
Joined: 17 Aug 04
Posts: 289
Credit: 44,103,664
RAC: 0
Message 45007 - Posted: 3 Oct 2012, 12:58:45 UTC

Just reporting some good news. all my 24 eu zip files have now uploaded at 175 kbps. Well done to the team @ Oxford! and thank Jonathan Miller CPDN SysAdmin
ID: 45007 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 45011 - Posted: 3 Oct 2012, 22:33:26 UTC
Last modified: 4 Oct 2012, 1:45:13 UTC

I managed to get my remaining 16 EUs to upload 'overnight'.
So it's not fixed yet, just "getting there".

Data is still being moved off a couple of servers to storage, but more is coming in just as fast.
I've been watching this in the messages on one of my computers, as 16 files slowly uploaded.

According to the Status page yesterday, there were over 135,000 tasks running, and now it says 127,477, so it's coming down.

Just thinking out loud, if only a quarter of those "running" were due to pending uploads, and each one only had a quarter of their files waiting, that's about 90,000 zips fighting each other for disk space.

It must be somewhat like a person running an ultra-marathon through vast swarms of stampeding elephants, rinos and wildebeests, while juggling a dozen sharp knives.

There's been more hardware and software failures since the weekend, but Jonathan and Andy have their eyes on things.
Backups: Here
ID: 45011 · Report as offensive     Reply Quote
Profile Byron Leigh Hatch @ team Carl ...
Avatar

Send message
Joined: 17 Aug 04
Posts: 289
Credit: 44,103,664
RAC: 0
Message 45013 - Posted: 3 Oct 2012, 23:25:33 UTC - in response to Message 45011.  

Les thank you for your post, like you say we're not out of woods yet,
there could be more bumps in the road ahead.
best wishes to the team @ Oxford!
and thank you again to Andy and Jonathan, CPDN SysAdmins for a job well done!

Byron
ID: 45013 · Report as offensive     Reply Quote
JimMcCarthy_StellarSolns
Avatar

Send message
Joined: 3 Sep 08
Posts: 23
Credit: 41,989,607
RAC: 2,734
Message 45014 - Posted: 4 Oct 2012, 0:51:08 UTC
Last modified: 4 Oct 2012, 0:51:35 UTC

Yes, thank you Les and others who work tirelessly to keep everything up and running.

I just wanted to add an 'FYI' that the uploader server issues again seem to be interfering with attempts to download work from the 'reference site' onto a windows machine which I've just set up for CPDN number crunching. Searching the forum archives, my symptoms are the same as those experienced and explained in message 44708 (http://climateapps2.oerc.ox.ac.uk/cpdnboinc/forum_thread.php?id=7442&nowrap=true#44708). Per the advice given there, I'll just remain patient while everything is returned to normal.

Thanks again,

-- Jim
ID: 45014 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4309
Credit: 16,356,117
RAC: 5,224
Message 45018 - Posted: 4 Oct 2012, 6:32:14 UTC - in response to Message 45011.  

According to the Status page yesterday, there were over 135,000 tasks running,


My main machine has two tasks running but four more in the queue. As these are listed as being in progress when i look at the computer's page, I presume that those 135,000 include those queued on machines but not yet started? A trivial point, I know given the problems with hardware and software etc but it piqued my curiosity and I wondered how many tasks are actually, "in progress" My other linux machine doesn't have any in the queue at the moment so my own average would be half of those listed. I will leave it to someone else with more machines to work out something with more statistical validity!
ID: 45018 · Report as offensive     Reply Quote
Eirik Redd

Send message
Joined: 31 Aug 04
Posts: 391
Credit: 219,888,554
RAC: 1,481,373
Message 45019 - Posted: 4 Oct 2012, 7:40:28 UTC - in response to Message 45018.  

According to the Status page yesterday, there were over 135,000 tasks running,


My main machine has two tasks running but four more in the queue. As these are listed as being in progress when i look at the computer's page, I presume that those 135,000 include those queued on machines but not yet started? A trivial point, I know given the problems with hardware and software etc but it piqued my curiosity and I wondered how many tasks are actually, "in progress" My other linux machine doesn't have any in the queue at the moment so my own average would be half of those listed. I will leave it to someone else with more machines to work out something with more statistical validity!



The 135,000 number is inaccurate for at least two reasons. First, as you noted, some indeterminate number of those are waiting "Ready to start" on somebody's host(s). Another indeterminate number have downloaded to hosts that will never finish the task(s).

I have 6 machines running 24/7 - 3 of them are somewhat fast. There's very little in the queue "Ready to start" but a whole lot pending upload. I am only letting each machine go online once a day to update stats and trickle up and download my preferred wu's from other projects. I let only one of them at a time stay online for a day (intil its upload queue clears, then I leave it online) -- that means one is network enabled each day until its upload queue clears - it will take a few days to finish the 80+ uploads (per fast host) that are still pending. My slower hosts are all caught up and online for new work. Running on a slowish DSL. Expect uploads to catch up within a couple of days. Getting downloads from time to time (probably old wu's that timed out and got resubmitted automatically)

Figuring an overall reduction factor to adjust the supposed 130,000 tasks "out there" would be real difficult. I have prefs set to start downloading when any task is withing 28 hours of completing. So there's less than 25% ratio "Ready to start" versus "Running" here. I still have my 3 faster hosts that have more tasks "uploading" than they have "running" It will be a while - and I'm not going to push my uploads because there's lots of other people with worse network than what I have and I'm not going to do anything to overload the fragile servers.
ID: 45019 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 45021 - Posted: 4 Oct 2012, 8:08:22 UTC
Last modified: 4 Oct 2012, 8:09:17 UTC

OK, my mistake. The label is actually Tasks in progress

This is an abbreviation for: I've sent this number of work units to client computers, and they aren't yet on my work list as being completed or failed. Therefore they're still out there somewhere..

As for new work that's occasionally being received, that's due to the resubmission script being fired up to slowly produce new data sets in the sequence of that past work that has been returned intact.
Backups: Here
ID: 45021 · Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4

Message boards : Number crunching : Uploads not working

©2024 climateprediction.net