climateprediction.net home page
HADAM3P's too much RAC weight?

HADAM3P's too much RAC weight?

Message boards : Number crunching : HADAM3P's too much RAC weight?
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · Next

AuthorMessage
old_user294426

Send message
Joined: 20 Feb 06
Posts: 158
Credit: 1,251,176
RAC: 0
Message 37988 - Posted: 12 Sep 2009, 9:40:58 UTC - in response to Message 37986.  
Last modified: 12 Sep 2009, 9:43:42 UTC

The formula for RAC is:
Old_RAC * 0.5^(1/7) + Today\'s_credit * (1-0.5^(1/7)) = New_RAC

That is a weighted average so that more recent credit has a higher weight. It also means that your RAC has a half-life of 7 days.

Thanks, 3rkko, for the actual formula.
Yields a hotchpotch of historical average and current credit.
Useless information, which ends up echoing the current credits..
Totally agree with Belfry. Scrap RAC completely.

Average for past 30 days would be a good alternative (or other period).
A simple formula could be:
(Old_Av * 29 + Today\'s_Credit) / 30
It does not remove the first of the previous 30 days correctly, but would be a good compromise.
And it would be understated at start and take some time to \"die away\" on cessation of crunching.
But this can be adjusted by No-of_days and Zero_days being remembered as data each day.

What I would really like to see, is an extra line on the Total Credits graph inclusive of Pending Credits.
So that it should show a reasonably straight line for a steady work output.
That would be really great to monitor crunching progress.
The gap between the two lines plainly displaying the lag caused by the Pending Credits.

Keith
ID: 37988 · Report as offensive     Reply Quote
Profile astroWX
Volunteer moderator

Send message
Joined: 5 Aug 04
Posts: 1496
Credit: 95,522,203
RAC: 0
Message 37989 - Posted: 12 Sep 2009, 18:59:41 UTC - in response to Message 37988.  

What I would really like to see, is an extra line on the Total Credits graph inclusive of Pending Credits.
So that it should show a reasonably straight line for a steady work output.
That would be really great to monitor crunching progress.
The gap between the two lines plainly displaying the lag caused by the Pending Credits.

Keith

That might be useful for other projects but meaningless, even erroneous, for CPDN. \'Pending credit\' is not used here. Values sometimes appear but are ignored. Credit is recomputed, in total, every day (assuming the script runs), based on total Trickles received and Trickle value for different Model-types. (That\'s why all CPDN credit goes with you if you leave a team.)

"We have met the enemy and he is us." -- Pogo
Greetings from coastal Washington state, the scenic US Pacific Northwest.
ID: 37989 · Report as offensive     Reply Quote
old_user294426

Send message
Joined: 20 Feb 06
Posts: 158
Credit: 1,251,176
RAC: 0
Message 37990 - Posted: 13 Sep 2009, 13:24:03 UTC - in response to Message 37989.  

What I would really like to see, is an extra line on the Total Credits graph inclusive of Pending Credits.
So that it should show a reasonably straight line for a steady work output.
That would be really great to monitor crunching progress.
The gap between the two lines plainly displaying the lag caused by the Pending Credits.

Keith

That might be useful for other projects but meaningless, even erroneous, for CPDN. \'Pending credit\' is not used here. Values sometimes appear but are ignored. Credit is recomputed, in total, every day (assuming the script runs), based on total Trickles received and Trickle value for different Model-types. (That\'s why all CPDN credit goes with you if you leave a team.)

Yes.
I was in SETI mode when I was thinking that, and should not really have put it onto my SPDN message.
I realised that some time after completing the message. You are quite right, of course.
Keith
ID: 37990 · Report as offensive     Reply Quote
Profile Purple Rabbit
Avatar

Send message
Joined: 1 Sep 04
Posts: 23
Credit: 5,067,166
RAC: 2,347
Message 38009 - Posted: 18 Sep 2009, 18:18:45 UTC

Can the problem be as simple as the HADAM3P\'s trickle more often (for a given computer) than other models? The weighted RAC algorithm may be preferentially responding to this.

Forgive me if I missed this in previous discussions. It\'s a long, rambling thread :-)
ID: 38009 · Report as offensive     Reply Quote
Virtual Boss*
Avatar

Send message
Joined: 14 May 08
Posts: 29
Credit: 776,852
RAC: 0
Message 38013 - Posted: 20 Sep 2009, 18:10:54 UTC

Quoted
The formula for RAC is:
Old_RAC * 0.5^(1/7) + Today\'s_credit * (1-0.5^(1/7)) = New_RAC


Is this really the formula used by CPDN?
And is it applied individually to each WU type, then added for a total?

If so then then an extra zero in the equation (as below in red)for HADAM3P could explain an increase in RAC by a factor of approx 3.7 times expected for this WU type.

Old_RAC * 0.5^(1/7) + Today\'s_credit * (1-0.05^(1/7)) = New_RAC

I know Milo has checked the formulas ... but it would be easy to miss if you are not specifically looking for it.

Note: This error would not affect the normal 7 day half-life, but would cause RAC to go from Normal to more than double in just 5 days.

(I may be wrong... but it seems to match the symptoms)

Bruce
ID: 38013 · Report as offensive     Reply Quote
Profile JIM

Send message
Joined: 31 Dec 07
Posts: 1152
Credit: 22,100,600
RAC: 2,970
Message 38048 - Posted: 1 Oct 2009, 4:56:36 UTC

Now that the Am3p’s are history (at least for the present) we can stop worrying about inflated RAC’s. Mine is sinking faster than a lead canoe. Oh well, back to an RAC of 600.

ID: 38048 · Report as offensive     Reply Quote
Profile mo.v
Volunteer moderator
Avatar

Send message
Joined: 29 Sep 04
Posts: 2363
Credit: 14,611,758
RAC: 0
Message 38052 - Posted: 5 Oct 2009, 1:34:47 UTC

A member saw CPDN\'s RAC graph and assumed the whole project was in terminal decline.
Cpdn news
ID: 38052 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 943
Credit: 34,322,262
RAC: 11,319
Message 38054 - Posted: 5 Oct 2009, 8:53:40 UTC - in response to Message 38052.  

A member saw CPDN\'s RAC graph and assumed the whole project was in terminal decline.
Now that HADAM3P tasks are no longer available, I\'ve switched my hosts back to CPDN Beta. Beta credits are transferred to the main project, but RAC isn\'t - so in a few days, my red and blue lines at BOINCstats will cross.

Another reason not to take any notice of RAC.
ID: 38054 · Report as offensive     Reply Quote
Belfry

Send message
Joined: 19 Apr 08
Posts: 179
Credit: 4,306,992
RAC: 0
Message 38056 - Posted: 5 Oct 2009, 17:21:30 UTC

I would like to know:

Is Virtual Boss\'s elegant explanation correct?

Does the Loch Ness Monster exist?

What lies beyond the edge of the Universe?
ID: 38056 · Report as offensive     Reply Quote
Profile Warped

Send message
Joined: 12 Sep 04
Posts: 34
Credit: 1,017,702
RAC: 0
Message 38439 - Posted: 2 Dec 2009, 11:27:03 UTC - in response to Message 38056.  

I would like to know:

Is Virtual Boss\'s elegant explanation correct?

Does the Loch Ness Monster exist?

What lies beyond the edge of the Universe?


Suggested answers:
I think so.
Yes, at least in theory.
The Mulitverse.
Warped
ID: 38439 · Report as offensive     Reply Quote
Profile Milo Thurston
Volunteer moderator
Volunteer developer

Send message
Joined: 2 Mar 06
Posts: 253
Credit: 363,646
RAC: 0
Message 38440 - Posted: 2 Dec 2009, 17:02:39 UTC - in response to Message 38013.  



I know Milo has checked the formulas ... but it would be easy to miss if you are not specifically looking for it.


I\'ve been looking again and I can\'t see any sign of any such formula in the code. It looks like RAC is the expavg_credit value from the database, which our stats script derives directly from the credit per timestep. Specifically, the RAC looks like the sum of credit_per_timestep for the models run.
ID: 38440 · Report as offensive     Reply Quote
3rkko

Send message
Joined: 12 Feb 08
Posts: 66
Credit: 4,877,652
RAC: 0
Message 38452 - Posted: 5 Dec 2009, 8:50:18 UTC
Last modified: 5 Dec 2009, 8:57:44 UTC

This Boinc wiki page explains how RAC is calculated for all Boinc projects.

The official formula is:
New_RAC = Old_RAC*weight + New_Credit*(1-weight)
where weight = e^(-ln(2)*T/HL)

But if you know math you can see that e^(-ln(2)*T/HL) = 0.5^(T/HL)
where HL = half-life = 604800 seconds (7*24*60*60), and T is time since last RAC calculation.

In the formula I posted earlier in this thread I changed the time from seconds to days which means HL=7 and, because CPDN calculates credit only once a day, I set T=1, which leads to weight=0.5^(1/7). You will obviously not find this modified formula in the code.
ID: 38452 · Report as offensive     Reply Quote
Profile Milo Thurston
Volunteer moderator
Volunteer developer

Send message
Joined: 2 Mar 06
Posts: 253
Credit: 363,646
RAC: 0
Message 38467 - Posted: 8 Dec 2009, 15:54:30 UTC

Our calculation is not the same as on the BOINC Wiki.
I\'ve been discussing this with some of the board moderators and I think that a change I\'ve made today to the stats scripts should fix things.
ID: 38467 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 943
Credit: 34,322,262
RAC: 11,319
Message 38468 - Posted: 8 Dec 2009, 15:55:51 UTC

Yes, mine looks better now.
ID: 38468 · Report as offensive     Reply Quote
transient

Send message
Joined: 3 Oct 06
Posts: 43
Credit: 8,017,057
RAC: 0
Message 38470 - Posted: 8 Dec 2009, 17:28:12 UTC - in response to Message 38467.  

Our calculation is not the same as on the BOINC Wiki.


It isn\'t? That surprises me a bit. Is this a general thing or CPDN-only? Does this mean we have CPDN-RAC, which can\'t rightly be added to the RAC of other projects? Because of the apples<>oranges thingy?
ID: 38470 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 943
Credit: 34,322,262
RAC: 11,319
Message 38474 - Posted: 8 Dec 2009, 17:44:19 UTC - in response to Message 38470.  

Our calculation is not the same as on the BOINC Wiki.
It isn\'t? That surprises me a bit. Is this a general thing or CPDN-only? Does this mean we have CPDN-RAC, which can\'t rightly be added to the RAC of other projects? Because of the apples<>oranges thingy?
I think that what Milo means is that CPDN can\'t use the exact calculation formula listed in the Wiki, because CPDN has to handle trickles and so on.

So they have their own bit of maths in the back end, which gets the same answer as the Wiki technique, but can\'t be found by a search for the Wiki text in the source code. But it\'s comparable to other projects, and can be used to generate an aggregate for competition purposes.

At least, it can be now......
ID: 38474 · Report as offensive     Reply Quote
old_user32530

Send message
Joined: 9 Dec 04
Posts: 1
Credit: 359,100
RAC: 0
Message 38480 - Posted: 8 Dec 2009, 20:20:37 UTC - in response to Message 38467.  

Our calculation is not the same as on the BOINC Wiki.
I\'ve been discussing this with some of the board moderators and I think that a change I\'ve made today to the stats scripts should fix things.


I think there\'s been some border damages : many teams have seen their credits disappeared

-91 954 777 for my team L\'Alliance Francophone. and others were impacted. Is there i\'ve understood in the wrong way?

++

lamoule

excuse my poor English but as i\'ve may have guessed, I\'m French.
ID: 38480 · Report as offensive     Reply Quote
Profile Leprechaun

Send message
Joined: 5 Aug 04
Posts: 6
Credit: 554,572
RAC: 0
Message 38481 - Posted: 8 Dec 2009, 20:45:42 UTC

This resembles with me. More than -65.000 Credits drawn off. Why? What does this deal with the correction of the RAC?
Wiki German Language, Wiki in deutscher Sprache
View
ID: 38481 · Report as offensive     Reply Quote
Profile mo.v
Volunteer moderator
Avatar

Send message
Joined: 29 Sep 04
Posts: 2363
Credit: 14,611,758
RAC: 0
Message 38482 - Posted: 8 Dec 2009, 21:15:03 UTC

Bonsoir, Guten Abend, Hi to everyone

The BoincStats total credit lists for CPDN are chaotic. Some big crunchers have lost millions of credits. I posted about this an hour ago in a private forum for CPDN moderators and programmers. Milo will see my post tomorrow but he\'s doing other things this evening.

When we know what the problem is we will post in the forum News thread at the top of the Number Crunching section.

This is clearly an accident and I am sure it will be corrected.
Cpdn news
ID: 38482 · Report as offensive     Reply Quote
Profile Milo Thurston
Volunteer moderator
Volunteer developer

Send message
Joined: 2 Mar 06
Posts: 253
Credit: 363,646
RAC: 0
Message 38488 - Posted: 8 Dec 2009, 22:43:59 UTC - in response to Message 38482.  



This is clearly an accident and I am sure it will be corrected.


Indeed so; apologies for these problems. I\'ve only made a small change to the stats script and would not expect it to have such an effect. It should be fixed next time the stats code is run but in any case investgating it will be my first priority tomorrow.
ID: 38488 · Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · Next

Message boards : Number crunching : HADAM3P's too much RAC weight?

©2024 climateprediction.net