set of weights summing to 1. If one is modified, how do you balance weights back to 1
Author
Hi,
Suppose you have a set of weights, P1, P2, ... Pn. They all add up to 1.
One of the weights is the modified, say P1 is now P1*m. So now they do not add up to 1.
Is there a formula/algorithm that will modify all the other weights, while retaining the new value for P1, so that they all add up to 1 again?
Thanks in advance.
If you want to retain their sum, you will have to change the other weights, but the point is: how do you want them to change? You can choose to just change one weight other than the one you've altered or either change all of them.
Suppose you have weights (0.5, 0.2, 0.3) and change it to (0.7, 0.2, 0.3). Both (0.7, 0.1, 0.2) and (0.7, 0.0, 0.3) are solutions to you problem.
Do you have any restrictions on that?
Does the new weight value have to be absolute? Since they have to sum up 1, can't you just do this:
Suppose you have weights (0.5, 0.2, 0.3) and change it to (0.7, 0.2, 0.3). Both (0.7, 0.1, 0.2) and (0.7, 0.0, 0.3) are solutions to you problem.
Do you have any restrictions on that?
Does the new weight value have to be absolute? Since they have to sum up 1, can't you just do this:
(0.5, 0.5) -> (1.5, 0.5) -> (0.75, 0.25)original -> change weight -> normalize
how do you want them to change? that is the real question.
if
P1 + x => P1
then all other elements could be treated as
Pi - x/(n-1) => Pi
or you could treat it like a vector. after any change take
sqrt ( sum( Pi * Pi ) ) => magnitude
Pi / magnitude => Pi
(this doesnt keep the new P1, but it keeps the relative ratios better)
edit: thinking the same thing fcoelho, but i'm too slow.
if
P1 + x => P1
then all other elements could be treated as
Pi - x/(n-1) => Pi
or you could treat it like a vector. after any change take
sqrt ( sum( Pi * Pi ) ) => magnitude
Pi / magnitude => Pi
(this doesnt keep the new P1, but it keeps the relative ratios better)
edit: thinking the same thing fcoelho, but i'm too slow.
I'm assuming a couple of things here. First of all, I'm assuming the new value of P1 is not going to be outside the 0...1 range itself. Secondly, I assume you want to adjust the values so that they're still in the same relative ratio to each other, but you don't want to touch P1 once you've manually adjusted it.
So basically, after you set P1 to some new value, you want P2 + P3 + ... + Pn = 1 - P1
So what you do is, you add up P2 + P3 + ... + Pn and work out what it currently sums to, say Scurr. So the forumla is Scurr * factor = 1 - P1 and therefore factor = (1 - P1) / Scurr.
So you just multiple each element by (1 - P1) / Scurr.
Taking a simple example, say you've got 0.5, 0.2 and 0.3 initially. You adjust 0.5 down to 0.4. That means you've now got 0.4, 0.2, 0.3 which doesn't add up to one. You simply calculate Scurr, which is 0.2 + 0.3 = 0.5. Using the above formula, factor = (1 - P1) / Scurr = (1 - 0.4) / 0.5 = 1.2
Just multiply the remaining numbers by 1.2 and you end up with: 0.4 + 0.24 + 0.36 = 1
So basically, after you set P1 to some new value, you want P2 + P3 + ... + Pn = 1 - P1
So what you do is, you add up P2 + P3 + ... + Pn and work out what it currently sums to, say Scurr. So the forumla is Scurr * factor = 1 - P1 and therefore factor = (1 - P1) / Scurr.
So you just multiple each element by (1 - P1) / Scurr.
Taking a simple example, say you've got 0.5, 0.2 and 0.3 initially. You adjust 0.5 down to 0.4. That means you've now got 0.4, 0.2, 0.3 which doesn't add up to one. You simply calculate Scurr, which is 0.2 + 0.3 = 0.5. Using the above formula, factor = (1 - P1) / Scurr = (1 - 0.4) / 0.5 = 1.2
Just multiply the remaining numbers by 1.2 and you end up with: 0.4 + 0.24 + 0.36 = 1
Quote:Original post by CodekaJust a note. You're probably going to have the old value of P1 lying around so instead of summing up all values P2->Pn just use ( 1 - old_P1 ). Which makes factor = ( 1 - P1 ) / ( 1 - old_p1 ).
So what you do is, you add up P2 + P3 + ... + Pn and work out what it currently sums to, say Scurr. So the forumla is Scurr * factor = 1 - P1 and therefore factor = (1 - P1) / Scurr.
Quote:Original post by Peter5897Quote:Original post by CodekaJust a note. You're probably going to have the old value of P1 lying around so instead of summing up all values P2->Pn just use ( 1 - old_P1 ). Which makes factor = ( 1 - P1 ) / ( 1 - old_p1 ).
So what you do is, you add up P2 + P3 + ... + Pn and work out what it currently sums to, say Scurr. So the forumla is Scurr * factor = 1 - P1 and therefore factor = (1 - P1) / Scurr.
Good point!
In principle the answer depends on what that weighted average is being used for. However, most of the situations where I have used weighted averages in the past would be such that the correct solution is to renormalize (divide each weight by the sum of all the weights), like fcoelho suggested.
I assume you're using a neural net (or something similar) and updating weights using gradient descent (or something like it). If this is the case, then what you have, essentially, is the minimization problem,
min f(x) s.t. n^T x = 1
where x is a vector in R^N, and n is a vector of ones also in R^N. (n^T x = 1 represents the must-add-up-to-one constraint).
The algorithm to use is called projected gradient descent. Basically, compute the gradient of f with respect to x (you get a vector in R^n), and then project this vector onto the plane defined by n^T x = 1.
The projection is done,
grad_proj = grad<br><br>where we can call P = the "projection matrix."<br><br>But n^T n = N, and n n^T is just an NxN matrix full of ones, so P is just a matrix that is 1 - 1/N on the diagonal and -1/N everywhere else. This matrix has a very special structure, so obviously you don't need to actually store it at any time; just implement premultiplication of a vector by this matrix as some very simple nested loops.
min f(x) s.t. n^T x = 1
where x is a vector in R^N, and n is a vector of ones also in R^N. (n^T x = 1 represents the must-add-up-to-one constraint).
The algorithm to use is called projected gradient descent. Basically, compute the gradient of f with respect to x (you get a vector in R^n), and then project this vector onto the plane defined by n^T x = 1.
The projection is done,
grad_proj = grad<br><br>where we can call P = the "projection matrix."<br><br>But n^T n = N, and n n^T is just an NxN matrix full of ones, so P is just a matrix that is 1 - 1/N on the diagonal and -1/N everywhere else. This matrix has a very special structure, so obviously you don't need to actually store it at any time; just implement premultiplication of a vector by this matrix as some very simple nested loops.
Author
Codeka: Those assumptions are exactly correct. Sorry for not mentioning those. Obviously renormalization is out of the question because then the new weight value is modified as well.
Thanks guys, the Scurr stuff works great. I've run into a snag when a weight gets to 0, it never goes back up again. I just put in a temporary fix which is if a weight is 0 then I change the value to 0.0000001 or something, seems to work fine.
Emergent: No neural nets at play here. The weights are not being adjusted according to desired outputs or anything like that. It's completely manual. When a user prefers some 'thing' more than another 'thing', s/he increases the weight of that thing, so I have to adjust all other weights accordingly.
Thanks again.
Thanks guys, the Scurr stuff works great. I've run into a snag when a weight gets to 0, it never goes back up again. I just put in a temporary fix which is if a weight is 0 then I change the value to 0.0000001 or something, seems to work fine.
Emergent: No neural nets at play here. The weights are not being adjusted according to desired outputs or anything like that. It's completely manual. When a user prefers some 'thing' more than another 'thing', s/he increases the weight of that thing, so I have to adjust all other weights accordingly.
Thanks again.
Quote:Original post by IFooBar
I've run into a snag when a weight gets to 0, it never goes back up again.
Out of curiosity, why would you want or expect it to?
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
