Jump to content

  • Log In with Google      Sign In   
  • Create Account


Integer scaling


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
7 replies to this topic

#1 lawnjelly   Members   -  Reputation: 429

Like
0Likes
Like

Posted 19 September 2013 - 03:10 AM

Hi, this is probably really easy, but..

 

how do I best do a fast, accurate integer scaling (0.0 - 1.0) of 2 unsigned chars (8 bit values)?

[Note: this is the same problem as doing accurate alpha-blending. But it is also useful in other areas, e.g. audio.]

 

e.g.

 

a = 100; // test number

b = 250; // scale factor 0 - 255, which should correspond to a 0.0 to 1.0 factor

 

result = (a * b) / 255;

 

However in this case the divide by a non-magic number (2, 4, 8, 16, etc) is presumably not as good as dividing by 256 which can be a bitshift? Although perhaps these days in most CPUs this is not an issue as memory bandwidth is more important?

 

And finally, what is the *best* way of doing this, allowing floating point stuff (and perhaps SIMD, maybe there is an instruction for it?).

 

*edit*

Variations I've seen:

 

1) Just divide by 256 (the bitshift)

 

(and ignore the fact that 255 x 255 will come out as 254 as the result)

 

Pros: Works fast

Cons: Might be ok in many uses but isn't accurate.

 

2) result = (a * (b+1)) / 256;

 

Pros: Should be almost as fast as the above.

when b = 0

e.g. 255 * 1 = 255,

255 / 256 = 0 CORRECT

when b = 255

255 * 256 = 65280

65280 / 256 = 255 CORRECT

 

Cons: Are there some cases where this gives different results than the 'true' result? Not sure

 

*/edit*



Sponsor:

#2 Álvaro   Crossbones+   -  Reputation: 10625

Like
1Likes
Like

Posted 19 September 2013 - 08:08 AM

I can propose

 

3) result = (a * (1 + 2*b) + 256) / 512

 

None of those proposals give you the true result in all situations.

 

Option 1 disagrees with the true result in 47056 cases. 

Option 2 disagrees with the true result in 18760 cases.

Option 3 disagrees with the true result in 7720 cases.

 

What I am using as the true result is this:

result = (2 * a * b + 255) / 510

 

I like options 2 and 3 because they do the right thing for b=0 and b=255.

 

However, you have to ask yourself how expensive the division by 510 really is. Is your program noticeably faster with one formula than with another? If not, stick to the true formula.

 

[EDIT: Note that the compiler can use some tricks to compute the division by 510 as a multiplication by some magic constant and a couple of bit shifts. A detailed explanation of how this would work can be found here.]


Edited by Álvaro, 19 September 2013 - 08:18 AM.


#3 lawnjelly   Members   -  Reputation: 429

Like
0Likes
Like

Posted 19 September 2013 - 08:29 AM

Wow thanks Alvaro. I'm sure a few of you have come across this exact issue.

 

I just was looking at it this morning, wondering at the best solution, as I'm doing an photoshop-like program at the moment so can sacrifice speed for getting the most accurate solution, but still want the 'fastest' accurate solution.

 

Must admit it makes my tiny brain go all fuzzy lol.

 

So is the

 

result = (2 * a * b + 255) / 510

 

more accurate than

 

result = (a * b) / 255

 

?

If so I'll change my code for this.



#4 Álvaro   Crossbones+   -  Reputation: 10625

Like
1Likes
Like

Posted 19 September 2013 - 08:57 AM

The problem with (a*b)/255 is that it rounds down. If a is 89 and b is 209, a*b/255 is the integer 72, although the result of the division before truncation is actually 72.945098..., which is almost 73. I would say the correct value in that case is 73.

 

So the formula that I take to be `true' is

int((a*b)/255.0+0.5)

 

However, that formula involves converting between integers and floating-point numbers, which can be slow. But it's easy to fix if we move the +0.5 into the fraction like this:

int((a*b+127.5)/255.0) == int((2*a*b+255)/510.0) == (2*a*b+255)/510

 

Using (a*b)/255 results in an average darkening of the image by about half a level.



#5 LorenzoGatti   Crossbones+   -  Reputation: 2306

Like
0Likes
Like

Posted 19 September 2013 - 09:09 AM

(a*(b+1))/256 is wrong for numerous values, like a=185, b=186. Found by computer search.
Produci, consuma, crepa

#6 Álvaro   Crossbones+   -  Reputation: 10625

Like
0Likes
Like

Posted 19 September 2013 - 09:12 AM

(a*(b+1))/256 is wrong for numerous values, like a=185, b=186. Found by computer search.

Yes, and I posted above for how many values it is wrong: 18760 of the total 65536 possibilities. Of course I also found that with a little program.



#7 lawnjelly   Members   -  Reputation: 429

Like
0Likes
Like

Posted 19 September 2013 - 10:21 AM

This is probably purely academic now, but it's kind of interesting lol smile.png

 

Alvaro, as your formula:

 

3) result = (a * (1 + 2*b) + 256) / 512

 

seems to essentially be 'diluting' the inaccuracy (as far as I can see), is it possible to use 'bigger numbers' to make the approximation method more accurate?

 

i.e. would using 1024 instead of 512 make it better? Essentially using whatever you could fit into say, a 32 bit unsigned int?

 

Haha my maths is clearly lost here lol biggrin.png



#8 Álvaro   Crossbones+   -  Reputation: 10625

Like
1Likes
Like

Posted 19 September 2013 - 11:15 AM

The point of my formula (3) is that it is a proper scaling (i.e., of the form floor(a*fraction+0.5), where `fraction' is a real number between 0 and 1) and it does the right thing for b=0 and b=255.

 

But yes, you can get a more precise formula using larger numbers. This one seems to work perfectly:

(a*b*65793+8388608)/16777216
 






Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS