• Create Account

# Integer scaling

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

7 replies to this topic

### #1lawnjelly  Members   -  Reputation: 764

Like
0Likes
Like

Posted 19 September 2013 - 03:10 AM

Hi, this is probably really easy, but..

how do I best do a fast, accurate integer scaling (0.0 - 1.0) of 2 unsigned chars (8 bit values)?

[Note: this is the same problem as doing accurate alpha-blending. But it is also useful in other areas, e.g. audio.]

e.g.

a = 100; // test number

b = 250; // scale factor 0 - 255, which should correspond to a 0.0 to 1.0 factor

result = (a * b) / 255;

However in this case the divide by a non-magic number (2, 4, 8, 16, etc) is presumably not as good as dividing by 256 which can be a bitshift? Although perhaps these days in most CPUs this is not an issue as memory bandwidth is more important?

And finally, what is the *best* way of doing this, allowing floating point stuff (and perhaps SIMD, maybe there is an instruction for it?).

*edit*

Variations I've seen:

1) Just divide by 256 (the bitshift)

(and ignore the fact that 255 x 255 will come out as 254 as the result)

Pros: Works fast

Cons: Might be ok in many uses but isn't accurate.

2) result = (a * (b+1)) / 256;

Pros: Should be almost as fast as the above.

when b = 0

e.g. 255 * 1 = 255,

255 / 256 = 0 CORRECT

when b = 255

255 * 256 = 65280

65280 / 256 = 255 CORRECT

Cons: Are there some cases where this gives different results than the 'true' result? Not sure

*/edit*

### #2Álvaro  Crossbones+   -  Reputation: 19901

Like
1Likes
Like

Posted 19 September 2013 - 08:08 AM

I can propose

3) result = (a * (1 + 2*b) + 256) / 512

None of those proposals give you the true result in all situations.

Option 1 disagrees with the true result in 47056 cases.

Option 2 disagrees with the true result in 18760 cases.

Option 3 disagrees with the true result in 7720 cases.

What I am using as the true result is this:

result = (2 * a * b + 255) / 510

I like options 2 and 3 because they do the right thing for b=0 and b=255.

However, you have to ask yourself how expensive the division by 510 really is. Is your program noticeably faster with one formula than with another? If not, stick to the true formula.

[EDIT: Note that the compiler can use some tricks to compute the division by 510 as a multiplication by some magic constant and a couple of bit shifts. A detailed explanation of how this would work can be found here.]

Edited by Álvaro, 19 September 2013 - 08:18 AM.

### #3lawnjelly  Members   -  Reputation: 764

Like
0Likes
Like

Posted 19 September 2013 - 08:29 AM

Wow thanks Alvaro. I'm sure a few of you have come across this exact issue.

I just was looking at it this morning, wondering at the best solution, as I'm doing an photoshop-like program at the moment so can sacrifice speed for getting the most accurate solution, but still want the 'fastest' accurate solution.

Must admit it makes my tiny brain go all fuzzy lol.

So is the

result = (2 * a * b + 255) / 510

more accurate than

result = (a * b) / 255

?

If so I'll change my code for this.

### #4Álvaro  Crossbones+   -  Reputation: 19901

Like
1Likes
Like

Posted 19 September 2013 - 08:57 AM

The problem with (a*b)/255 is that it rounds down. If a is 89 and b is 209, a*b/255 is the integer 72, although the result of the division before truncation is actually 72.945098..., which is almost 73. I would say the correct value in that case is 73.

So the formula that I take to be true' is

int((a*b)/255.0+0.5)

However, that formula involves converting between integers and floating-point numbers, which can be slow. But it's easy to fix if we move the +0.5 into the fraction like this:

int((a*b+127.5)/255.0) == int((2*a*b+255)/510.0) == (2*a*b+255)/510

Using (a*b)/255 results in an average darkening of the image by about half a level.

### #5LorenzoGatti  Crossbones+   -  Reputation: 3984

Like
0Likes
Like

Posted 19 September 2013 - 09:09 AM

(a*(b+1))/256 is wrong for numerous values, like a=185, b=186. Found by computer search.

Omae Wa Mou Shindeiru

### #6Álvaro  Crossbones+   -  Reputation: 19901

Like
0Likes
Like

Posted 19 September 2013 - 09:12 AM

(a*(b+1))/256 is wrong for numerous values, like a=185, b=186. Found by computer search.

Yes, and I posted above for how many values it is wrong: 18760 of the total 65536 possibilities. Of course I also found that with a little program.

### #7lawnjelly  Members   -  Reputation: 764

Like
0Likes
Like

Posted 19 September 2013 - 10:21 AM

This is probably purely academic now, but it's kind of interesting lol

3) result = (a * (1 + 2*b) + 256) / 512

seems to essentially be 'diluting' the inaccuracy (as far as I can see), is it possible to use 'bigger numbers' to make the approximation method more accurate?

i.e. would using 1024 instead of 512 make it better? Essentially using whatever you could fit into say, a 32 bit unsigned int?

Haha my maths is clearly lost here lol

### #8Álvaro  Crossbones+   -  Reputation: 19901

Like
1Likes
Like

Posted 19 September 2013 - 11:15 AM

The point of my formula (3) is that it is a proper scaling (i.e., of the form floor(a*fraction+0.5), where fraction' is a real number between 0 and 1) and it does the right thing for b=0 and b=255.

But yes, you can get a more precise formula using larger numbers. This one seems to work perfectly:

(a*b*65793+8388608)/16777216

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

PARTNERS