# Quantization errors

This topic is 5478 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

What are these errors? Please explain. Thank you

##### Share on other sites
What's the context? Quantization means, basically, taking a number and rounding it to fit some step size. For example, if you have a step size of 0.125 and you have a number 9.70132, then a quantization of 9.70132 to a step size of 0.125 is 9.625. I obtained this by calcing 0.125 * int(9.70132/0.125) = 0.125 * int(77.61056) = 0.125*77 = 9.625.

Now, if you are trying to represent 9.70132 but you are using, for example, hardware that quantizes to 0.125, then the error is the quantized number, 9.625, minus the number you want to represent, 9.70132, or 9.625 - 9.70132 = -0.07632.

Quantization is kind of a key factor in the sampling theory that is used in digital images and video (think textures and texture sampling), digital audio (think wave and mp3 files that are quantized to a certain bit rate), and other things.

Hope this helps.

##### Share on other sites
thanks
Yes your post was helpful. I just read about sin/cos lookup tables and quantization errors and couldn't find out how to match those two things together.

B.

##### Share on other sites
Quote:
 Original post by bogusthanks Yes your post was helpful. I just read about sin/cos lookup tables and quantization errors and couldn't find out how to match those two things together.B.

Oh, right, so the quantization error is the error between the actual sin/cos and the value you get by quantizing the input angle to one of the angles in the table.

• ### Game Developer Survey

We are looking for qualified game developers to participate in a 10-minute online survey. Qualified participants will be offered a \$15 incentive for your time and insights. Click here to start!

• 15
• 22
• 17
• 13
• 46