Sign in to follow this  

Quantization errors

Recommended Posts

What's the context? Quantization means, basically, taking a number and rounding it to fit some step size. For example, if you have a step size of 0.125 and you have a number 9.70132, then a quantization of 9.70132 to a step size of 0.125 is 9.625. I obtained this by calcing 0.125 * int(9.70132/0.125) = 0.125 * int(77.61056) = 0.125*77 = 9.625.

Now, if you are trying to represent 9.70132 but you are using, for example, hardware that quantizes to 0.125, then the error is the quantized number, 9.625, minus the number you want to represent, 9.70132, or 9.625 - 9.70132 = -0.07632.

Quantization is kind of a key factor in the sampling theory that is used in digital images and video (think textures and texture sampling), digital audio (think wave and mp3 files that are quantized to a certain bit rate), and other things.

Hope this helps.

Share this post

Link to post
Share on other sites
Original post by bogus
Yes your post was helpful. I just read about sin/cos lookup tables and quantization errors and couldn't find out how to match those two things together.


Oh, right, so the quantization error is the error between the actual sin/cos and the value you get by quantizing the input angle to one of the angles in the table.

Share this post

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this