# DCT: Dct-values fit into my data type

This topic is 4275 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

I'm a beginner and I think this is a pretty dumb question. Sadly I really can't find the correct answer for myself. I also think this may fit best into the "Math"-Forum and that you guys here really can help me. It's all about image compression... I have my image with my RGB or YUV values. Each of them ( R, G, B or Y, U, V ) is 8 bit ( -128... 127 ). I put them into a short or double and do my dct: void dct(char input[64], int output[64] ); After that I divide them by my quantisizer ( let's say 2 ) or a quantisizer matrix. void quant(int block[64]) { ( divide everything through the quantisizer... ) } Here comes my big problem: I need a char output too. If I use the bigger data type ( let's say int ) my compression get's even worse and not betther. I have my quantisized dct values. Most values are between -128 and 127. That's ok, they fit into my char-output data type. But there are some values that won't fit into that range. If I just make -128 or 127 my program works pretty well, but I get ugly artifacts ( noise ) at the corners of objects in my image. When I unquantisize my dct coefficients and do inverse dct I get, because of the improper input values, some values that are also over 127 or unter -128. So I have to set them to 127 or -128 too. Has somebody a solution?

##### Share on other sites
Assuming the "standard" DCT-II 8x8 transform, the transform should map values in [0,255] onto [-2047,2048].
This means that before the quantization, the output values require 12-bits of precision. Dividing the output values by 24=16, should squeeze it into [-127,128] range.

There's more to the compression process than just performing the transformation.
Usually, you should perform the following steps to compress the image:
1. Perform the transformation.
2. Divide the output by some quantization matrix. A different matrix can be used for each channel.
3. Compress the result by using RLE compression.
4. Further compress the result by using Huffman Encoding.

Note: Since the data is stored in a matrix, in step 3, we scan it using a ZigZag order starting from index (0,0).

The idea behind this compression is that steps 1 and 2 will result in a matrix with many repeating zeros and similar small values.

##### Share on other sites
If you really solved the problem I'm thinking about for three months - you are sent by heaven!! :) :) :)

Doesn't the quality suffer when dividing by 16 because of precision?!?
What should I do if I get values out of my range after undct? Is there a good recipe?

Greetings,
Jan Schiefer!

##### Share on other sites
Don't store your DCT coefficients as int's or anything like that, store them as *bits* (similar to the way huffman encoding stores each codeword as a number of bits). You can either decide on a fixed number of bits for each coefficient (it doesn't have to be the same for all of them, give more bits to those that come first in the 'zigzag' order, least to those that come last) or have it decided at runtime and store the number of bits per coefficient in your file format.

1. 1
2. 2
3. 3
Rutin
21
4. 4
5. 5
gaxio
10

• 14
• 30
• 13
• 11
• 11
• ### Forum Statistics

• Total Topics
631778
• Total Posts
3002310
×