Sign in to follow this  

simple video codec from scratch

This topic is 4586 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I've been going through the code I have for a 2D engine that uses Direct X D3D 8.1 as the base. I implemented a class that wraps around video for Windows (vfw) and it works sweet with the ability to render decompressed video to textures and then render to the screen using D3D. I also managed to get the audio going which was a disaster in the previous attempt. I don't like however the tonne of codecs that are needed and the "loss of control" . Also... decoding video is very expensive as the high compression comes at a cost when decoding. After the frame is decoded by VFW I then have to lock a texture and convert 24 bit RBG bitmap into the 32 bit texture which is another overhead. I was thinking about writing a simple codec (if there is such a thing) and making it optimized for D3D playback. (ie. 512*512 stretched). Each frame would be encoded to a DXT1 texture and embedded into a stream. A 512*512 texture that was 768KB uncompressed uses 128KB after DXT1 compression and retains very good quality. By my calulations it works out to be 4 bits per pixel down from 24. However I feel 128KB/frame is still a bit high....or is this acceptable? The same texture saved as a good JPEG was around 74K but bearing in mind the JPEG would involve a lot more to uncompress. The compressing could have a few optimizations like run length compression or a check to see if scanlines are identical. Maybe I would be better of getting hold of some JPEG decoding code and optimize the hell out of it. The aim at the end of the day would be a simple but robust codec that is fast and simple to decode. I looked at Smacker but it's tuned to 8 bit color and is a bit outdated (the price tag is also very scary). Any thoughts, ideas or flames?

Share this post


Link to post
Share on other sites
Quote:

However I feel 128KB/frame is still a bit high....or is this acceptable?


Yes, it's HUGE :) but it all depends on what you plan to encode and on your hardware limitations...
As an example, most 1cd Divx movie are encoded at around 4k per frame (800kbit/s).

The video codec you suggest is simple but very inefficient...
The first thing you might want to try is 'virtually' splitting each frame in 8x8 blocks and not saving blocks that have not significantly changed since the last frame. That would be a start at exploiting temporal coherency and might even be doable on DXT1 directly (but in my opinion using DXT is not a good idea anyway).

The biggest gain in size in video compression comes from motion compensation. Even a simple RLE motion compensated codec might be able to beat DXT's 6:1 compression ratio on non trivial footage and should perform very well on stuff like anime.

As for Smacker, I don't know how skilled you are in programming, but doing better is really quite easy.

Share this post


Link to post
Share on other sites
Ok, thanks for the pointers.

I was thinking about an 8*8 check and seeing if the difference between frames was negligible. (will mean writing some code to do so)

I would assume with this I would have to have some key frames occasionally.

I wanted to avoid frames relying on the previous but I guess to get anywhere near 4K per frame I will have to do something of the sort.

I understand how JPEG converts to a different color representation and can discard some information. I want to avoid conversions RGB->YUV etc if possible.

I will accept for the time I have to spend on this I'm not going to write anything like DivX.

Share this post


Link to post
Share on other sites
The good thing about YUV is that you can downsample the two chrominance channels (UV), encode them with far fewer bits than the luminance channel (Y) and the difference will be very hard to tell. The conversion is really easy to do, not that slow (even less if you use SIMD) and directly gives you a non negligible compression ratio on stills. In my opinion, it's possible to avoid, yet it's well worth the CPU and effort if you plan to do some compression by yourself.

Just in case: http://www.fourcc.org/index.php?http%3A//www.fourcc.org/yuv.php
Look for the YCbCr link as it is a more practical format than YUV for integer compression.

4k/frame was just an example. You might not need to store 90 minutes of 640x360 video for a game. Also bitrate is of little importance. What's really important is: what quality for how much? :)

If you feel like you should really try to motion compensate between two frames just so that you get an idea of what to expect. Brute force motion vector search is awfuly slow but only at encoding time so you might not care much. Usually for two consecutive complex frames in a movie, a simple motion compensation from the first to the second (with no correction) gives you something very close to the source material. So the correction frame is almost always 0.

Share this post


Link to post
Share on other sites
OK...

I think in order to get half decent compression and some savings where blocks are the same as those on the previous frame I will have to do the following.

Have a restriction that the dimensions width and height be a multiple of 8, I could even go futher with this and restrict to powers of 2.

I need to find some good code (I know it's somewhere) that fully compresses a 8*8 block and similar code to fully decompress a previously compressed 8*8 block.

Steps :-

Divide the image into 8*8 blocks and do the following for each block

1. Shift the block
2. Perform a DCT on the block
3. Quantize the block
4. Subtract the last DC coefficient from the current DC coefficient
5. Zigzag the block
6. Zero run length encode the block
7. Break down the non-zero coefficients into variable-length binary numbers & their lengths
8. Entropy encode the run lengths & binary number lengths
9. Write the entropy encoded information & binary numbers to the output


ALGORITHM - DECODER

1. Read bit-by-bit from the file to get the DC coefficient, non-zero AC coefficients and the run lengths.
2. Get the DC coefficient and the 63 AC coefficients.
3. Then we dezigzag the block.
4. Add the last DC coefficient to the current DC coefficient
5. Next we dequantize the block
6. Next we perform the Inverse Discrete Cosine Transform .
7. Finally, we shift the block by adding 128 to each value in the block

Merge all the blocks to get the final block .

However seeing what has to be done...no wonder CPU gets gobbled, also I would just get the compressing done and concerntrate more on the decompressing.

Also from what I can gather the above is repeated for each YCrCb component as each is compressed individually.


Share this post


Link to post
Share on other sites
Well to simply start you off with codec writing, I'd suggest you look at Huffyuv - it's an open source, fairly small (source code wise) and straight forward codec for fast lossless compression (gives between 3:2 and 2:1 compression on general input)

Share this post


Link to post
Share on other sites
Quote:
Original post by whitde
However seeing what has to be done...no wonder CPU gets gobbled, also I would just get the compressing done and concerntrate more on the decompressing.


No, the CPU should not care about that. You're basically doing MJPEG compression wich aside from the disk accesses (not very good bitrate for it fails to exploit temporal coherency) is very fast.

Quote:
Also from what I can gather the above is repeated for each YCrCb component as each is compressed individually.


Yes, even if you can save some time with the downsampled chrominance channels (only half the luminance decompression work to do for the chrominance).

Share this post


Link to post
Share on other sites
If I get the 8*8 compress/decompress going with a decent average compression rate I will do something about exploiting temporal coherency.

Seems with the RGB->YUV and YUV->RGB fixed point integer arithmatic is the only way to go.

Share this post


Link to post
Share on other sites

This topic is 4586 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this