Jump to content

  • Log In with Google      Sign In   
  • Create Account






new experimental video codec, BTIC1C...

Posted by BGB, 19 September 2013 · 439 views

video codec compression DXT
Reused text...

Once again, it is a codec intended for quickly decoding into DXT for use in animated textures and video-mapped textures.

It is a direct continuation of my effort to implement Apple Video, but with a slight rename.

I ended up calling it BTIC1C as it is (despite being based on Apple Video), still within the same general "family" as my other BTIC1 formats, as otherwise I probably would have needed to call it "BTIC3" or something.

There may also be a Deflate-compressed variant, which will increase compression at some cost in terms of decoding performance.


ADD:
Test of using codec for video capture...


----
basically, created a newer video codec experiment I am calling BTIC1C:
http://cr88192.dyndns.org:8080/wiki/index.php/BTIC1C

it is basically a modified Apple Video / RPZA with a few more features glued on.


the idea is based on the observation that the RPZA and DXT block structures are sufficiently similar that it is possible to convert between them at reasonably high speeds (mostly by feeding bytes through tables and similar).

this is basically how my current implementation works, basically directly transcoding to/from DXT, and using some of my existing DXT related logic to handle conversions to/from RGBA space, and also things like lossy block quantization.

this then allows the decoder to operate at around 450-480 Mpix/s (for a single-threaded decoder, decoding to DXT). though, granted, the image quality isn't particularly great (slightly worse than DXT at best).


my additions were mostly adding support for alpha transparency, and LZ77 based dictionary compression.

the alpha transparency is basically compatible with existing decoders (seems to be compatible with FFmpeg, which displays an opaque version of the image).

there is support for blended-alpha transparency, but this will not work correctly if decoded by a decoder which expects RPZA (so non-blended transparency is needed).

the LZ77 support is not compatible with existing (RPZA) decoders, but results in a modest increase in compression (in my tests, a 25-50% size reduction with some of the clips tested). this does not add any real cost in terms of decoding speed. the dictionary compression is done in terms of runs of blocks (rather than bytes).


compressed file sizes are "competitive" with M-JPEG, though with M-JPEG generally able to deliver higher image quality (for pretty much anything above around 75% quality), and compressing better with clips which don't effectively utilize the abilities of the codec (it is most effective at compressing images with little movement and large flat-colored areas).

for clips though with mostly static background images and large flat-colored areas, such as "cartoon" graphics, the codec seems particularly effective, and does much better than M-JPEG in terms of size/quality (it requires setting the JPEG quality very low, resulting in an awful-looking mess).

additional competitiveness is added by being around 6-7 times faster than my current M-JPEG decoder (currently does around 70-90 Mpix/s with a single-threaded decoder, when decoding directly to DXT).


comparison between this and a prior format, BTIC1A, is that this format compresses better, but has lower decode speeds.

tests have implied that Huffman compression would likely help with compression (based on a few experiments deflate-compressing the frames), but would likely hurt performance regarding the decode speeds.


I have not done tests here for multi-threaded decoding.


or such...





You certainly love developing codecs :)

trying to find the ones that are "just right" mostly...

 

as-in, for-video goes fast and has competitive size/quality tradeoffs, matches the use-case, and which I can hack my usual sets of extensions onto, ... (*1). basically, animated-textures and video-maps aren't really standard use-cases for video codecs.

 

or other limitations, like my audio-codec efforts mostly being motivated by RAM issues, where as-is I have a lot of RAM pressure, and more conventional codecs didn't really fit the use-case (fine-grain random access via a mixer, *2).

 

 

*1: ideally, I want designs which are "baseline compatible" with existing implementations, when possible (IOW: where it is possible to produce video which existing decoders can still decode, albeit lacking some features). then you can see the contents in a video player (vs a file which just gives a missing-codec error message, or requires a special player).

 

*2: Vorbis wouldn't really work here absent a fairly specialized encoder and some hackery, or having the mixer use much bigger sample-blocks (the BTAC variants were designed around the mixer, which internally operates on a 172Hz tick).

 

though, having the mixer be able to directly use Vorbis (without needing to decode it to PCM or transcode it to BTAC) may be an eventual option.

an AAC variant could be a better fit (than Vorbis), but is legally uncertain.

 

 

I might consider using this new one (in its RPZA backwards-compatible mode) for in-engine video capture as-well, mostly because its encode speed is also a little faster, so is likely less prone to result in duplicate frames. (as-is, most of the time is going into the initial RGBA->DXT1 conversion, *3).

 

the quality is a little lower (I suspect mostly due to 15 bpp), but I suspect this wont matter as much after MovieMaker has its way with it (conversion to MP4 tends to result in a more noticeable reduction in image quality).

 

also, sadly, there is less room for hacking extensions onto the format without breaking existing decoders, though it still seems possible (a few extensions use ignored bits, another option is putting all extension data after the main image, *4).

 

though granted, I have only been testing against FFmpeg. others may not necessarily respond so gracefully (what would QuickTime do? ...).

 

*3: I get currently between 70 and 120 Mpix/s encoding depending mostly on which DXT conversion function I use.

 

*4: the FFmpeg decoder seems to stop decoding at this point, so it may be possible to get away with encoding additional image data and layers after the end of the base image.

 

ex (RGBA DXT5 RPZA-compatible mode):

RGB-Base, Alpha-Base, BT1C image headers, layers / etc...

if this works, then the only "real" restriction is being unable to use LZ compression in the base-image.

PARTNERS