I don't have any experience with file compression and i want to use zlib to compress my terrain. I was wondering if zlib can be used to decompress a region of interest, edit the data and compress it back?
The reason i'm asking is because i want to edit my terrain which is drawn using gpu clipmaps. So the heightmap isn't broken down into chunks. I want to be able to apply any changes i make to the terrain to the compressed heightmap. Is that possible?
Question about zlib?
don't have any experience with file compression and i want to use zlib to compress my terrain. I was wondering if zlib can be used to decompress a region of interest, edit the data and compress it back?
The encoding is a streaming enconding. If you want to use it, you need to compress the whole file again. Either subdivide your file into chunks and collect all chunks in an according file (i.e. zip), using zlib to compress/decompress each chunk. Or take a look at image compression algorithm which are able to work on a subregion only.
Heightmap chunks are likely to not compress terribly well, unless you take some special measures. Just like a digital photo from your camera does not compress well.
Zlib, like all LZ-based compressors, matches the next substring on a stream with (ideally) all possible substrings that it has seen before. Heightmap data will probably have some repetitions on a binary level, but unless your heightmap is something like a plane or a ramp, it is not likely that there will be many, or long ones. It's just not the nature of a heightmap.
Before compressing, you will definitively want to do a transform on your data to make it more compression-friendly. Some good choices may be:
- quantization
- delta encoding
- wavelets
Quantization simply reduces the number of possible values. This means fewer possible combinations in substrings, and this means a higher likelihood of having many matches. Of course, this reduces the quality of the heightmap.
Choosing 8 or 16 bit integers instead of float values is also a form of "quantization". This makes sense insofar as floating point values are always a bit tricky for being the same on a binary level. Depending on how you generate that data, a rounding error that is not visible in any way may give floating point values that aren't the same (and therefore don't match when the compressor runs over them).
Delta encoding exploits the fact that most terrain data is "kind of flat" or "kind of smoothly continuous". By encoding one vertex as the difference to a preceding neighbour vertex, many numbers will be zero or close to zero. Many similar/identical values yield a higher likelihood of finding many matches. You can additionally quantize, too, if you want. Delta encoding as such does not reduce quality, it is the same data represented differently.
Wavelets can be seen as a more mathematical formulation of delta encoding. They have the additional property of delivering lower levels of detail for free. For a perfect result, you'll need to dig a bit in mathematics, but a simple Haar transform is straighforward (like, 3-5 lines of code) and already provides quite good results. You can imagine a wavelet visually as walking over a texture's mip map levels, starting from the lowest 1x1 pixel mipmap, interpolating when scaling up, and encoding just the error. The interesting property with wavelets is that they also contain many values close to zero, and it does not make a very noticeable difference if you set the close-to-zero values to zero. Also, you kind of get LOD "for free", because the wavelet representation with the tail of the data left out is nothing but a lower level of detail!
With any kind of transform, you must make sure that vertices at the patch border are the same on neighbouring patches, or you run into trouble with cracks/holes (one very simple way of ensuring this is to treat the border pixels separately, but there are probably more space-conserving strategies). Or, you can use techniques such as degenerate triangles (which are no more degenerate when a crack would be visible!) or skirts to hide the cracks.
Zlib, like all LZ-based compressors, matches the next substring on a stream with (ideally) all possible substrings that it has seen before. Heightmap data will probably have some repetitions on a binary level, but unless your heightmap is something like a plane or a ramp, it is not likely that there will be many, or long ones. It's just not the nature of a heightmap.
Before compressing, you will definitively want to do a transform on your data to make it more compression-friendly. Some good choices may be:
- quantization
- delta encoding
- wavelets
Quantization simply reduces the number of possible values. This means fewer possible combinations in substrings, and this means a higher likelihood of having many matches. Of course, this reduces the quality of the heightmap.
Choosing 8 or 16 bit integers instead of float values is also a form of "quantization". This makes sense insofar as floating point values are always a bit tricky for being the same on a binary level. Depending on how you generate that data, a rounding error that is not visible in any way may give floating point values that aren't the same (and therefore don't match when the compressor runs over them).
Delta encoding exploits the fact that most terrain data is "kind of flat" or "kind of smoothly continuous". By encoding one vertex as the difference to a preceding neighbour vertex, many numbers will be zero or close to zero. Many similar/identical values yield a higher likelihood of finding many matches. You can additionally quantize, too, if you want. Delta encoding as such does not reduce quality, it is the same data represented differently.
Wavelets can be seen as a more mathematical formulation of delta encoding. They have the additional property of delivering lower levels of detail for free. For a perfect result, you'll need to dig a bit in mathematics, but a simple Haar transform is straighforward (like, 3-5 lines of code) and already provides quite good results. You can imagine a wavelet visually as walking over a texture's mip map levels, starting from the lowest 1x1 pixel mipmap, interpolating when scaling up, and encoding just the error. The interesting property with wavelets is that they also contain many values close to zero, and it does not make a very noticeable difference if you set the close-to-zero values to zero. Also, you kind of get LOD "for free", because the wavelet representation with the tail of the data left out is nothing but a lower level of detail!
With any kind of transform, you must make sure that vertices at the patch border are the same on neighbouring patches, or you run into trouble with cracks/holes (one very simple way of ensuring this is to treat the border pixels separately, but there are probably more space-conserving strategies). Or, you can use techniques such as degenerate triangles (which are no more degenerate when a crack would be visible!) or skirts to hide the cracks.
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement