Jump to content

  • Log In with Google      Sign In   
  • Create Account


Question about zlib?


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
4 replies to this topic

#1 Waaayoff   Members   -  Reputation: 780

Like
0Likes
Like

Posted 27 June 2012 - 02:32 PM

I don't have any experience with file compression and i want to use zlib to compress my terrain. I was wondering if zlib can be used to decompress a region of interest, edit the data and compress it back?

The reason i'm asking is because i want to edit my terrain which is drawn using gpu clipmaps. So the heightmap isn't broken down into chunks. I want to be able to apply any changes i make to the terrain to the compressed heightmap. Is that possible?
"Spending your life waiting for the messiah to come save the world is like waiting around for the straight piece to come in Tetris...even if it comes, by that time you've accumulated a mountain of shit so high that you're fucked no matter what you do. "

Sponsor:

#2 Waaayoff   Members   -  Reputation: 780

Like
0Likes
Like

Posted 29 June 2012 - 01:49 AM

Anyone?
"Spending your life waiting for the messiah to come save the world is like waiting around for the straight piece to come in Tetris...even if it comes, by that time you've accumulated a mountain of shit so high that you're fucked no matter what you do. "

#3 Ashaman73   Crossbones+   -  Reputation: 6877

Like
1Likes
Like

Posted 29 June 2012 - 03:37 AM

don't have any experience with file compression and i want to use zlib to compress my terrain. I was wondering if zlib can be used to decompress a region of interest, edit the data and compress it back?

The encoding is a streaming enconding. If you want to use it, you need to compress the whole file again. Either subdivide your file into chunks and collect all chunks in an according file (i.e. zip), using zlib to compress/decompress each chunk. Or take a look at image compression algorithm which are able to work on a subregion only.

#4 Waaayoff   Members   -  Reputation: 780

Like
0Likes
Like

Posted 29 June 2012 - 04:40 AM

Alright thanks :)
"Spending your life waiting for the messiah to come save the world is like waiting around for the straight piece to come in Tetris...even if it comes, by that time you've accumulated a mountain of shit so high that you're fucked no matter what you do. "

#5 samoth   Crossbones+   -  Reputation: 4642

Like
0Likes
Like

Posted 29 June 2012 - 05:16 AM

Heightmap chunks are likely to not compress terribly well, unless you take some special measures. Just like a digital photo from your camera does not compress well.

Zlib, like all LZ-based compressors, matches the next substring on a stream with (ideally) all possible substrings that it has seen before. Heightmap data will probably have some repetitions on a binary level, but unless your heightmap is something like a plane or a ramp, it is not likely that there will be many, or long ones. It's just not the nature of a heightmap.

Before compressing, you will definitively want to do a transform on your data to make it more compression-friendly. Some good choices may be:
- quantization
- delta encoding
- wavelets

Quantization simply reduces the number of possible values. This means fewer possible combinations in substrings, and this means a higher likelihood of having many matches. Of course, this reduces the quality of the heightmap.
Choosing 8 or 16 bit integers instead of float values is also a form of "quantization". This makes sense insofar as floating point values are always a bit tricky for being the same on a binary level. Depending on how you generate that data, a rounding error that is not visible in any way may give floating point values that aren't the same (and therefore don't match when the compressor runs over them).

Delta encoding exploits the fact that most terrain data is "kind of flat" or "kind of smoothly continuous". By encoding one vertex as the difference to a preceding neighbour vertex, many numbers will be zero or close to zero. Many similar/identical values yield a higher likelihood of finding many matches. You can additionally quantize, too, if you want. Delta encoding as such does not reduce quality, it is the same data represented differently.

Wavelets can be seen as a more mathematical formulation of delta encoding. They have the additional property of delivering lower levels of detail for free. For a perfect result, you'll need to dig a bit in mathematics, but a simple Haar transform is straighforward (like, 3-5 lines of code) and already provides quite good results. You can imagine a wavelet visually as walking over a texture's mip map levels, starting from the lowest 1x1 pixel mipmap, interpolating when scaling up, and encoding just the error. The interesting property with wavelets is that they also contain many values close to zero, and it does not make a very noticeable difference if you set the close-to-zero values to zero. Also, you kind of get LOD "for free", because the wavelet representation with the tail of the data left out is nothing but a lower level of detail!

With any kind of transform, you must make sure that vertices at the patch border are the same on neighbouring patches, or you run into trouble with cracks/holes (one very simple way of ensuring this is to treat the border pixels separately, but there are probably more space-conserving strategies). Or, you can use techniques such as degenerate triangles (which are no more degenerate when a crack would be visible!) or skirts to hide the cracks.

Edited by samoth, 29 June 2012 - 07:38 AM.





Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS