if you are wanting to do something like Minecraft, then yes, saving the chunks to disk is probably a good idea.

also, given the potentially steep memory and storage costs that may pop up with voxel terrain, investigating the use of data-compression may also be a good idea.

FWIW: my engine uses compressed voxels both in their on-disk and in-memory format, typically decompressing and recompressing chunks as-needed (typically inactive chunks revert to a compressed form in-memory, and may be dynamically decompressed when the chunk is accessed).

at least for storage, Minecraft stores its chunks in a deflate-compressed form, typically also packaged up inside of "region" files (each of which hold a 32x32 grid of chunks).

I think you should go with a octree format for voxel data storage. The concept with an octree is that say you have a 5x5x5 cube of dirt blocks (125 blocks). Rather than save each and every single one, wouldn't you rather just save one large block the same size? It sounds sorta complicated, but once you understand how it works in a whole, its actually pretty straight forward.

I found this blog entry on the implementation of a Minecraft like game using octrees. It just so happens to also have some source code attached (at the bottom of the entry)... :) You might also want to see the rest of his entries as here as later he revises his octree format a bit.

I would explore the octree approach, though that also comes with its own challenges of figuring out how to represent the octree in a structure so as to make it efficient to store and read as well, because the number of octree leaves for each chunk can differ greatly from one another.

Edit: Never mind, I realized you can store the octree data by assinging each leaf for each level a different number ID. This makes it much easier to understand and apply. All the solid areas can then be represented by the biggest leaf possible that encloses it by only storing these numbers. Numbering them, for example, starting with the top-most level (single leaf) is 0, next level down are 1-8, next level down 9 through 72, etc. Going down one level to subdivide a leaf by multiplying the current leaf's ID by 8, and then adding 1 through 8 to get the inner leaves.

RLE is better if there are a lot of the same types of blocks in a certain area. It keeps track of them in the following manner:

say you have 5 A blocks, 3 B blocks, and 1 C blocks) as follows:

AAAAABBBC

RLE would save that as:

5A3B1C

Notice that the Z takes up a single byte ('Z') in the first line, while it takes up two bytes ('1Z') in the second. This really becomes a problem when you encounter a lot of different object types.

In my opinion, Octrees are much more space and memory efficient, but they will be harder to parse (from the programmer's standpoint). The reason is that RLE must be decompressed in memory where as Octrees don't.

RLE is better if there are a lot of the same types of blocks in a certain area. It keeps track of them in the following manner:

say you have 5 A blocks, 3 B blocks, and 1 C blocks) as follows:

AAAAABBBC

RLE would save that as:

5A3B1C

Notice that the Z takes up a single byte ('Z') in the first line, while it takes up two bytes ('1Z') in the second. This really becomes a problem when you encounter a lot of different object types.

In my opinion, Octrees are much more space and memory efficient, but they will be harder to parse (from the programmer's standpoint). The reason is that RLE must be decompressed in memory where as Octrees don't.

whether or not a single item takes 1 or 2 bytes depends on the type of RLE.

in many forms of RLE, a single item will usually only take a single byte.

for example, a PCX-like RLE:

0x00-0xBF: encoded directly

0xC0-0xFF: RLE run (1-64 bytes), followed by byte.

another strategy:

most bytes, passed through (except the escape byte).

0xFE=Escape byte

0xFE <count> <value> = RLE run

or, possibly, a compromise:

0x00-0xBF: passed through

0xC0-0xCF <value>: RLE run 1-16 (0xC0 <value> = Escaped Byte)

0xD0 <count> <value>: RLE Run (16-271)

0xD1 <count16> <value>: RLE Run (272-65536 / 65551)

0xD2-0xDF: reserved

0xE0-0xFF: passed through

in my engine, most of the voxel terrain is kept compressed in memory as well, but chunks may be decompressed on access, and revert to a compressed form later.

RLE is used here, mostly because it compresses/decompresses quickly.

block-level RLE can be faster than byte-level RLE, and avoids needing to flatten out or reconstitute the blocks (into a collection of "byte planes" or similar), but has the disadvantage that it doesn't compress as well (only blocks which are exactly the same may be compressed).

a more "fancy" idea I had also considered was basically doing something more PNG-like, basically using a Paeth predictor and Deflate, but this wouldn't be as usable for in-memory compression, due the higher encoding/decoding costs.

another trick is essentially skipping chunks which only contain a single type of voxel (such as air or stone), treating them like a single large block, but this works more in my engine because chunks are 16x16x16 (and stack vertically as well).

haven't personally messed with using octrees this way.

You're right, I forgot to mention that.

I personally use octrees all the time. I've only used RLE once in a really early project I did a few years back. I chose it over octrees back then since it was much easier to implement and I wasn't too comfortable with C++ at the time to work with octrees.