Terrain and chunks...
well, now more work is once again going into working on the voxel terrain system.
this is, at present, one of the bigger users of time and memory-resources in my 3D engine.
so, a few new things:
added "random think events", which basically allow random chunk-updates to occur (for example, grass now spreads and cactuses now grow, ...);
went and fully added "infinite terrain" support, though practically it is still not nearly as "infinite" as in Minecraft, it is at least possible to travel several km from the origin and return without too much issue (chunks and regions will load and unload as needed);
I am considering adding biomes, and started some work on this;
I am also adding support for indexed chunks.
I had tried, not entirely successfully, to change the region size in a "friendly" way, but ended up reverting to the prior region size mostly as this started resulting in "creeping corruption".
basically, I was wanting to have more symmetric cube-shaped regions (256x256x256), rather than asymmetric ones (512x512x128), but such a change naturally breaks compatibility with the pre-existing region files. I tried using a hack to "fudge over it", but this causes things to start quickly falling apart.
I was thinking it could be possible to replace the current use of spawner-blocks with random-think-events, which would basically again make it more like MC, where mobs can spawn on any qualifying block, rather than needing explicit spawner blocks to spawn them.
index chunks are basically an in-development memory-saving trick.
currently, there are 2 types of chunks:
raw chunks, where each voxel requires 8 bytes (leading to 32kB for a 16x16x16 chunk);
RLE chunks, where the chunk is stored in an RLE-compressed format (it is decompressed when accessed, and raw chunks revert to this form if not accessed for a certain number of ticks).
with an index chunk, it instead will hold a table of voxel values, and each voxel will hold an index into this table.
this should save a fair bit of memory in the case of chunks where only a moderately small number (< 256) unique voxels exist, without requiring explicit decompression (and only requires ~ 6kB for voxel data rather than 32kB).
they will also have their own RLE compressed form as well.
the main intent of the index-chunks is to reduce the need for dynamic RLE compression.
for their RLE form, I had idly considered using a predictor, but predictors aren't free, and I don't really know of an "ideal" predictor strategy for 8-bit indexed data anyways (IOW: cheap+effective).
a predictor would make more sense for a more aggressively compressed format (IOW: one with entropy coding), rather than a bytewise RLEB variant.
index chunks will not fully replace raw chunks as:
there is still a need for chunks with possibly more than 256 unique voxels (word-indices are possible but are less competitive, *1);
writing voxels to index chunks is somewhat more expensive (need to index the voxel-value).
*1: especially for chunks which are maybe mostly stone or dirt and maybe a few other things, which I suspect are typically the majority, given my region files are indicate an average of around 640 bytes per RLE chunk, which would not happen if there were any significant number of unique voxels in each chunk. nevermind that I gained compression by hacking RLEB to support runs > 256 bytes (IOW: very long runs all of the same value), ...
though, strickly speaking, the memory-break-even point is around 2k-3k unique voxels per chunk.
the indexing cost will also increase as the voxel-count increases.