Since my previous post went unanswered I will try once more, maybe stating what my problem is in a more clear way.
That said, IMHO handling terrain chunks and batching meshes of multiple objects are two distinct use cases. They are so distinct that both require their own solution. I would generate a single vertex buffer for terrain, replacing regions of content when necessary. Why? Because rendering terrain will happen also in situations where more than a single chunk is visible at a time. If you would use multiple buffer objects, then you would switch buffers even during a single rendering pass of terrain. Using multiple memory blocks would mean to duplicate chunks in GPU memory. On the other hand buffer memory management would be relatively easy, assuming that the memory footprint of chunks is fix.
What about terrain chunks whose size is not fixed? I have modifiable terrain (mesh is extracted from 3D binary volume) so size may differ greatly based on how fragmented the terrain is - it can be flat hill with lowest amount of vertices, but also can be totally rough and fractured with much bigger mesh surface and thus vertices amount. So far I used single VBO per chunk, but I plan on having really big draw distance and it starts to hurt especially that chunks go in 3 dimensions and there can be several hundreds of highest LOD chunks (of course more LOD levels and concatenating chunks into bigger ones will also be implemented but still I think there is a lot of VAO switching that could be avoided).
I'm trying to figure out some custom allocator that would manage a single VBO (or few big ones, depending on total size of terrain geometry), but I can't figure out how to modify chunks that were changed, and also stream new chunks in/out as player moves. With a buffer of fixed size with N slots it would be easy - just swap old chunk with new chunk (whether new one is because player moved or digged existing chunk, doesn't matter), but I can't figure it out for dynamic chunks.
My initial idea was to allocate slightly more than needed - check what's the max size of chunk that exists now, allocate maybe 1.5x as much so all chunks will be padded and this will allow for them to grow or shrink without affecting other chunks in the memory. Problems start if there is situation where chunks grow beyond fixed size - what to do then? It also wastes a bit of memory of course, but not sure if that would be any significant amount.
Another idea would be to not pad to fixed size, but just allocate big enough buffer and put chunks one by one in linear way. If chunk that already exists changes it's geometry, I check if it's the same size or smaller - if so I just overwrite that chunk data. This will leave some portion of buffer orphaned, and it will probably be so small chunk of memory that it will be wasted forever. This can lead to buffer fragmentation at some point and also requires some fancy allocator that will keep track of memory use and find a place for new chunk to occupy if it exceeds it's current location space.