I am currently in the phase of implementing in my 3D space game a procedurally generated asteroid field. Firstly I generate 10 different asteroid meshes (by displacing the vertices of a sphere). Next, for each asteroid, I do the following: 1) pseudo-randomly pick of the 10 mesh-models; 2) instance it in a pseudo-randomly chosen position; 3) assign an ID based on position; 4) rotate and re-scale the instances to make the look of asteroids more varied.
That process works well, but of course for really high number of asteroids in a big asteroid field, this is not ideal since the game ends up with too many instances to render. In order to increase the number of asteroids while decreasing the performance cost, what I am trying to do is to use billboards/imposters for the many thousands of asteroid instances that are just far away in the scene.
There are two basic approaches I can think of about how to implement that. The first attempt is to just store in a texture atlas for each of the 10 mesh-models many different view angles from which these models can been spotted from afar, then choose a position based on camera angle and position. The problem with that is the following: since the thousands of instances have different re-scaling, a quad used for billboarding/impostoring wouldn't really resemble the real instance since depending on the view angle, stretching the quad would not result in the same asteroid shape as each instance that is intended to be represented.
The second attempt could be to, instead of saving any texture-atlas beforehand, to re-render the models meshes to texture when camera rotates above a given threshold. The problem with that seems obvious: for many thousands of asteroids using those billboards, re-rendering model meshes to texture would end up being pretty expensive.
So, in a nutshell, my main question in generic format is: which billboard/impostor technique can I use to handle my many thousands of instances when these instances are each re-scaled?
