Doing just the directional sunlight as a real-time shadow map was running pretty slow in the engine as well, so this whole approach seemed like a bad idea. One of the reasons it was slow was that I was doing some tricky ps.1.1 code to get 16 bit shadow maps by using multiple color channels, and I was forced to apply this both during shadow creation as well as shadow testing.
So, I decided to perform the shadow test at level creation time, and store the results in a sort of lightmap. Now, I didn't care about doing radiosity, just direct lighting, so I didn't think it would be too hard. Turned out it was a fairly big PITA to get it all working right. It's not a traditional lightmap, b/c I don't store color in the texture, just a scalar term from [0..1]. I premultiply a % occlusion shadow term as well as the distance attenuation from the light. I could also include a filter if I desired, like static cloud cover, or to give the impression of trees overhead, etc.
Now, one of my requirements was NO lightmap artifacts. This may seem obvious, but there have been several commercial games with light leaks, resolution problems, and seams between lightmaps that I have seen. Having a top-down view made this even more important, b/c seams show up on floors much more obviously from top-down compared to a 1st person perspective.
So, how do you go about doing lightmaps?
Firstly, you need a method to determine shadowed/unshadowed wrt each light. I already had raycasting routines from my vertex shadow code. Each ray would travel through the octtree of world chunks and figure out which nodes it intersected. Then the ray would be sent down the AABB tree of opaque triangles for each world chunk until it hit something.
I added an option to jitter the light position and cast multiple rays, then averaged the results to produce a occlusion percentage.
Secondly, you need to store the result in the lightmap. For this, you need to know which triangle goes into which lightmap, and you need to know its position in the lightmap, and also the size of the lightmap texture. Also, you typically choose a lightmap density, which is how many texels per meter you will store. For AG, I use between 4 and 6 texesl per meter.
For speed reasons, you want to pack many closeby triangles into the same atlas of lightmaps. So, one texture contains many lightmaps. Here is an example lightmap of a pointlight from the AG engine.
There are two general approaches to deciding on lightmap atlas size. The first method chooses a fixed sized atlas, like 512x512. If there are too many triangles in the atlas, you end up shrinking some UV triangles to make them all fit. This is a fairly popular method used by texture packing programs. They will go to great lengths to pack things in efficiently, but if there is no room, they will effectively reduce the texel density by shrinking lightmap triangles in UV space.
I didn't want this approach, b/c it can cause artifacts. Imagine two world chunks next to each other. They meet at a flat floor. If one chunk also contains part of a detailed statue, it may run out of lightmap space in the atlas, and shrink some UV triangles. If it does this to the floor, you will see a seam between one chunk and another.
This is yet another case of things that work well for objects & characters breaking down when you want seamless world geometry.
So, I used the other approach : changing atlas size to fit the triangles at hand. I try to pack all lightmap triangles into a small lightmap texture, like 64x64, and grow it to 128x64, then 128x128, etc. until I get to 1024x1024. I actually don't yet handle running out of space at this point, b/c it is very unlikely to run out of space, due to the way I break up my world into chunks. Basically, each world chunk gets its own lightmap atlas. I don't allow any world chunk to have more than 4096 triangles in it, or be above a certain volume, so unless I had some very fractally dense geometry, I don't have to worry about it. None of my levels so far have more than 512x512 lightmap atlases.
Another task is to choose a lightmap projection for your triangles. In other words, how do you assign each triangle in world space to a lightmap triangle in UV space?
I use a pretty standard method of taking each triangle's face normal, finding it's major axis ( x, y or z ), and using the other two axes for lightmap UV coordinates. So, if a part of the floor was facing mainly positive y ( up ), then I would use the world space x coordinate to map the lightmap U, and the world space Z to map to lightmap V. Once this is accomplished, I then go through all triangles connected in world space, and find neighbors that also are facing the same direction. These groups of triangles are called a 'chart'. A group of 'charts' make up an 'atlas', which corresponds to the final lightmap texture.
Once you have your charts, the next step is to pack the charts into the atlas. The goal is to be efficient, so your lightmaps don't waste texture space. I have an entire update planned to go into this problem in more detail, because it's not trivial.
Finally, once you've packed your charts into an atlas, and generated the final lightmap UV coordinates ( and there are some subtleties here too ), now you need to perform the lighting at each texel in the lightmap. You need to turn this lightmap texel into a world space position, and then perform lighting, and store the result in that texel. There is the additional complication of texels slightly outside of the lightmap UV triangle, or off the chart completely. These will still be filtered with when using bilinear filtering, so you can't ignore them. And, it's not a great idea to do the normal cheats either. In an upcoming entry, I'll go into detail about how I make sure I handle this issue to have no lightmap seams.
Here is an early lightmap debugging screenshot, where I colored each triangle individually, and packed them into charts.
Jack