Hi, (I'm no Dx9 expert, so I'm not sure whether this is even possible)

Samith has a good point. And if your target is performance, I'd recommend to follow his method. But if you really want to render more than 8 lights:

Store your lights in your own structure, saving them somewhere. Then when rendering your mesh, do the following: (Not checked for bugs)

.. 
// Split all your lights in the scene up to arrays* with 8 elements in each
// Make sure that at least one element is returned, even though that is has
// 0 lights, making sure that the mesh is drawn
std::vector<std::vector<LightType>> lights;
SplitLights(&lights, 8);

// lights could look like this::
// Element: ( max 8 )
//     LightA
//     LightB 
//     LightC
// Element: (max 8, ideally these lights should be in element 0, but you get the idea...)
//     LightD
//     LightE 
//     LightF

// Render the mesh without any lights
RenderMesh();

EnableAdditiveBlending();

for( int i = 0; i < lights.size(); i++)
{
    until(lights[i].size()) SetDXLight(light[i]);

    // Render the mesh with ONLY the lights
    RenderMesh();
}

This might not be the best nor fastest, but it should work and provide you your results.

EDIT: You might need to enable some kind of additive blending.

Hope this helps.

-MIGI0027

From the lights that the camera can see in the current frame, pick the 8 closest to the camera.

Yep, but if you have to specify a limit on the lights you can draw, well, if you have more lights in the viewable scene that you can draw, it will look bad regardless of what you do. Light popping in/out of existence.

The best you can do is to select which lights you're going to draw as best as possible (ie, the ones that will have most impact in the scene).

That's why when an engine has a hard limit on the amount of lights of certain types, level designers do their best to make scenes that work and look good under those limits.

Say for example, the limit on shadow casting lights for Skyrim is 4. Go over that and the lights fck up. Thus no single interior cell* in the entire game (a game which has hundreds of hand crafted interior cells) has more than 4 shadow casting lights.

* I don't remember exactly if the limit was per interior cell or just by room/bounding volume + whatever room you see through the portals.

Personally, if I'd have a realistic test case in which I need to have more than 100x the amount of supported lights, I'd start considering other methodologies, such as deferred shading.

Personally, if I'd have a realistic test case in which I need to have more than 100x the amount of supported lights, I'd start considering other methodologies, such as deferred shading.

This.

Unless there is a specific reason you want to do forward rendering with 1000 lights, I would do some research in deferred shading and perhaps use that.

It also depends a bit on how the lights are spread through the scene though, although some kind of spatial data structure is recommended regardless. If you're going to have a high density of lights within your view, it's very likely never going to be looking good regardless of how you do it (with forward rendering), unless you give up performance for some fancy alghorithm which I am not sure if it would be feasible if it exists.

If it's going to be something along the line of a city, at least where most of the lights are occluded by something, you should certainly have some kind of spatial data structure and probably checks if the light is even worth using (fully occluded by a wall you're standing next to for example).

Deferred shading is the way to go with so many lights, but artistically speaking there are too many lights in there. There are different "kind" of lights you want in a scene:

1. lights you want to see (i.e. light bulbs). You can use some halo textures and light maps to bake these in, having little performance impact

2. lights that you want to see reflected, like if you have a car in a show room and you want lots of lights bouncing off its surface, you usually solve this using some environment texture mapping effect

3. light that actually affect stuff which can be also per-vertex or per-pixel, the later being the real performance hogs.

From an artistic point of view however the less lights you use, the less confusing is for the player and the more dramatic effects you can obtain. If you look at professional films and photos you can see there are only a few lights that are used in a clever way to produce astounding effects. The more lights you add, the more fatigue and confused the player will get.