Hardware vertex processing is a lot faster than software vertex processing, but it's also very limited (maximum number of lights, for example) and the capabilities depend on the type of video card used. So I was wondering, is it a good idea to use hardware vertex processing without lighting, and just do the lighting calculations myself? That way the GPU handles the simple matrix transformations and the CPU handles the lighting calculations. Is this a good idea or is this even slower than normal software vertex processing?

If you're looking to have tons of lights affecting each object, consider doing your lighting in a deferred shading / deferred lighting style, which decouples the geometry from the lighting.

Otherwise, yes, you'll run into hardware limits trying to stuff the data for every light through a vertex shader. ~4-8 is probably feasible, though.

Hardware vertex processing would be by far the better option unless you are talking about some seriously old / low-end graphics hardware. Remember that doing vertex processing in SW: a) can make you CPU bound, and b) will require you to stream vertices across the PCIE bus, which can easily become a limiter.

If you want a lot of lights, there is deferred shading as RDragon1 mentioned. Also if you have a large amount of unclipped geometry data (approx. num_vertices > pixels * back_to_front_overdraw_factor), it may even be more efficient do to the lighting calculations in the pixel shader (not to mention the better image quality you would get by interpolating normals instead of lit-vertex colors).

Thanks everyone, your replies have really helped me.

One more question: is the method I mentioned still useful to calculate vertex lighting for static models (like the terrain), or should I just use shaders everywhere?