Hypothesizing a new lighting method.

Started by
17 comments, last by MJP 10 years, 2 months ago

Hi Everybody. This is my first post so please be nice. I'm new to graphics programming with OpenGL and have only done basic tutorials thus far.

After going through the deferred rendering tutorial at http://ogldev.atspace.co.uk/, I was kind of left with the impression that the process of deferred lighting is a bit backwards. Forward rendering looks great with all lights but the only problem is that rendering everything causes an absurd amount of iteration (render time = light count * mesh count). Deferred rendering shows promise (light count + mesh count) but it also means that, in some implementations, you may have to drastically increase the amount of vertices rendered. Additionally, shadows are a bit more difficult to implement. I was thinking of another approach to rendering lights with the speed of deferred rendering but the visual quality of forward rendering.

Basically, there could be two cube maps stored on the GPU which could hold the color, intensity, and position of every light in a scene. The renderer would place every light of the scene into these cube maps in a first pass, then render each mesh in a second pass. The first cube map could add individual RGB and Intensity values of every light, clamped to four individual byte values while the second cube map could hold each light's position (clamped to 3 individual bytes). Next, after each light has been rendered to the cube maps, every mesh of the scene can sample them when they're processed in another pass by the pixel/fragment shader. This means that you get the speed of a deferred renderer (render time = light count + mesh count) with a drastically reduced memory footprint.

Would anyone be able to shed some light as to the plausibility of rendering a scene this way? Right now, I have calculated that both buffers would be a total size of about 2.6 Mb, which seems like a hell of a lot less than what is typically required by a deferred renderer. Like I said above, I'm a graphics novice; this is all just theory and I have no idea how it would be implemented in practice, I'm just looking for feedback.

My memory calculation is as follows:

4 bytes for the first cube map + 3 bytes for the second cube map = 7 total bytes per pixel, across both cubemaps (math whiz here).

Each cube map is made of 6 2D textures.

Each 2D texture is 256 pixels wide by 256 pixels high.

Therefore, the total memory used by the entire process is 7 bytes * 256 wide * 256 high * 6 textures = 2752512 bytes = 2.6 Mb total.

Advertisement

That could only work for non positional lights.

That could only work for non positional lights.

I was initially going to test this with point lights but I guess I'll have to figure out how to add directional and spot lighting. Throwing more textures at the GPU in order to hold each light's properties will end up using as much memory as regular deferred rendering.
At least with point lights, the light vector could be calculated pretty easily (in the fragment shader) by getting the vector from the camera to a mesh and reflecting that against the mesh's surface so that you can find what light[s] to pluck out from the first cube map. Once you do that, then you could also extract the lights position from the second cube map.

"This means that you get the speed of a deferred renderer (render time = light count + mesh count) with a drastically reduced memory footprint."

The speed of a deferred renderer has nothing to do with your algorithm at all. The speed in deferred comes in the fact that you run the pixel shader once for every pixel on the screen. You make pixel shader overdraw = 0. In your algorithm you would still draw an object with a pixel shader and then another object can come along and redraw over that pixel and have to run a pixel shader again (2x for the same pixel).

The algorithm itself doesnt make much sense to me. Why is it a cube map again? What does a mesh use to find the correct light in the cubemap? Why wouldnt it just be a 2D texture of the lights? How exactly does the light get mapped into the cubemap?(its not a 3d Array, its simply 6 faces that designate up down left righ bottom top.

Yes, you don't have the experience enough to finalize your algorithm. I'm planning to do something similar to this, however you have some holes to get to the end and you are in no way related to deferred rendering by this approach.

NBA2K, Madden, Maneater, Killing Floor, Sims http://www.pawlowskipinball.com/pinballeternal

The speed of a deferred renderer has nothing to do with your algorithm at all. The speed in deferred comes in the fact that you run the pixel shader once for every pixel on the screen. You make pixel shader overdraw = 0. In your algorithm you would still draw an object with a pixel shader and then another object can come along and redraw over that pixel and have to run a pixel shader again (2x for the same pixel).

In this, the pixel shader would be run once for every light to store it in the cube maps, then once more for every mesh when rendering to a final output buffer; just like a deferred renderer. Overdraw can occur regardless of what rendering method is used but it can be reduced if each object is sorted in the scene based on its position from the camera (but I'm not really worried about it at the moment).

The algorithm itself doesnt make much sense to me. Why is it a cube map again? What does a mesh use to find the correct light in the cubemap? Why wouldnt it just be a 2D texture of the lights? How exactly does the light get mapped into the cubemap?(its not a 3d Array, its simply 6 faces that designate up down left righ bottom top.

The lights would be stored in a cubemap for nothing more than indexing really. I imagine it to be a 3D lookup table to get an angle of incidence between the camera, a mesh, and a light in the scene.

So for the first pass on the scene, each light's position is stored in one cube map, then its RGB and intensity values in the other. Think of having a skybox but with a bunch of lights' colors and positions only. In order to actually get the data into the cubemaps, I was imagining having the 6 sides of the cube maps set up as render targets that would be bound to the output of a fragment shader.

After each light has been rendered to, then in a second fragment shader, a scene's meshes could be rendered to from the view of a camera. Then, as I noted earlier, you could find out how to color each fragment by getting a vector from the camera's position towards the mesh, reflect that vector off the mesh, and finally find which light that reflection vector is pointing at in the cube maps. Since each light's position is stored, you can determine how to color a fragment based on its position and intensity with regards to the final scene.

Ideas are just that. Consider throwing together a prototype/proof of concept that actually shows the lighting method and relevant shaders, and perhaps allows for some preliminary verification and performance testing. Also consider looking into tiled forward rendering, as I think that's where things are headed in the future.

SlimDX | Ventspace Blog | Twitter | Diverse teams make better games. I am currently hiring capable C++ engine developers in Baltimore, MD.


In this, the pixel shader would be run once for every light to store it in the cube maps

Generally, a cubemap represents the scene from a certain view point. For instance, they can be used to render the scene around a point and used to draw rough environment reflections on objects. "rough" because it will only be accurate from one specific position in the scene. Of course it works well for things like a skybox (which is "infinitely" far away and looks the same from every point in the scene).

So... what exactly are you putting in your cubemap? Is the end result that there will be a single texel somewhere in the cubemap for each light?


you could find out how to color each fragment by getting a vector from the camera's position towards the mesh, reflect that vector off the mesh, and finally find which light that reflection vector is pointing at in the cube maps.

What are the chances that that reflection vector will line up exactly with the texel that contains the light information? Pretty small. Or does your cubemap contain large areas that cover a light's influence? If so, how would you handle overlapping lights?

(also, reflecting the eye vector off the mesh will help you calculate the specular component, what about diffuse?)

Maybe I'm just not understanding your description, but I still don't see how your algorithm makes any sense. As Promit suggested, try coming up with a proof of concept. I think it will quickly become obvious where things fall apart.

I'm not going to go into much depth here, but almost everything you have said really shows your inexperience. Based on that, I would just say don't plan on doing this. You can make games look good with forward rendering, deferred whatever. Don't try to implement the greatest lighting model when you aren't fully grasping things. Really quickly:

"then once more for every mesh when rendering to a final output buffer; just like a deferred renderer."Overdraw can occur regardless of what rendering method is used"
Uh....deferred rendering has 0 pixel shader overdraw. The initial filling of the buffers of course will have some overdraw, but its not like its wasting time running crazy pixel shaders. All lighting shaders are run once for ever pixel. 0 pixel shader (lighting) overdraw.

"but it can be reduced if each object is sorted in the scene based on its position from the camera (but I'm not really worried about it at the moment)."
Yes. In a deferred renderer you would be doing this as well to reduce the G-Buffer initial pass overdraw, but not the lighting pass overdraw.

"Think of having a skybox but with a bunch of lights' colors and positions only."
So I have a 3d light: A and B. A = vec3(-100,10,0) B = vec3(-110,10,0). Where would these be rendered into the cube map? If you use simply their position as a vector to render to the cube map....they render to the same spot. A cube map is 6 2D textures, it is not a 3d array. A 3D array would be a cube map with slices cut through it. and even then an actual 3d Texture wouldnt work for your algorithm. Not going to explain to you why because I don't think you will get it.


"Then, as I noted earlier, you could find out how to color each fragment by getting a vector from the camera's position towards the mesh, reflect that vector off the mesh, and"
This wont work. If you are actually rendering the cube map lights as actual spherical geometry... then what you have essentially is cube mapping/environment mapping. But you would have to render all the lights each frame for each mesh. So much wrong with this algorithm.
And if you are talking about rendering geometry, what happens when 2 lights geometry overlap in the cube map?

NBA2K, Madden, Maneater, Killing Floor, Sims http://www.pawlowskipinball.com/pinballeternal

"What are the chances that that reflection vector will line up exactly with the texel that contains the light information? Pretty small. Or does your cubemap contain large areas that cover a light's influence? If so, how would you handle overlapping lights?"

Same thing I said pretty much. If you are only rendering to 1 pixel your light information, then it is still wrong. Noither approaches will work.

NBA2K, Madden, Maneater, Killing Floor, Sims http://www.pawlowskipinball.com/pinballeternal

Ok so I guess the idea needs a bit (or rather a lot) more work. Thanks you guys for the feedback, I'll try to rework my lighting method based on all your suggestions.

This topic is closed to new replies.

Advertisement