Jump to content

  • Log In with Google      Sign In   
  • Create Account

Hypothesizing a new lighting method.


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
20 replies to this topic

#1 hamsham   Members   -  Reputation: 267

Like
0Likes
Like

Posted 19 January 2014 - 12:53 AM

Hi Everybody. This is my first post so please be nice. I'm new to graphics programming with OpenGL and have only done basic tutorials thus far.

 

After going through the deferred rendering tutorial at http://ogldev.atspace.co.uk/, I was kind of left with the impression that the process of deferred lighting is a bit backwards. Forward rendering looks great with all lights but the only problem is that rendering everything causes an absurd amount of iteration (render time = light count * mesh count). Deferred rendering shows promise (light count + mesh count) but it also means that, in some implementations, you may have to drastically increase the amount of vertices rendered. Additionally, shadows are a bit more difficult to implement. I was thinking of another approach to rendering lights with the speed of deferred rendering but the visual quality of forward rendering.

Basically, there could be two cube maps stored on the GPU which could hold the color, intensity, and position of every light in a scene. The renderer would place every light of the scene into these cube maps in a first pass, then render each mesh in a second pass. The first cube map could add individual RGB and Intensity values of every light, clamped to four individual byte values while the second cube map could hold each light's position (clamped to 3 individual bytes). Next, after each light has been rendered to the cube maps, every mesh of the scene can sample them when they're processed in another pass by the pixel/fragment shader. This means that you get the speed of a deferred renderer (render time = light count + mesh count) with a drastically reduced memory footprint.

 

Would anyone be able to shed some light as to the plausibility of rendering a scene this way? Right now, I have calculated that both buffers would be a total size of about 2.6 Mb, which seems like a hell of a lot less than what is typically required by a deferred renderer. Like I said above, I'm a graphics novice; this is all just theory and I have no idea how it would be implemented in practice, I'm just looking for feedback.

 

My memory calculation is as follows:

4 bytes for the first cube map + 3 bytes for the second cube map = 7 total bytes per pixel, across both cubemaps (math whiz here).

Each cube map is made of 6 2D textures.

Each 2D texture is 256 pixels wide by 256 pixels high.

Therefore, the total memory used by the entire process is 7 bytes * 256 wide * 256 high * 6 textures = 2752512 bytes = 2.6 Mb total.

 

 



Sponsor:

#2 kalle_h   Members   -  Reputation: 1386

Like
0Likes
Like

Posted 19 January 2014 - 08:51 AM

That could only work for non positional lights.



#3 hamsham   Members   -  Reputation: 267

Like
0Likes
Like

Posted 19 January 2014 - 11:06 AM

That could only work for non positional lights.

I was initially going to test this with point lights but I guess I'll have to figure out how to add directional and spot lighting. Throwing more textures at the GPU in order to hold each light's properties will end up using as much memory as regular deferred rendering.
At least with point lights, the light vector could be calculated pretty easily (in the fragment shader) by getting the vector from the camera to a mesh and reflecting that against the mesh's surface so that you can find what light[s] to pluck out from the first cube map. Once you do that, then you could also extract the lights position from the second cube map.

#4 dpadam450   Members   -  Reputation: 921

Like
2Likes
Like

Posted 19 January 2014 - 11:54 AM

"This means that you get the speed of a deferred renderer (render time = light count + mesh count) with a drastically reduced memory footprint."

 

The speed of a deferred renderer has nothing to do with your algorithm at all. The speed in deferred comes in the fact that you run the pixel shader once for every pixel on the screen. You make pixel shader overdraw = 0. In your algorithm you would still draw an object with a pixel shader and then another object can come along and redraw over that pixel and have to run a pixel shader again (2x for the same pixel).

The algorithm itself doesnt make much sense to me. Why is it a cube map again? What does a mesh use to find the correct light in the cubemap? Why wouldnt it just be a 2D texture of the lights? How exactly does the light get mapped into the cubemap?(its not a 3d Array, its simply 6 faces that designate up down left righ bottom top.

Yes, you don't have the experience enough to finalize your algorithm. I'm planning to do something similar to this, however you have some holes to get to the end and you are in no way related to deferred rendering by this approach.



#5 hamsham   Members   -  Reputation: 267

Like
0Likes
Like

Posted 19 January 2014 - 01:49 PM

The speed of a deferred renderer has nothing to do with your algorithm at all. The speed in deferred comes in the fact that you run the pixel shader once for every pixel on the screen. You make pixel shader overdraw = 0. In your algorithm you would still draw an object with a pixel shader and then another object can come along and redraw over that pixel and have to run a pixel shader again (2x for the same pixel).

 

In this, the pixel shader would be run once for every light to store it in the cube maps, then once more for every mesh when rendering to a final output buffer; just like a deferred renderer. Overdraw can occur regardless of what rendering method is used but it can be reduced if each object is sorted in the scene based on its position from the camera (but I'm not really worried about it at the moment).

 

 

The algorithm itself doesnt make much sense to me. Why is it a cube map again? What does a mesh use to find the correct light in the cubemap? Why wouldnt it just be a 2D texture of the lights? How exactly does the light get mapped into the cubemap?(its not a 3d Array, its simply 6 faces that designate up down left righ bottom top.

 

The lights would be stored in a cubemap for nothing more than indexing really. I imagine it to be a 3D lookup table to get an angle of incidence between the camera, a mesh, and a light in the scene.

 

So for the first pass on the scene, each light's position is stored in one cube map, then its RGB and intensity values in the other. Think of having a skybox but with a bunch of lights' colors and positions only. In order to actually get the data into the cubemaps, I was imagining having the 6 sides of the cube maps set up as render targets that would be bound to the output of a fragment shader.

 

After each light has been rendered to, then in a second fragment shader, a scene's meshes could be rendered to from the view of a camera. Then, as I noted earlier, you could find out how to color each fragment by getting a vector from the camera's position towards the mesh, reflect that vector off the mesh, and finally find which light that reflection vector is pointing at in the cube maps. Since each light's position is stored, you can determine how to color a fragment based on its position and intensity with regards to the final scene.



#6 Promit   Moderators   -  Reputation: 7177

Like
0Likes
Like

Posted 19 January 2014 - 02:26 PM

Ideas are just that. Consider throwing together a prototype/proof of concept that actually shows the lighting method and relevant shaders, and perhaps allows for some preliminary verification and performance testing. Also consider looking into tiled forward rendering, as I think that's where things are headed in the future.


Edited by Promit, 19 January 2014 - 02:27 PM.


#7 phil_t   Crossbones+   -  Reputation: 3904

Like
1Likes
Like

Posted 19 January 2014 - 03:33 PM


In this, the pixel shader would be run once for every light to store it in the cube maps

 

Generally, a cubemap represents the scene from a certain view point. For instance, they can be used to render the scene around a point and used to draw rough environment reflections on objects. "rough" because it will only be accurate from one specific position in the scene. Of course it works well for things like a skybox (which is "infinitely" far away and looks the same from every point in the scene).

 

So... what exactly are you putting in your cubemap? Is the end result that there will be a single texel somewhere in the cubemap for each light?

 

 


you could find out how to color each fragment by getting a vector from the camera's position towards the mesh, reflect that vector off the mesh, and finally find which light that reflection vector is pointing at in the cube maps.

 

What are the chances that that reflection vector will line up exactly with the texel that contains the light information? Pretty small. Or does your cubemap contain large areas that cover a light's influence? If so, how would you handle overlapping lights?

 

(also, reflecting the eye vector off the mesh will help you calculate the specular component, what about diffuse?)

 

Maybe I'm just not understanding your description, but I still don't see how your algorithm makes any sense. As Promit suggested, try coming up with a proof of concept. I think it will quickly become obvious where things fall apart.



#8 dpadam450   Members   -  Reputation: 921

Like
0Likes
Like

Posted 19 January 2014 - 10:01 PM

I'm not going to go into much depth here, but almost everything you have said really shows your inexperience. Based on that, I would just say don't plan on doing this. You can make games look good with forward rendering, deferred whatever. Don't try to implement the greatest lighting model when you aren't fully grasping things. Really quickly:

"then once more for every mesh when rendering to a final output buffer; just like a deferred renderer."Overdraw can occur regardless of what rendering method is used"
Uh....deferred rendering has 0 pixel shader overdraw. The initial filling of the buffers of course will have some overdraw, but its not like its wasting time running crazy pixel shaders. All lighting shaders are run once for ever pixel. 0 pixel shader (lighting) overdraw.

"but it can be reduced if each object is sorted in the scene based on its position from the camera (but I'm not really worried about it at the moment)."
Yes. In a deferred renderer you would be doing this as well to reduce the G-Buffer initial pass overdraw, but not the lighting pass overdraw.

"Think of having a skybox but with a bunch of lights' colors and positions only."
So I have a 3d light: A and B.  A = vec3(-100,10,0) B = vec3(-110,10,0). Where would these be rendered into the cube map?  If you use simply their position as a vector to render to the cube map....they render to the same spot. A cube map is 6 2D textures, it is not a 3d array. A 3D array would be a cube map with slices cut through it. and even then an actual 3d Texture wouldnt work for your algorithm. Not going to explain to you why because I don't think you will get it.


"Then, as I noted earlier, you could find out how to color each fragment by getting a vector from the camera's position towards the mesh, reflect that vector off the mesh, and"
This wont work. If you are actually rendering the cube map lights as actual spherical geometry... then what you have essentially is cube mapping/environment mapping. But you would have to render all the lights each frame for each mesh. So much wrong with this algorithm.
And if you are talking about rendering geometry, what happens when 2 lights geometry overlap in the cube map?
 



#9 dpadam450   Members   -  Reputation: 921

Like
0Likes
Like

Posted 19 January 2014 - 10:04 PM

"What are the chances that that reflection vector will line up exactly with the texel that contains the light information? Pretty small. Or does your cubemap contain large areas that cover a light's influence? If so, how would you handle overlapping lights?"

 

Same thing I said pretty much. If you are only rendering to 1 pixel your light information, then it is still wrong. Noither approaches will work.



#10 hamsham   Members   -  Reputation: 267

Like
0Likes
Like

Posted 20 January 2014 - 01:52 AM

Ok so I guess the idea needs a bit (or rather a lot) more work. Thanks you guys for the feedback, I'll try to rework my lighting method based on all your suggestions.

#11 Styves   Members   -  Reputation: 1022

Like
1Likes
Like

Posted 20 January 2014 - 07:44 AM

I'd like to also put it on record that shadow mapping with deferred shading is no harder than it is with forward shading, and in fact many games use a very simple deferred approach for shadows (Crysis 1 for example) rather than applying shadows in the forward pass.



#12 samoth   Crossbones+   -  Reputation: 4765

Like
0Likes
Like

Posted 20 January 2014 - 08:45 AM

This may only work at all if you sample "cones" rather than reflection vectors, and only for lights at infinity. Surfaces are not perfect mirrors, so one point on the screen that you look at does not correspond to exactly one point in the "sky" (i.e. the cubemap). It corresponds to an area. Unless you come up with something very clever (something like a distance map with a direction vector to the closest "cardinal light" might actually be an idea...) you will need to do a lot of samples so it looks kind of good -- and then it's cheaper to just do normal forward rendering and evaluating all lights as usual.

 

Also, if a light is not "at infinity", let's say 5 meters above you, then an object which is at your position and an object which is  5 meters away will get light from the same angle and with the same intensity. Which is... just wrong, and just looks very bad.

 

Also I'm afraid that you are trying to solve the wrong problem. First of all, forward shading doesn't look better than deferred shading. It will usually look somewhat different, but not necessarily "better" as such. The one big advantage of forward shading is that transparency is a no-brainer (and also antialiasing is easier to get working), but other than that there is no reason why correctly implemented deferred shading should look any worse. On the contrary, you can get a much more consistent lighting and put a lot more shader work into every pixel.

 

On the other hand, deferred shading is not faster.  It is more scaleable in respect of geometry and lights (in particular shadow-casting lights) at the expense of higher memory and bandwidth demands. Deferred shading has a complexity of numbers of lights multiplied with [at most] the resolution plus a more-or-less constant setup, instead of numbers of lights multiplied with number of vertices. Also, deferred shading (if no such thing as clustered DS is done) needs one shadow map at a time, for the one light that is currently being shaded whereas forward shading needs shadow maps for all lights that may affect any of the drawn geometry at once.



#13 kalle_h   Members   -  Reputation: 1386

Like
0Likes
Like

Posted 20 January 2014 - 01:59 PM

First of all, forward shading doesn't look better than deferred shading. It will usually look somewhat different, but not necessarily "better" as such. The one big advantage of forward shading is that transparency is a no-brainer (and also antialiasing is easier to get working), but other than that there is no reason why correctly implemented deferred shading should look any worse. On the contrary, you can get a much more consistent lighting and put a lot more shader work into every pixel.

 

Most deferred renderers will sacrifice some precision with normals, position(depth reconstruction is not 100% accurate) and with other paramaters that can cause some quality loss on final image. Luckily this is mostly last gen games where this is visible.



#14 Funkymunky   Members   -  Reputation: 641

Like
0Likes
Like

Posted 21 January 2014 - 03:16 PM

This technique would work for the one point in the scene that you render as the center of the cube maps.  When you render the lights into the cube map, you're doing it relative to this one point.  So that one point would be lit correctly.  Everything else would be skewed.



#15 Hodgman   Moderators   -  Reputation: 30351

Like
0Likes
Like

Posted 21 January 2014 - 05:27 PM

I used a technique on the Wii that's very similar to the one mentioned in the OP, to get lots of dynamic lights on it's crappy hardware... except, instead of global cube-maps for the scene, each object had it's own cube-map, so that directionality and attenuation worked properly (at least on a per-mesh granularity - not very good for large meshes). Also, instead of cube-maps, we used sphere-maps for simplicity.... and you can't just render a light into a single texel, you have to render a large diffused blob of light.

The lighting is obviously much more approximate than doing it traditionally per-pixel -- the surface normal is evaluated per-pixel, but the attenuation and direction to light are evaluated per mesh. This means that for small meshes, it's pretty good, but for large meshes, all lights start to look like directional lights.

The other down-side is that you can't use nice BRDF's like Blinn-Phong or anything...

 

In general, this is part of a family of techniques known as image based lighting, and yes, it's common to combine a bunch of non-primary lights into a cube-map, etc -- e.g. think of every pixel in your sky-dome as a small directional light. Using it as a replacement for primary lights, across an entire scene, is a bit less popular.



#16 kalle_h   Members   -  Reputation: 1386

Like
1Likes
Like

Posted 22 January 2014 - 07:41 PM

I used a technique on the Wii that's very similar to the one mentioned in the OP, to get lots of dynamic lights on it's crappy hardware... except, instead of global cube-maps for the scene, each object had it's own cube-map, so that directionality and attenuation worked properly (at least on a per-mesh granularity - not very good for large meshes). Also, instead of cube-maps, we used sphere-maps for simplicity.... and you can't just render a light into a single texel, you have to render a large diffused blob of light.

The lighting is obviously much more approximate than doing it traditionally per-pixel -- the surface normal is evaluated per-pixel, but the attenuation and direction to light are evaluated per mesh. This means that for small meshes, it's pretty good, but for large meshes, all lights start to look like directional lights.

The other down-side is that you can't use nice BRDF's like Blinn-Phong or anything...

 

In general, this is part of a family of techniques known as image based lighting, and yes, it's common to combine a bunch of non-primary lights into a cube-map, etc -- e.g. think of every pixel in your sky-dome as a small directional light. Using it as a replacement for primary lights, across an entire scene, is a bit less popular.

For kinghunt 

 

I did very similar trick but instead of sphere maps I used spherical harmonics. I also calculated aproximated visibility function per object against all other objects. This was so fast that even particles could be light emitters.


Edited by kalle_h, 22 January 2014 - 07:42 PM.


#17 samoth   Crossbones+   -  Reputation: 4765

Like
0Likes
Like

Posted 23 January 2014 - 08:18 AM

While I can see how image-based lighting might work great on mobile devices, how do these translate to desktops where the ALU/TEX ratio is much higher? I would guess that using literally abundant ALU to compute a dozen lights that are visible from one fragment may very well be faster than to do yet an additional texture lookup?



#18 dpadam450   Members   -  Reputation: 921

Like
0Likes
Like

Posted 23 January 2014 - 12:26 PM

Just look up forward+ rendering.



#19 Hodgman   Moderators   -  Reputation: 30351

Like
1Likes
Like

Posted 23 January 2014 - 03:03 PM

While I can see how image-based lighting might work great on mobile devices, how do these translate to desktops where the ALU/TEX ratio is much higher? I would guess that using literally abundant ALU to compute a dozen lights that are visible from one fragment may very well be faster than to do yet an additional texture lookup?

In console/PC games, it's standard to use IBL for background/fill/bounce/ambient lights (GI), instead of a flat ambient colour, and then compute the direct lighting analytically.

 

For film-quality IBL, you don't pre-convolve the probes, and each pixel has to read thousands of importance-sampled values from the IBL probes and integrate them using the BRDF (which is both ALU and TEX heavy)...



#20 Frenetic Pony   Members   -  Reputation: 1311

Like
0Likes
Like

Posted 23 January 2014 - 03:47 PM

 

While I can see how image-based lighting might work great on mobile devices, how do these translate to desktops where the ALU/TEX ratio is much higher? I would guess that using literally abundant ALU to compute a dozen lights that are visible from one fragment may very well be faster than to do yet an additional texture lookup?

In console/PC games, it's standard to use IBL for background/fill/bounce/ambient lights (GI), instead of a flat ambient colour, and then compute the direct lighting analytically.

 

For film-quality IBL, you don't pre-convolve the probes, and each pixel has to read thousands of importance-sampled values from the IBL probes and integrate them using the BRDF (which is both ALU and TEX heavy)...

 

 

Which is why most people pre-convolve the probes. I'm not sure UE4 does, then again I'm not sure exactly what it is they're even doing. All I've gathered is "cube map array" and their ability to relight the probes in realtime.






Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS