Lightsmark realtime GI - how do they do it?

Started by
14 comments, last by zedz 16 years, 5 months ago
Hi, During my daily scan for info on realtime GI, I came across the Lightsmark demo from the Stepan Hrbek at Lightsprint. It's very inmpressive! Does anyone know how this is done? There is some info at the authors website, and it seems that he is rendering per-pixel material properties to textures that are used as input to a real time radiosity solver. Similiar to deferred shading. This way the solver can get all it needs directly and it does not need a separate scene description (like many realtime GI-papers out there do). There is no info on the radiosity solver except a reference to this old paper by Stepan Hrbek. Unfortunaly it's in Czech. I've tried a few online translators without luck. Prehaps anyone (the author?) would like to give me some pointers? Thanks, Jonas
Advertisement
Hi.

I know, i'm from Czech Republic, i read the paper long ago (I've got implementation of Global Illumination, which isn't really the same, but they're quite similar).

Their method.
I give an example - you have an scene with one light and some shadow casters. You generate an shadow map for light (into alpha channel i think) and into rgb channel you give a form - Energy taken by pixel (Direct Diffuse light - n.l) / color of pixel (texture pixel :-), if captured energy isn't near zero, you set these pixels as new lights and turn off the first light. You'll generate a new shadow map from the "new lights" and do the same calculation (Energy taken by pixel (Direct Diffuse light - n.l) / color of pixel (texture pixel :-)) for every pixel of these shadow maps. After all passes you need - render to screen. (I'm not gonna write my method here, it's almost the same)

I hope this is understandable, because i'm not a perfect translator.

My current blog on programming, linux and stuff - http://gameprogrammerdiary.blogspot.com

The creator talks a bit about his approach in this thread over at beyond3d.

To me it sounds a bit similar to the paper on Splatting Indirect Illumination
Yeah the first bounce is a bit like the SIL paper, although that's where the techniques diverge. I believe (by my interpretation of the B3D thread) after that the irradience results are being read back to the CPU and the light is being propogated - radiosity-style - per vertex. This implies a few things:

1) You need a fairly tesselated mesh to get any real details.

2) The technique does not handle any occlusion past the first light ray. While this actually makes the light data lower frequency (so #1 isn't as noticable), it misses some pretty important effects that are even captured by very simple approximations like AO.

3) The read-back to the CPU is IMHO unnecessary. GPUs can efficiently evaluate radiosity and light transfer and avoid the extra data movements/CPU load.

It's certainly a neat demo though, and a good step in the right direction. What it isn't is the "solution to all real-time GI". Like any other of the recent GI techniques, it models some chosen subset of the rendering equation.
Quote:Original post by AndyTX
1) You need a fairly tesselated mesh to get any real details.


Demo runs with original untesselated mesh taken from WoP.
It can be improved by tesselation, but what's the point NOW if 19 of 20 won't see any difference?

Quote:Original post by AndyTX
2) The technique does not handle any occlusion past the first light ray. While this actually makes the light data lower frequency (so #1 isn't as noticable), it misses some pretty important effects that are even captured by very simple approximations like AO.


Again, I try to improve realtime render, there's no point in adding light paths that are too expensive and their effects are minor NOW. But of course if techniques you talk about are very simple, they can be added easily.

Quote:Original post by AndyTX
3) The read-back to the CPU is IMHO unnecessary. GPUs can efficiently evaluate radiosity and light transfer and avoid the extra data movements/CPU load.


Why do you think? What's the best GPU only demo? Demos I know are simpler and slower. And I don't think their makers were dumb, they faced real limitations of GPU.

Quote:Original post by AndyTX
It's certainly a neat demo though, and a good step in the right direction. What it isn't is the "solution to all real-time GI". Like any other of the recent GI techniques, it models some chosen subset of the rendering equation.


Yes, I never said it's solution to all problems.
Jonas, I would write it in English next time, sorry :)
Paper described also CPU monte carlo technique in two versions
- updating only parts of scene where objects move (shooting from light sources to dynamic objects and beyond to cast shadows)
- updating whole scene in each frame
but both had some issues, so I left them, starting GPU shadowmap based work as Vilem described.
Dee, don't get me wrong, I'm not trying to be overly critical (nor are most people, including most @ B3D). However it is very important to understand the limitations of various techniques, particularly when GI is involved. Many, many people have and will continue to push "real time GI" systems which vary a huge amount in what they actually model. While I totally agree that you have to make concessions to run in real-time, let me reiterate that it's very important to know what those concessions are.

Quote:Original post by Dee_
Demo runs with original untesselated mesh taken from WoP.

Right, and the lighting is fairly low frequency. It is not enough to capture something like ambient occlusion from indirect light, even if the system did handle occlusion on bounces.

Quote:Original post by Dee_
It can be improved by tesselation, but what's the point NOW if 19 of 20 won't see any difference?

Do you have user studies to back that up, or is it just speculation? The former is meaningful, that latter is not.

Quote:Original post by Dee_
Again, I try to improve realtime render, there's no point in adding light paths that are too expensive and their effects are minor NOW.

Again, you have to back up statements about the visual impact of various features empirically, or with user testing. Look into the HDR tone mapping literature as they certainly realize the importance of this sort of testing.

Quote:Original post by Dee_
Why do you think? What's the best GPU only demo? Demos I know are simpler and slower. And I don't think their makers were dumb, they faced real limitations of GPU.

I've seen (and written) real-time photon mapping, caustics, radiosity, etc. stuff on the GPU and it's not terribly difficult (all of these techniques are highly parallel). I'd also cite the work on "Splatting Indirect Illumination" and Samuli Laine's instant radiosity stuff. AMD also has a D3D10.1 demo coming up that uses a spatial sampling method of GI. All of the above run fairly efficient on GPUs, despite their limitations, and indeed many of them model even more complex phenomena then lightscape. Again I'm not knocking the chosen tradeoffs in your demo, but don't be misinformed about the existence of efficient alternative methods.

Quote:Original post by Dee_
Yes, I never said it's solution to all problems.

True, but you did at say that it "moves the state of the art significantly", which some might take issue with. Again I think you've got a neat system there that makes some good tradeoffs to achieve performance. But you *need* to explain the limitations clearly, define how the algorithm scales with any number of parameters and ideally even outline how it works in more detail to convince people. Sure that's opening it up to being criticized, etc. and there's never a shortage of people to do that, but that's how academia and the "competition of ideas" works.

So kudos to your work so far - we'd like to hear more detail and offer suggestions for improvement.

[Edited by - AndyTX on October 29, 2007 3:37:08 PM]
Thanks for your input everyone.
Quote:... the irradience results are being read back to the CPU and the light is being propogated - radiosity-style - per vertex
To me, this implies that some kind of data structure is needed on the CPU to accelerate the radiosity computation. In the Beyond3D thread, Dee mensions that form factors for static geometry is pre-computed and for dynamic geometry they are computed on the fly. So if I get this right there is a fair amount of (CPU) work involved to update this data structure for the dynamic geometry?

This seems to be a step that most current RTGI methods requires.
Dachsbacher et al. and Dong et al. both provides great work but requires updating of a CPU data structure for dynamic geometry. The GPU memory footprint is also HUGE for anything else than simple scenes.
Laine et al. needs a CPU raytracer for the VPL management. Area lights are not supported and more than a single light contributing to indirect illumination is probably not practical.

I would like to avoid CPU work as much as possible, but right now it seems hard.
It'll be interesting to se what people will be doing with the cube maps arrays in DX 10.1.
Quote:I've seen (and written) real-time photon mapping, caustics, radiosity, etc. stuff on the GPU

Currently I'm doing research on interactive illumination for my masters thesis project. If you have any recomendations or pointers I'm all ears :)

Edit: Dee, I want to add that I think your work is very admirable. Keep it up!
Quote:Original post by Dark_Nebula
This seems to be a step that most current RTGI methods requires.
Dachsbacher et al. and Dong et al. both provides great work but requires updating of a CPU data structure for dynamic geometry. The GPU memory footprint is also HUGE for anything else than simple scenes.

Definitely it's often useful to do complex data structure creation/update on the CPU due to the rather poor scatter/pack/etc. support on current GPU architectures. CUDA/CTM help a bit, but there are still things that are more efficiently done on a conventional CPU. This sort of "breaking the graphics pipeline" is what - for instance - Neoptica is all about (until they got bought by Intel). It's a powerful technique, particularly on architectures with high-speed connections between the various memories/processors (PS3).

Quote:Original post by Dark_Nebula
Laine et al. needs a CPU raytracer for the VPL management. Area lights are not supported and more than a single light contributing to indirect illumination is probably not practical.

Actually the use of a CPU raytracer was just for convenience I believe... since they were only casting a few hundred rays any way would be fast enough, but as they mention in the paper it could certainly be done using the shadow map (that they already had) or occlusion queries, both entirely on the GPU.

Multiple lights would still scale linearly so there's no theoretical complexity issues, but indeed on current hardware it may fall into the realm of "not real-time" pretty quickly. That said I think such a technique could be pretty useful moving forward; it looks pretty good IMHO, and runs well even on current hardware.

Quote:Original post by Dark_Nebula
It'll be interesting to se what people will be doing with the cube maps arrays in DX 10.1.

Check out some of the ATI 3x00 previews; one of the main D3D10.1 demos that they're showcasing is a "real time GI" demo that uses cube map arrays. By my understanding (the information is pretty sparse right now) it basically just samples the 3D volume and generates small shadow cube maps at each of the sampling points. These then get transformed into some other representation for efficient sampling while shading. Simple, but a good step I think.

Quote:Original post by Dark_Nebula
Currently I'm doing research on interactive illumination for my masters thesis project. If you have any recomendations or pointers I'm all ears :)

The papers you referenced above are where I'd start :) Personally I'd probably play around with something similar to Laine's stuff since it seems somewhat promising (as long as you don't need multiple indirect bounces) and could certainly run entirely on the GPU. I'd also look into a deferred rendering-style instant radiosity method like "Splatting Indirect Illumination".

A good chunk of the stuff that I've played with involved a real-time GPU raytracer, so unfortunately that's not much help to you unless you have one of those lying around ;)
Quote:Actually the use of a CPU raytracer was just for convenience I believe... since they were only casting a few hundred rays any way would be fast enough, but as they mention in the paper it could certainly be done using the shadow map (that they already had) or occlusion queries, both entirely on the GPU.
Yes, using the shadow map is probably a feasible way to determine the set of visible VPLs. And one can probabply also render a color view (in low-res) from the lights POV and use that to determine the colors of the VPLs as well. This is accually one of the things I've planed to try :)
Quote:Multiple lights would still scale linearly so there's no theoretical complexity issues, but indeed on current hardware it may fall into the realm of "not real-time" pretty quickly.
Yes. Memory might be a problem also. Since their technique relies on the fact that they only need to update only a few shadow maps for the VPLs per frame. IIRC they use 256 256*256 shadow maps (32MB) per light. Together with GBuffer, SM's for direct lighting, reflection maps, scene textures and so on, it might be a real challange.
Quote:That said I think such a technique could be pretty useful moving forward; it looks pretty good IMHO, and runs well even on current hardware.
Agreed. Despite its limitations, I think it's worth playing around with.
Quote:Check out some of the ATI 3x00 previews
Where can I find these? Or do I have to wait untill november 15? :)
Quote: it basically just samples the 3D volume and generates small shadow cube maps at each of the sampling points. These then get transformed into some other representation for efficient sampling while shading.
From what I get from the leaked ATI paper, it's Irradiance volumes plus some form of dynamic AO. Interesting indeed. I like this approach and I'll probably take a look at that as well.
Quote:A good chunk of the stuff that I've played with involved a real-time GPU raytracer, so unfortunately that's not much help to you unless you have one of those lying around ;)
Heh, no I don't ;) But thanks for your input!

This topic is closed to new replies.

Advertisement