12 replies to this topic

[jcabeleira]

Ei, guys!
I'd like to show you guys the results of my attempt on implementing the real-time global illumination used by the Unreal Engine 4 and discuss the results I got.

Here is an overview of the implementation:

• I do not own a DX11 grade graphics card so my implementation uses simple 3D textures instead of Sparse Voxel Octrees and the voxelization is performed on the CPU with raytracing . For each voxel, the raytracer sends a ray in each axis direction (-X, +X, -Y, +Y, -Z, +Z) with distance clamping to avoid intersecting geometry outside the voxel and stores the color and normal of the intersection point.
• Six 128x128x128 3D textures are used to represent the anisotropic radiance of the voxel volume, i.e. the radiance that goes in each axis direction (-X, +X, -Y, +Y, -Z, +Z).The points that were calculated before are rendered into these volume textures, their illumination is calculated and they are injected into the 3D textures by weighting their normals against the direction that each of the 3D texture represents thus giving us how much of the lighting goes into each direction.
• After the 3D textures are created we need to generate their mipmaps so we can perform cone tracing. This step requires a custom mipmapping step that adds the radiance of neighbouring texels instead of averaging becase without it empty voxels wich are black would darken the radiance in the deeper mipmap levels.
• Once the volumes are ready the GI is rendered with a full screen pass where 16 uniformly distributed cones are cast. For each cone, 20 samples are taken along it with a fixed distance between samples. For each sample, the sample radius is calculated from the cone angle and the distance to the source point and used to calculate the correspondent mipmap level. The anisotropic radiance is sampled from the 6 volumes and weighted by the cone direction.

Regarding performance, these are my thoughts:

• Using a fullscreen pass for the GI, with 16 cones per pixel, 20 samples per cone gives an average of 2 FPS on my GTX260 Mobile.
• Using a configuration similar to the UE4, rendering at 1/3 of the screen resolution (which should be fairly equivalent to their 3x3 interlacing method) with 9 cones per pixel, 20 samples per cone (it is unknown how much samples UE4 uses for each cone) I obtain 10 FPS.
• It would be interesting to see how this runs on a more powerfull GPU like the GTX680 that it was used on the UE4 presentation
• I'm also curious to see if a Sparse Voxel Octree would increase or reduce the performance (please share your thoughts on this)

Regarding quality:

• The overall quality fairly independent on the number of cones. Usually, 9 cones provide acceptable quality while 16 cones provide good quality and 32 is similar to 16 cones while providing subtle fine details.
• Specular reflections are not that good for two reasons: the low resolution of the volume and the fact that the reflection has no GI, only direct illumination, which makes them look displaced from the scene even for glossy reflections.
• In certain situation some bleeding shows up on the corners, probably due to the discretization of the scene and low resolution of the volume

The following screenshots were taken with the following configuration: fullscreen resolution, 16 cones per pixel, 40 samples per cone.

Edited by jcabeleira, 19 September 2012 - 10:28 AM.

[Kaptein]

if this is all realtime, then i salute you =) and i hope it works just like you want it to
volumetric rendering REALLY needs a giant boost!

[D.V.D]

I downloaded the demo on your site, im pretty sure its the same one but I only get a black screen O.o

[cronocr]

Congratulations, that engine is amazing

[jameszhao00]

This looks pretty cool!

- How are you choosing your grid bounds?
- Is the voxelization done at runtime? I assume no? ("voxelization is performed on the CPU with raytracing")
- The intersection generated by the +X ray is injected into the -X 3D texture?
- During cone trace, how are you dealing with occlusion?
- "which makes them look displaced from the scene even for glossy reflections." What does this mean? Shouldn't Eye <-> Glossy <-> Diffuse <-> Light work?

Also, is there SSAO in those screenshots?

Awesome stuff

Edited by jameszhao00, 19 September 2012 - 10:33 PM.

[MrOMGWTF]

Any chances of gettings source code?
And does it support indirect occlusion?

- During cone trace, how are you dealing with occlusion?

This.

Edited by MrOMGWTF, 20 September 2012 - 08:19 AM.

[jcabeleira]

Thanks for your replies, guys.

I downloaded the demo on your site, im pretty sure its the same one but I only get a black screen O.o

Sorry to disappoint you but the demo on my site is a bit old and doesn't include these new features. The black screen must be caused by some incompatibility with your graphics card, unfortunately the demo was only tested on an Nvidia GTX 260 so it should work well on any NVidia card above that. Also, make sure you have updated drivers.

How are you choosing your grid bounds?
- Is the voxelization done at runtime? I assume no? ("voxelization is performed on the CPU with raytracing")
- The intersection generated by the +X ray is injected into the -X 3D texture?
- During cone trace, how are you dealing with occlusion?
- "which makes them look displaced from the scene even for glossy reflections." What does this mean? Shouldn't Eye <-> Glossy <-> Diffuse <-> Light work?

Also, is there SSAO in those screenshots?

The grid is fixed at the world origin with dimensions of 30x30x30 meters.
The voxelization can be redone in runtime but it is not real-time. The raytracing takes about 1 second and inserting the points into the volumes is slow as hell taking about 8 seconds. This step is far from optimized because I'm not even using vertex buffers for rendering the points, I'm rendering them with glBegin/glEnd (shame on me, I know). EDIT: i've replaced the point injection with vertex arrays and now it is done instantaneously.
The intersection generated by the +X ray is not necessarly injected into the -X volume. The rays are cast in all 6 directions only to ensure the voxel representation of the scene has no holes. The injection into the 6 destination volumes depends only on the normal of the point which is what represents the direction of the radiance.
Occlusion is dealt by keeping track of the accumulated opacity of all the samples processed for that cone. Think of it as regular transparency blending, where each sample is a semi transparent window that partially occludes the next sample.
The reflections don't look good because the voxel representation of the scene contains only direct lighting. So, what you'll see in the reflection is the scene lit by your light sources but being completely black on the shadows.
Regarding the SSAO, yes I have it. But the cool thing about this technique is that if you use enough cones you don't even need SSAO since the technique provides that effect for free and with much more realism.

Edited by jcabeleira, 20 September 2012 - 04:52 PM.

[MrOMGWTF]

Thanks for your replies, guys.

I downloaded the demo on your site, im pretty sure its the same one but I only get a black screen O.o

Sorry to disappoint you but the demo on my site is a bit old and doesn't include these new features. The black screen must be caused by some incompatibility with your graphics card, unfortunately the demo was only tested on an Nvidia GTX 260 so it should work well on any NVidia card above that. Also, make sure you have updated drivers.

How are you choosing your grid bounds?
- Is the voxelization done at runtime? I assume no? ("voxelization is performed on the CPU with raytracing")
- The intersection generated by the +X ray is injected into the -X 3D texture?
- During cone trace, how are you dealing with occlusion?
- "which makes them look displaced from the scene even for glossy reflections." What does this mean? Shouldn't Eye <-> Glossy <-> Diffuse <-> Light work?

Also, is there SSAO in those screenshots?

The grid is fixed at the world origin with dimensions of 30x30x30 meters.
The voxelization can be redone in runtime but it is not real-time. The raytracing takes about 1 second and inserting the points into the volumes is slow as hell taking about 8 seconds. This step is far from optimized because I'm not even using vertex buffers for rendering the points, I'm rendering them with glBegin/glEnd (shame on me, I know).
The intersection generated by the +X ray is not necessarly injected into the -X volume. The rays are cast in all 6 directions only to ensure the voxel representation of the scene has no holes. The injection into the 6 destination volumes depends only on the normal of the point which is what represents the direction of the radiance.
Occlusion is dealt by keeping track of the accumulated opacity of all the samples processed for that cone. Think of it as regular transparency blending, where each sample is a semi transparent window that partially occludes the next sample.
The reflections don't look good because the voxel representation of the scene contains only direct lighting. So, what you'll see in the reflection is the scene lit by your light sources but being completely black on the shadows.
Regarding the SSAO, yes I have it. But the cool thing about this technique is that if you use enough cones you don't even need SSAO since the technique provides that effect for free and with much more realism.

What about my question, hm?

[jcabeleira]

Any chances of gettings source code?
And does it support indirect occlusion?

What about my question, hm?

Hehe. Sorry MrOMGWTF, I must have missed your questions.
I may release the source code along with a demo in the future, but for now the source code is a bit too messy for publishing.
What do you mean by indirect occlusion?

[MrOMGWTF]

Any chances of gettings source code?
And does it support indirect occlusion?

What about my question, hm?

Hehe. Sorry MrOMGWTF, I must have missed your questions.
I may release the source code along with a demo in the future, but for now the source code is a bit too messy for publishing.
What do you mean by indirect occlusion?

I mean that, there is a white wall, a green wall, and a blue wall occluding green wall. The green wall will be still illuminationg the white wall, but it shouldn't. Because the blue wall is occluding the green wall. Shouldn't you stop tracing at the first intersection you find? Also, you do cone tracing for each pixel, yeah?

@To users that -1 my post:
Let the hate flow through you.

Edited by MrOMGWTF, 21 September 2012 - 12:27 AM.

[jameszhao00]

...
I mean that, there is a white wall, a green wall, and a blue wall occluding green wall. The green wall will be still illuminationg the white wall, but it shouldn't. Because the blue wall is occluding the green wall. Shouldn't you stop tracing at the first intersection you find? Also, you do cone tracing for each pixel, yeah?

Cone tracing voxel mipmaps means you progressively lookat higher and higher level mipmaps. A higher level mipmap stores an occlusion distribution built from child voxels (and not concrete occluders). In his case, I think he's just storing average occlusion for the voxel, and not something that varies by direction/position/etc.

[jcabeleira]

I mean that, there is a white wall, a green wall, and a blue wall occluding green wall. The green wall will be still illuminationg the white wall, but it shouldn't. Because the blue wall is occluding the green wall. Shouldn't you stop tracing at the first intersection you find? Also, you do cone tracing for each pixel, yeah?

Cone tracing voxel mipmaps means you progressively lookat higher and higher level mipmaps. A higher level mipmap stores an occlusion distribution built from child voxels (and not concrete occluders). In his case, I think he's just storing average occlusion for the voxel, and not something that varies by direction/position/etc

jameszhao00, you're right. The voxels of the highest resolution mipmap are either completely opaque or completely transparent. But for the lower resolution voxels they usually are partially transparent due to the averaging of the voxels. Therefore, calculating occlusion is not as simple as looking for the first intersection (i.e. the first fully opaque voxel), when sampling the voxels we need to keep track of the accumulated opacity of all the voxels we have sampled so far. When the accumulated opacity reaches 1.0 we can stop tracing because the next samples would be completely occluded.

[Lightness1024]

Hey man, I read your thesis, nice work there ! too much specialized for your particular scenery (not scalable enough) in my opinion but still interesting.
notably the novel par where you use screen space sky visibility.
anyways, the sparse voxel octree will allow you to go down to 512*512*512 leaf precision which is better than your 128 precision, particularly for light leaks in interiors.
also, i havn't read a leak suppressor technique in Crassin's paper, but clearly it should be possible to apply the central difference scheme used in Crytek's LPV to try to suppress leaks. Careful though, I have implemented it and I can tell you it sometime causes severe artefacts depending on the empirical strength of the anisotropic value you choose. But the good part is that with a 512 division it should be much less noticeable.

About the specular, you should really try to fix it because it is one of the things that gives the most of the wow factor for this technique and it only requires ONE cone tracing where you alreay have 20 ! so it should not hurt your perf.

about the perf using sparse octree, you will get lesser performance, i'm sure you are familiar with the horrors Crassin had to cope with using global shared list of sleeping threads and multi passes until they are empty, not even talking about the bricks lateral two passes communication, this is just hell on earth and I feel like he must be a god of GPU debugging to got that working OK, or he is just bullshitting us with his paper.

the only idea is to get better precision by avoiding the otherwise necessary 9GB of data with a dense grid. remainder: in his paper he already needs 1GB this is huge.

one question for you: I didn't follow very thoroughly the part in Crassin's paper where he talks about light-view buffer to perform multi bounce GI, though it seems like a crucial part of his method, did you implement that at all ??