Jump to content
  • Advertisement

Imperfect Environment Maps



In 22 our lighting environment is dominated by sunlight, however there are many small emissive elements everywhere.

A typical scene from '22'

What we want is for all these bright sunlit metal panels and the many emissive surfaces to be reflected off the vehicles. Being a high speed racing game, we need a technique with minimal performance impacts, and at the same time, we would like to avoid large baked data sets in order to support easy track editing within the game.

This week we got around to trying a technique presented 10 years ago for generating large numbers of shadow maps extremely quickly: Imperfect Shadow maps. In 2008, this technique was a bit ahead of its time -- as indicated by the performance data being measured on 640 x 480 image resolutions at 15 frames per second! :)
It is also a technique for generating shadows, for use in conjunction with a different lighting technique -- Virtual Point Lights.

In 22, we aren't using Virtual Point Lights or Imperfect Shadow Maps! However, in the paper they mention that ISMs can be used to add shadows to environment map lighting... By staying up too late and misreading this section, you could get the idea that you could use the ISM point-cloud rendering ideas to actually generate large numbers of approximate environment maps at low cost... so that's what we implemented :D

Our gameplay code already had access to a point cloud of the track geometry. This data set was generated by simply extracting the vertex positions from the visual mesh of the track - a portion is visualized below:
A portion of the track drawn as a point cloud

Next we somehow need to associate lighting values with each of these points... Typically for static environments, you would use a light baking system for this, which can spend a lot of time path-tracing the scene (or similar), before saving the results into the point cloud. To keep everything dynamic, we've instead taken inspiration from screen-space reflections. With SSR, the existing images that you're rendering anyway are re-used to provide data for reflection rays. We are reusing these images to compute lighting values for the points in our point cloud. After the HDR lighting is calculated, the point cloud is frustum culled and each point projected onto the screen (after a small random offset is applied to it). If the projected point is close in depth to the stored Z-buffer value at that screen pixel, then the lighting value at that pixel is transferred to the point cloud using a moving average. The random offsets and moving average allow many different pixels that are nearby the point to contribute to its color.
An example of the opaque HDR lighting

Over many frames, the point cloud will eventually be colored in now. If the lighting conditions change, then the point cloud will update as long as it appears on screen. This works well for a racing game, as the camera is typically looking ahead at sections of track that the car is about to drive into, allowing the point cloud for those sections to be updated with fresh data right before the car drives into those areas.

Now, if we take the points that are nearby a particular vehicle and project them onto a sphere, and then unwrap that sphere to 2D UV coordinates (at the moment, we are using a world-space octahedral unwrapping scheme, though spheremaps, hemispheres, etc are also applicable. Using view-space instead of world space could also help hide seams), then we get an image like below. Left is RGB components, right is Alpha, which encodes the solid angle that the point should've covered if we'd actually drawn them as discs/spheres, instead of as points.Nearby points have bright alpha, while distant points have darker alpha.
splat_rgb.png.ac3e423386cedf3beb99e9fa3bf535be.png splat_a.png.8476b2f59d33a092608d417d261baa73.png

We can then feed this data through a blurring filter. In the ISM paper they do a push-pull technique using mipmaps which I've yet to implement. Currently, this is a separable blur weighted by the alpha channel. After blurring, I wanted to keep track of which pixels initially had valid alpha values, so a sign bit is used to keep track of this. Pixels that contain data only thanks to blurring, store negative alpha values in them. Below, left is RGB, middle is positive alpha, right is negative alpha:
blur1.png.fcd78e3a75e0149b618cf599ca268bcf.pngblur1a.png.4eeac739dd5787434cefb756df58ef5c.pngblur1ai.png.c6271914e0ff2505c7f891d798d122c9.png Pass 1 - horizontal

blur2.png.3884b49f7c8801344b53c21a048c2a7c.pngblur2a.png.b7709b3dbaed22168eb6bc1aae28e394.pngblur2an.png.abb6c5e9775061759b699da0263fa7be.png Pass 2 - vertical

blur3.png.71d42f5d354af88f0e096b139b3e5502.pngblur3a.png.e96800a12752c315d62304876c13f225.pngblur3an.png.8e57d231bed81602e146bf483ec1998a.png Pass three - diagonal

blur4.png.2962c027d48af4c7e95ca16862e83c56.png blur4aa.png.c9007144fe7960f5b7b8bd30ab9e7f12.pngPass four - other diagonal, and alpha mask generation

In the final blurring pass, the alpha channel is converted to an actual/traditional alpha value (based on artist-tweakable parameters), which will be used to blend with the regular lighting probes.
A typical two-axis separable blur creates distinctive box shapes, but repeating the process with a 45º rotation produces hexagonal patterns instead, which are much closer to circular :)
The result of this is a very approximate, blobby, kind-of-correct environment map, which can be used for image based lighting. After this step we calculate a mip-chain using standard IBL practices for roughness based lookups.

The big question, is how much does it cost though? On my home PC with a NVidia GTX780 (not a very modern GPU now!), the in-game profiler showed ~45µs per vehicle to create a probe, and ~215µs to copy the screen-space lighting data to the point cloud.

And how does it look? When teams capture sections of our tracks, emissive elements show that team's color. Below you can see a before/after comparison, where the green team color is now actually reflected on our vehicles :D




In those screens you can see the quick artist tweaking GUI on the right side. I have to give a shout out to Omar's Dear ImGui project, which we use to very quickly add these kinds of developer-GUIs.

  • Point Radius - the size of the virtual discs that the points are drawn as (used to compute the pre-blurring alpha value, dictating the blur radius).
  • Gather Radius - the random offset added to each point (in meters) before it's projected to the screen to try and collect some lighting information.
  • Depth Threshold - how close the projected point needs to be to the current Z-Buffer value in order to be able to collect lighting info from that piixel.
  • Lerp Speed - a weight for the moving average.
  • Alpha range - After blurring, scales how softly alpha falls off at the edge of the blurred region.
  • Max Alpha - A global alpha multiplier for these dynamic probes - e.g. 0.75 means that 25% of the normal lighting probes will always be visible.



Recommended Comments


Over many frames, the point cloud will eventually be colored in now. If the lighting conditions change, then the point cloud will update as long as it appears on screen. This works well for a racing game, as the camera is typically looking ahead at sections of track that the car is about to drive into, allowing the point cloud for those sections to be updated with fresh data right before the car drives into those areas.

I love how much this technique fits your game! You can get so creative with non-general solutions :).

Share this comment

Link to comment
On 8/10/2018 at 8:40 PM, Mussi said:

I love how much this technique fits your game! You can get so creative with non-general solutions :).

Yeah, for something like a first person shooter it wouldn't fit as well, because the point clouds would probably exhibit a lot of light leaking in typical indoor scenes. Our track surfaces are typically full of holes anyway, so a bit of light leaking is actually desirable. I guess you could fight against leaking by aggressively using the push-pull method on the depth buffer, as in the ISM paper. Maybe generate depth maps using the ISM technique first (as a Z-pre-pass) and then generate env-maps using only points that pass the depth test...

Ill try to post another update soon as I refine the technique a bit more, and capture a video of it in motion. As the car races past each point, a blobby reflection passes over the car, which actually really adds to the feeling of speed! :D

Seeing that it's completely dynamic, another enhancement I want to look into is attaching a point cloud onto each vehicle as well, so that the vehicles can reflect off of each other.

Share this comment

Link to comment

Great article and really inspiring.

I too have been reading some old papers, and I think there are interesting ideas that sometimes get forgotten, or not revisited with modern hardware.

Good stuff.

Share this comment

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Advertisement
  • Advertisement
  • Blog Entries

  • Similar Content

    • By Brandon Sharp
      This is a project I've been working on for awhile now. I'd save over all going on around a year. I did the Machbot 2.0 all from scratch including the textures. I spent countless hours trying to figure out how to get the models from Twisted Metal. I finally figured out how to manually extract the mesh. But the only problem was it was not UV mapped. So i pretty much had to go back in and remap everything. Which wasn't hard but the assembling of the model itself was a challenge. I did the best I could at placing stuff where it goes I'm sure there are things that are incorrect. All in all it was for this one render. Not sure if my models can be used as game assets but i do want to eventually make this into a fighting game. Both vehicles and bots. Let me know what you think and thank your for checking out my work. 

    • By ChenMo
      Hi, guys.
      I am developing a path tracing baking renderer now, which is based on OpenGL and OpenRL. It can bake some scenes e.g. the following, I am glad it can bake bleeding diffuse color. : )

      I store the irradiance directly, like this. Albedo * diffuse color has already come into irradiance calculation when baking, both direct and indirect together.

      After baking, In OpenGL fragment shader, I use the light map directly.

      I think I have got something wrong, due to most game engines don't do like this.
      Which kind of data should I choose to store into the light maps? I need diffuse only.
      Thanks in advance!
    • By jb-dev
      This is a picture of a yet to be completed bank. The idea is to generate simple geometries to fill the space between the floor and the ceiling. It's done like that because I can set the wall height of the generated level on the fly, so I need something generic and flexible.
    • By Iris_Technologies
      I had some doubts about hex formats(assembler output) and linkers:

      1.- So, I disassembly a raw binary(no ELF, PE, etc... headers) X64 assembly code and i got that result:

      0: 66 89 c8 mov ax,cx 3: e8 00 00 00 00 call 8 <gh> 0000000000000008 <gh>: 8: 66 89 c2 mov dx,ax I understand how Byte Offset works('66' is the byte ID 0, '89' is 1, 'c8' is 2 and on 3 the call instruction starts(that is why '3:' is there)) but, by that logic, shouldn't 'call gh' be translated to 'e8 00 00 00 00 00 00 00 08' instead of 'e8 00 00 00 00' since the byte offset of the first instruction of gh, which is 'mov   dx, ax' is 8 and the output is 64 bits?
      2.- Using the example of above, if endianness is little end., how the assembler would swap the bytes, by each instruction? Like:

      Original, no endiannes { 66 89 c8 e8 00 00 00 00(in case that would be correct and i'm wrong in the question 1.-) 66 89 c2 } to { c8 89 66 00 00 00 00 e8 c2 89 66 } 3.- And then, the big end. would be like the original, without endiannes, code of the question 2.-?
      4.- Suppose that i mark gh as .globl, then, would the assembler create a map table file where gh is in 'e8 00 00 00 00'(again, in case that would be correct and i'm wrong in question 1.-), and linker will look into such map file, and if another object file calls gh, the linker will then translate call gh as either 'e8 00 00 00 00'?
    • By totosi32
      I know that is a noob question but, between OpenGL 2.0 and OpenGL ES 2.0, which one got better performance to desktop and/or mobile devices?
      I have read in somewhere that the performance opengl is based on code but some games we can compare oepngl version performances, so idk.
      Which one of both use less CPU & GPU/ got better performance?

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!