Jump to content
  • Advertisement

Imperfect Environment Maps

Hodgman

2202 views

In 22 our lighting environment is dominated by sunlight, however there are many small emissive elements everywhere.

A typical scene from '22'

What we want is for all these bright sunlit metal panels and the many emissive surfaces to be reflected off the vehicles. Being a high speed racing game, we need a technique with minimal performance impacts, and at the same time, we would like to avoid large baked data sets in order to support easy track editing within the game.

This week we got around to trying a technique presented 10 years ago for generating large numbers of shadow maps extremely quickly: Imperfect Shadow maps. In 2008, this technique was a bit ahead of its time -- as indicated by the performance data being measured on 640 x 480 image resolutions at 15 frames per second! :)
It is also a technique for generating shadows, for use in conjunction with a different lighting technique -- Virtual Point Lights.

In 22, we aren't using Virtual Point Lights or Imperfect Shadow Maps! However, in the paper they mention that ISMs can be used to add shadows to environment map lighting... By staying up too late and misreading this section, you could get the idea that you could use the ISM point-cloud rendering ideas to actually generate large numbers of approximate environment maps at low cost... so that's what we implemented :D

Our gameplay code already had access to a point cloud of the track geometry. This data set was generated by simply extracting the vertex positions from the visual mesh of the track - a portion is visualized below:
A portion of the track drawn as a point cloud

Next we somehow need to associate lighting values with each of these points... Typically for static environments, you would use a light baking system for this, which can spend a lot of time path-tracing the scene (or similar), before saving the results into the point cloud. To keep everything dynamic, we've instead taken inspiration from screen-space reflections. With SSR, the existing images that you're rendering anyway are re-used to provide data for reflection rays. We are reusing these images to compute lighting values for the points in our point cloud. After the HDR lighting is calculated, the point cloud is frustum culled and each point projected onto the screen (after a small random offset is applied to it). If the projected point is close in depth to the stored Z-buffer value at that screen pixel, then the lighting value at that pixel is transferred to the point cloud using a moving average. The random offsets and moving average allow many different pixels that are nearby the point to contribute to its color.
An example of the opaque HDR lighting

Over many frames, the point cloud will eventually be colored in now. If the lighting conditions change, then the point cloud will update as long as it appears on screen. This works well for a racing game, as the camera is typically looking ahead at sections of track that the car is about to drive into, allowing the point cloud for those sections to be updated with fresh data right before the car drives into those areas.

Now, if we take the points that are nearby a particular vehicle and project them onto a sphere, and then unwrap that sphere to 2D UV coordinates (at the moment, we are using a world-space octahedral unwrapping scheme, though spheremaps, hemispheres, etc are also applicable. Using view-space instead of world space could also help hide seams), then we get an image like below. Left is RGB components, right is Alpha, which encodes the solid angle that the point should've covered if we'd actually drawn them as discs/spheres, instead of as points.Nearby points have bright alpha, while distant points have darker alpha.
splat_rgb.png.ac3e423386cedf3beb99e9fa3bf535be.png splat_a.png.8476b2f59d33a092608d417d261baa73.png

We can then feed this data through a blurring filter. In the ISM paper they do a push-pull technique using mipmaps which I've yet to implement. Currently, this is a separable blur weighted by the alpha channel. After blurring, I wanted to keep track of which pixels initially had valid alpha values, so a sign bit is used to keep track of this. Pixels that contain data only thanks to blurring, store negative alpha values in them. Below, left is RGB, middle is positive alpha, right is negative alpha:
blur1.png.fcd78e3a75e0149b618cf599ca268bcf.pngblur1a.png.4eeac739dd5787434cefb756df58ef5c.pngblur1ai.png.c6271914e0ff2505c7f891d798d122c9.png Pass 1 - horizontal

blur2.png.3884b49f7c8801344b53c21a048c2a7c.pngblur2a.png.b7709b3dbaed22168eb6bc1aae28e394.pngblur2an.png.abb6c5e9775061759b699da0263fa7be.png Pass 2 - vertical

blur3.png.71d42f5d354af88f0e096b139b3e5502.pngblur3a.png.e96800a12752c315d62304876c13f225.pngblur3an.png.8e57d231bed81602e146bf483ec1998a.png Pass three - diagonal

blur4.png.2962c027d48af4c7e95ca16862e83c56.png blur4aa.png.c9007144fe7960f5b7b8bd30ab9e7f12.pngPass four - other diagonal, and alpha mask generation

In the final blurring pass, the alpha channel is converted to an actual/traditional alpha value (based on artist-tweakable parameters), which will be used to blend with the regular lighting probes.
A typical two-axis separable blur creates distinctive box shapes, but repeating the process with a 45º rotation produces hexagonal patterns instead, which are much closer to circular :)
The result of this is a very approximate, blobby, kind-of-correct environment map, which can be used for image based lighting. After this step we calculate a mip-chain using standard IBL practices for roughness based lookups.

The big question, is how much does it cost though? On my home PC with a NVidia GTX780 (not a very modern GPU now!), the in-game profiler showed ~45µs per vehicle to create a probe, and ~215µs to copy the screen-space lighting data to the point cloud.
timing.thumb.png.a8b273a288cbe4de03a887f4061771f4.png

And how does it look? When teams capture sections of our tracks, emissive elements show that team's color. Below you can see a before/after comparison, where the green team color is now actually reflected on our vehicles :D

off.thumb.jpg.c744e738e0d088e19e4cbb220399ca4e.jpg

on.thumb.jpg.db2710ca1677096e087eda6478b05484.jpg

 

In those screens you can see the quick artist tweaking GUI on the right side. I have to give a shout out to Omar's Dear ImGui project, which we use to very quickly add these kinds of developer-GUIs.
tweaking.jpg.c6a2325038f857b895204741309578d0.jpg

  • Point Radius - the size of the virtual discs that the points are drawn as (used to compute the pre-blurring alpha value, dictating the blur radius).
  • Gather Radius - the random offset added to each point (in meters) before it's projected to the screen to try and collect some lighting information.
  • Depth Threshold - how close the projected point needs to be to the current Z-Buffer value in order to be able to collect lighting info from that piixel.
  • Lerp Speed - a weight for the moving average.
  • Alpha range - After blurring, scales how softly alpha falls off at the edge of the blurred region.
  • Max Alpha - A global alpha multiplier for these dynamic probes - e.g. 0.75 means that 25% of the normal lighting probes will always be visible.

 



0 Comments


Recommended Comments

Quote

Over many frames, the point cloud will eventually be colored in now. If the lighting conditions change, then the point cloud will update as long as it appears on screen. This works well for a racing game, as the camera is typically looking ahead at sections of track that the car is about to drive into, allowing the point cloud for those sections to be updated with fresh data right before the car drives into those areas.

I love how much this technique fits your game! You can get so creative with non-general solutions :).

Share this comment


Link to comment
On 8/10/2018 at 8:40 PM, Mussi said:

I love how much this technique fits your game! You can get so creative with non-general solutions :).

Yeah, for something like a first person shooter it wouldn't fit as well, because the point clouds would probably exhibit a lot of light leaking in typical indoor scenes. Our track surfaces are typically full of holes anyway, so a bit of light leaking is actually desirable. I guess you could fight against leaking by aggressively using the push-pull method on the depth buffer, as in the ISM paper. Maybe generate depth maps using the ISM technique first (as a Z-pre-pass) and then generate env-maps using only points that pass the depth test...

Ill try to post another update soon as I refine the technique a bit more, and capture a video of it in motion. As the car races past each point, a blobby reflection passes over the car, which actually really adds to the feeling of speed! :D

Seeing that it's completely dynamic, another enhancement I want to look into is attaching a point cloud onto each vehicle as well, so that the vehicles can reflect off of each other.

Share this comment


Link to comment

Great article and really inspiring.

I too have been reading some old papers, and I think there are interesting ideas that sometimes get forgotten, or not revisited with modern hardware.

Good stuff.

Share this comment


Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Advertisement
  • Advertisement
  • Blog Entries

  • Similar Content

    • By sevenfold1
      I'm looking for a open source PBR rendering engine that I can use.  Basic requirements are Windows (C/C++) and free to use type license.

      The first two hits I get on Google are:

      Filament
      https://github.com/google/filament/

      LuxCoreRender
      https://luxcorerender.org/

      Does anybody have any experience using any of these, or do you recommend something else that's better?
      Thanks.
       
    • By jb-dev
      This is a picture of a regular room. 
      Each rooms has a chance to have props with a certain layout.
      If a rock is spawn the player can try to break it by exploding it. These can spawn common collectibles like money and such...
      There are many different type of layout possible, including a totally random one.
      The generator simply picks a shape, an anchor position and the type of prop to be generated.
      The process itself is simply a 2D boolean array. We simply draw on that array like we would in a bitmap using traditional image manipulation algorithms.
      Once the array is set, then we simply add a given prop for each true cell of the array 
      We then finally remove every props that are either outside its room or obstructing an entry/exit point.
      Right now, only two types of props can be generated, although the algorithm itself is kinda flexible.
    • By Josheir
      This is a follow up to a previous post.  MrHallows had asked me to post the project, so I am going to with a new fresh thread so that I can get the most needed help. 
      I have put the class in the main .cpp to simplify for your debugging purposes.  My error is :  
      C1189 #error: OpenGL header already included, remove this include, glad already provides it  
      I tried adding : #define GLFW_INCLUDE_NONE, and tried adding this as a preprocessor definitions too. I also tried to change the #ifdef - #endif, except I just couldn't get it working. The code repository URL is :
      https://github.com/Joshei/GolfProjectRepo/tree/combine_sources/GOLFPROJ
       
      The branch is : combine_sources
      The Commit ID is: a4eaf31
      The files involved are : shader_class.cpp,  glad.h, glew.h
      glad1.cpp was also in my project, I removed it to try to solve this problem.
       
       
      Here is the description of the problem at hand:
      Except for glcolor3f and glRasterPos2i(10,10); the code works without glew.h.  When glew is added there is only a runtime error (that is shown above.) 
       
      I could really use some exact help.  You know like, "remove the include for gl.h on lines 50, 65, and 80.  Then delete the code at line 80 that states..."
       
      I hope that this is not to much to ask for, I really want to win at OpenGL.  If I can't get help I could use a much larger file to display the test values or maybe it's possible to write to an open file and view the written data as it's outputted.
       
      Thanks in advance,
      Josheir
    • By Swartz27
      I'm looking to create a small game engine, though my main focus is the renderer.
      I'm trying to decide which of these techniques I like better: Deferred Texturing or Volume Tiled Forward Shading ( https://github.com/jpvanoosten/VolumeTiledForwardShading ). Which would you choose,if not something else?
      Here are my current goals:
      I want to keep middleware to a minimum I want to use either D3D12 or Vulkan. However I understand D3D best so that is where I'm currently siding. I want to design for today's high-end GPU's and not worry too much about compatibility, as I'm assuming this is going to take a long time anyway I'm only interested in real-time ray-tracing if/when it can be done without an RTX-enabled card PBR pipeline that DOES NOT INCLUDE METALNESS. I feel there are better ways of doing this (hint: I like cavity maps) I want dynamic resolution scaling. I know it's simply a form of super-sampling, but I haven't found many ideal sources that explain super-sampling in a way that I would understand. I don't want to use any static lighting. I have good reasons which I'd be happy to explain. So I guess what I'm asking you fine people, is that if time were not a concern, or money, what type of renderer would you write and more importantly "WHY"?
      Thank you for your time.
    • By jb-dev
      This is a nicer version of the spa room, plants included!
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!