Jump to content
  • Advertisement

Imperfect Environment Maps

Hodgman

2606 views

In 22 our lighting environment is dominated by sunlight, however there are many small emissive elements everywhere.

A typical scene from '22'

What we want is for all these bright sunlit metal panels and the many emissive surfaces to be reflected off the vehicles. Being a high speed racing game, we need a technique with minimal performance impacts, and at the same time, we would like to avoid large baked data sets in order to support easy track editing within the game.

This week we got around to trying a technique presented 10 years ago for generating large numbers of shadow maps extremely quickly: Imperfect Shadow maps. In 2008, this technique was a bit ahead of its time -- as indicated by the performance data being measured on 640 x 480 image resolutions at 15 frames per second! :)
It is also a technique for generating shadows, for use in conjunction with a different lighting technique -- Virtual Point Lights.

In 22, we aren't using Virtual Point Lights or Imperfect Shadow Maps! However, in the paper they mention that ISMs can be used to add shadows to environment map lighting... By staying up too late and misreading this section, you could get the idea that you could use the ISM point-cloud rendering ideas to actually generate large numbers of approximate environment maps at low cost... so that's what we implemented :D

Our gameplay code already had access to a point cloud of the track geometry. This data set was generated by simply extracting the vertex positions from the visual mesh of the track - a portion is visualized below:
A portion of the track drawn as a point cloud

Next we somehow need to associate lighting values with each of these points... Typically for static environments, you would use a light baking system for this, which can spend a lot of time path-tracing the scene (or similar), before saving the results into the point cloud. To keep everything dynamic, we've instead taken inspiration from screen-space reflections. With SSR, the existing images that you're rendering anyway are re-used to provide data for reflection rays. We are reusing these images to compute lighting values for the points in our point cloud. After the HDR lighting is calculated, the point cloud is frustum culled and each point projected onto the screen (after a small random offset is applied to it). If the projected point is close in depth to the stored Z-buffer value at that screen pixel, then the lighting value at that pixel is transferred to the point cloud using a moving average. The random offsets and moving average allow many different pixels that are nearby the point to contribute to its color.
An example of the opaque HDR lighting

Over many frames, the point cloud will eventually be colored in now. If the lighting conditions change, then the point cloud will update as long as it appears on screen. This works well for a racing game, as the camera is typically looking ahead at sections of track that the car is about to drive into, allowing the point cloud for those sections to be updated with fresh data right before the car drives into those areas.

Now, if we take the points that are nearby a particular vehicle and project them onto a sphere, and then unwrap that sphere to 2D UV coordinates (at the moment, we are using a world-space octahedral unwrapping scheme, though spheremaps, hemispheres, etc are also applicable. Using view-space instead of world space could also help hide seams), then we get an image like below. Left is RGB components, right is Alpha, which encodes the solid angle that the point should've covered if we'd actually drawn them as discs/spheres, instead of as points.Nearby points have bright alpha, while distant points have darker alpha.
splat_rgb.png.ac3e423386cedf3beb99e9fa3bf535be.png splat_a.png.8476b2f59d33a092608d417d261baa73.png

We can then feed this data through a blurring filter. In the ISM paper they do a push-pull technique using mipmaps which I've yet to implement. Currently, this is a separable blur weighted by the alpha channel. After blurring, I wanted to keep track of which pixels initially had valid alpha values, so a sign bit is used to keep track of this. Pixels that contain data only thanks to blurring, store negative alpha values in them. Below, left is RGB, middle is positive alpha, right is negative alpha:
blur1.png.fcd78e3a75e0149b618cf599ca268bcf.pngblur1a.png.4eeac739dd5787434cefb756df58ef5c.pngblur1ai.png.c6271914e0ff2505c7f891d798d122c9.png Pass 1 - horizontal

blur2.png.3884b49f7c8801344b53c21a048c2a7c.pngblur2a.png.b7709b3dbaed22168eb6bc1aae28e394.pngblur2an.png.abb6c5e9775061759b699da0263fa7be.png Pass 2 - vertical

blur3.png.71d42f5d354af88f0e096b139b3e5502.pngblur3a.png.e96800a12752c315d62304876c13f225.pngblur3an.png.8e57d231bed81602e146bf483ec1998a.png Pass three - diagonal

blur4.png.2962c027d48af4c7e95ca16862e83c56.png blur4aa.png.c9007144fe7960f5b7b8bd30ab9e7f12.pngPass four - other diagonal, and alpha mask generation

In the final blurring pass, the alpha channel is converted to an actual/traditional alpha value (based on artist-tweakable parameters), which will be used to blend with the regular lighting probes.
A typical two-axis separable blur creates distinctive box shapes, but repeating the process with a 45º rotation produces hexagonal patterns instead, which are much closer to circular :)
The result of this is a very approximate, blobby, kind-of-correct environment map, which can be used for image based lighting. After this step we calculate a mip-chain using standard IBL practices for roughness based lookups.

The big question, is how much does it cost though? On my home PC with a NVidia GTX780 (not a very modern GPU now!), the in-game profiler showed ~45µs per vehicle to create a probe, and ~215µs to copy the screen-space lighting data to the point cloud.
timing.thumb.png.a8b273a288cbe4de03a887f4061771f4.png

And how does it look? When teams capture sections of our tracks, emissive elements show that team's color. Below you can see a before/after comparison, where the green team color is now actually reflected on our vehicles :D

off.thumb.jpg.c744e738e0d088e19e4cbb220399ca4e.jpg

on.thumb.jpg.db2710ca1677096e087eda6478b05484.jpg

 

In those screens you can see the quick artist tweaking GUI on the right side. I have to give a shout out to Omar's Dear ImGui project, which we use to very quickly add these kinds of developer-GUIs.
tweaking.jpg.c6a2325038f857b895204741309578d0.jpg

  • Point Radius - the size of the virtual discs that the points are drawn as (used to compute the pre-blurring alpha value, dictating the blur radius).
  • Gather Radius - the random offset added to each point (in meters) before it's projected to the screen to try and collect some lighting information.
  • Depth Threshold - how close the projected point needs to be to the current Z-Buffer value in order to be able to collect lighting info from that piixel.
  • Lerp Speed - a weight for the moving average.
  • Alpha range - After blurring, scales how softly alpha falls off at the edge of the blurred region.
  • Max Alpha - A global alpha multiplier for these dynamic probes - e.g. 0.75 means that 25% of the normal lighting probes will always be visible.

 



0 Comments


Recommended Comments

Quote

Over many frames, the point cloud will eventually be colored in now. If the lighting conditions change, then the point cloud will update as long as it appears on screen. This works well for a racing game, as the camera is typically looking ahead at sections of track that the car is about to drive into, allowing the point cloud for those sections to be updated with fresh data right before the car drives into those areas.

I love how much this technique fits your game! You can get so creative with non-general solutions :).

Share this comment


Link to comment
On 8/10/2018 at 8:40 PM, Mussi said:

I love how much this technique fits your game! You can get so creative with non-general solutions :).

Yeah, for something like a first person shooter it wouldn't fit as well, because the point clouds would probably exhibit a lot of light leaking in typical indoor scenes. Our track surfaces are typically full of holes anyway, so a bit of light leaking is actually desirable. I guess you could fight against leaking by aggressively using the push-pull method on the depth buffer, as in the ISM paper. Maybe generate depth maps using the ISM technique first (as a Z-pre-pass) and then generate env-maps using only points that pass the depth test...

Ill try to post another update soon as I refine the technique a bit more, and capture a video of it in motion. As the car races past each point, a blobby reflection passes over the car, which actually really adds to the feeling of speed! :D

Seeing that it's completely dynamic, another enhancement I want to look into is attaching a point cloud onto each vehicle as well, so that the vehicles can reflect off of each other.

Share this comment


Link to comment

Great article and really inspiring.

I too have been reading some old papers, and I think there are interesting ideas that sometimes get forgotten, or not revisited with modern hardware.

Good stuff.

Share this comment


Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Advertisement
  • Advertisement
  • Blog Entries

  • Similar Content

    • By lukash
      Hello everybody! I decided to write a graphics engine, the killer of Unity and Unreal. If anyone interested and have free time, join. High-level render is based on low-level OpenGL 4.5 and DirectX 11. Ideally, there will be PBR, TAA, SSR, SSAO, some variation of indirect light algorithm, support for multiple viewports and multiple cameras. The key feature is COM based (binary compatibility is needed). Physics, ray tracing, AI, VR will not. I grabbed the basic architecture from the DGLE engine. The editor will be on Qt (https://github.com/fra-zz-mer/RenderMasterEditor). Now there is a buildable editor. The main point of the engine is the maximum transparency of the architecture and high-quality rendering. For shaders, there will be no new language, everything will turn into defines.
    • By Doggolainen
      Hi,

      I wish to generate light map textures for objects in my scene containing ambient occlusion and global illumination.

      Currently I can do this in 3D Max, but it is taking too long on my single computer for this to be feasible. 

      Does anyone know any rendering farms that supports this (light baking)? I checked "Rebusfarm" and they did not support this feature.

      Any help appreciated,
      Doggolainen
    • By Nafei
      Tired of working on games that never get released? Then you're at the right place!
      Game Description
      Titanforged Entertainment is currently working on a 3D multiplayer game containing several board games. The environment of the game is something of an RPG tavern/potion/scroll shop and the game style is stylized with hand-painted texturing. You can think of the art style as in World of Warcraft or Fortnite or Warcraft 3, just to make it more clear. The environment is ONLY used for the looks. Meaning, the environment is only there as a lobby screen & waiting area. There will be a cinematic screen in the tavern and we will make it feel cozy and nice when people are in the menu's and waiting for a game. At the bottom of this post, you can see some screenshots from the game at its current state. Note that the screenshots are taken from the 3D software and are still not sculpted or textured which is why they look simple and plain. They will look more detailed at a later stage in development.
      The game contains multiple board (and card) games where people play against each other in for example Chess, Ludo, Snakes & Ladders, Uno, Poker, Scrabble, Drawing/Guessing and more. The boards we are developing will also be in full 3D. For example, Chess would have a castle/monastery environment with the pieces designed as character pieces. In this game you can play to gain ranks and medals. You earn coins from playing, which you can use to buy skins and different pieces in the coin shop. There are more features in the game but these are just some of them.
       
      Programmer Needed
      Titanforged is looking for a highly motivated and talented Programmer. You will be provided all the assets from the artists and it is your responsibility to bring life to the game. If you’re up to the challenge, we want to hear from you!
      Description:
      - We do not require a specific programming language. It is up to you, as long as everything is written nicely and the game is working as it's supposed to.
      - You should be able to program the game as a solo project. We only try to recruit members if necessary.
      - You need to have a passion for board games.
      - You are someone who finishes projects. Please do not apply if you have a hard time finishing a game. This will at least take 3-5 months before release.
      - You are confident when it comes to Networking and Multiplayer programming. The game will be released on Steam.
      - You should know how to build good security for the game to avoid cheating and 3rd party software.
      - You are able to spend at least 15 hours a week on the project. If you can spend more, that's great. Currently we are spending 35 hours a week on the art.
      - You are 18+ years old (Legal contract reasons)
       
      Big plusses (NOT REQUIRED):
      - You have previously released a game before.
      - You have a degree in your field.
      Notice that if you don't have a solid portfolio, you will be tested by creating a chess game. We will provide you the board, pieces and textures. You will "only" have to write the code. There is no AI / singleplayer programming.
       
      Payment
      The payment for the project is revenue share. This will be our first game to release on Steam and therefore we are currently all working for free. When the game is released on Steam, payment will be divided between each member as promised in the contract you will sign before working on the game.
      Your revenue share for the project will be 35% and is not negotiable. We predict high income for this game when it is released on Steam. A lot of the money will be saved in Titanforged Entertainment so that we can create a better next-game and afford anything we need. Our goal is to progress such that we can do this full time and pay our members monthly.
       
      Contact Information
      If you're interested in joining us for this project and hopefully for the future projects, you can add me on discord Nafei#9413 or write an email to Nafei@titanforged.net
       
      Screenshots




    • By INTwindwolf
      THE PROJECT

      INT is a 3D Sci-fi RPG with a strong emphasis on story, role playing, and innovative RPG features such as randomized companions. The focus is on the journey through a war-torn world with fast-paced combat against hordes of enemies. The player must accomplish quests like a traditional RPG, complete objectives, and meet lively crew members who will aid in the player's survival. Throughout the game you can side and complete missions through criminal cartels, and the two major combatants, the UCE and ACP, of the Interstellar Civil War.
      Please note that all of our current positions are remote work. You will not be required to travel.
      For more information about us, follow the links listed below.
      INT Official website
      IndieDB page
      Also follow social media platforms for the latest news regarding our projects.
      Facebook
      Twitter
      3D ENVIRONMENT MODELER
      We are looking for a 3D Modeller to create and polish 3D models for the game. You will be collaborating with the team in creating 3D models that meet polygon-count and texture quality requirements. You will also need to be able to enact upon instructions from the Art team Lead and the Project Lead.
      Your duties include:
      Create 3D models conforming to the polygon-count requirements. Skinning and un-wrapping of models created. Make adjustments to created models when required. REQUIREMENTS
      To be successful in this position, following requirements apply:
      Have working knowledge of 3D modelling suites. Understand import/export requirements for Unity Engine integration. Excellent self-management skills. Excellent attention to detail. Excellent communication skills. OTHER OPEN POSITIONS
      Unity Engine Programmer
      Website Manager
      3D Character Modeller
      REVENUE-SHARE
      This is a great opportunity to get into the game development industry. Being an Indie team we do not have the creative restrictions often imposed by publishers or other third parties. We are extremely conscientious of our work and continuously uphold a high level of quality throughout our project.
      We are unable to offer wages or per-item payments at this time. However revenue-sharing from crowd-funding is offered to team members who contribute 15-20 hours per week to company projects, as well as maintain constant communication and adhere to deadlines. Your understanding is dearly appreciated.
      TO APPLY
      Please send your Cover Letter, CV, Portfolio (if applicable), and other relevant documents/information to this email: JohnHR@int-game.net
      Thank you for your time! Please feel free to contact me via the email provided should you have any questions or are interested to apply for this position. We look forward to hearing from you!
      John Shen
      HR Lead
      Starboard Games LLC
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!