Jump to content
  • Advertisement
turanszkij

3D Blending local-global envmaps

Recommended Posts

In the past I developed a reflection probe/environment map system with PBR pipeline in mind. It had support for global and local environment maps. The locals only affect the scene inside their boundaries, which are OBBs essentially. There can be several local probes placed inside the scene and they are blended on top of each other (if they are intersecting). In a deferred renderer they would be rendered as boxes and they would sample from the Gbuffer the same time as lights are rendered. If there are only local probes, then there could be areas where reflection information is missing, which is undesirable with PBR rendering, as metals not receive any color. In those areas we should fall back to something, but here comes my question: what? I experimented with some solutions but I found them not really appealing:

  1. Fall back to sky color: This could work in outside areas, but indoors it will just break hard.
  2. Fall back to the probe closest to the camera: With some blending it could work so that it avoids "popping", but far away reflections will also change with the camera position, can be very distracting.
  3. Fall back to the probe closest to pixel world position: Has several problems
    1. How to determine per pixel which probe is closest?
    2. We should really retrieve 3 closest probes and blend them
    3. But 3 cubemap samples, distances, blending maybe even in object rendering shader?
    4. Maybe use 2 closest probes to camera and use those to blend? This produces straight line between affecting boundaries, but will result in popping when new probe gets close to camera which wasn't.
  4. Fall back to closest cubemap per object? Seems nice on static objects, but this can also break easily.

Does anyone have other solutions that they use? I would like to have a general solution to this problem.

Share this post


Link to post
Share on other sites
Advertisement

Can you, instead, ensure each piece of surface is affected by at least one probe? This way you could solve the problem offline by extending probe volumes.

3.2.: I think to do this in a robust manner, you need 4 closest probes from a voroni tetrahedralization. But this approach could also replace your current OBB approach completely, so not just a 'fallback'.

Eventually build a very low resolution volume grid of pointers to existing probes?

 

Share this post


Link to post
Share on other sites
39 minutes ago, JoeJ said:

Can you, instead, ensure each piece of surface is affected by at least one probe? This way you could solve the problem offline by extending probe volumes.

3.2.: I think to do this in a robust manner, you need 4 closest probes from a voroni tetrahedralization. But this approach could also replace your current OBB approach completely, so not just a 'fallback'.

Eventually build a very low resolution volume grid of pointers to existing probes?

 

Thank you for ideas. I wanted to avoid using many local probes to fill every surface, because they are heavier to compute than global probes (they need a ray-trace into OBB in pixel shader). The parallax effect which they bring and why I even use local probes is hardly visible on rough surfaces, so those would benefit in having just a global probe computed for them.

I also want to avoid offline methods, I want something fully real time (this is for a hobby project). But a grid of probe pointers sounds like a neat idea. I just implemented a new system for them and storing the probes inside a texturecubearray, so indexing would be easy even in a forward+ rendering object shader. A problem with a regular grid is that the local probes are projected as boxes so the box sides would be visible in the reflections and would be distracting if the probes don't fit the room. I'll be toying with this idea though.

Share this post


Link to post
Share on other sites

Yeah as above you can make it into an art/content problem :)

In development builds, render any pixel not covered by a probe in flashing pink so that content creators can see the error. 

You can make the OBB/parallax correction feature optional, to allow "localised global" probes. You might bathe a whole building in a non-parallax probe, then add a few parallax probes to important rooms only. 

Going in other directions though, you can fall back to other data sets besides probes. If you have lightmaps, you can fall back to them (I've done runtime lightmap baking on a PS3/360 game once :) ), or AO bakes tinted with ambient colours. We've often defined ambience on a spherical domain with three colours - up/side/down, weighted by sat(n.y), 1-abs(n.y) and sat (1-n.y), where (n=world normal and y=up). You could define these in particular regions the same way that you define your probes currently, for cases where an artist wants to fix the global sky leaking in, but doesn't want to add the cost of another runtime probe. 

Share this post


Link to post
Share on other sites

In your specific case I would go with a very simple solution:

1. Set of global probes. Nearest one covers entire scene.

2. On top of that blend your local probes.

Share this post


Link to post
Share on other sites
14 hours ago, turanszkij said:

I also want to avoid offline methods, I want something fully real time

But the probe positions are still static data i guess? What limitations could you expect from an offline method? 

You could use a voxelization of the scene, flag voxels inside probe OOBs and use remaining unlit voxels to extend closest probe OOB. You already have voxelization, and it could be realtime or progressively updated if really needed.

Share this post


Link to post
Share on other sites
18 hours ago, knarkowicz said:

In your specific case I would go with a very simple solution:

1. Set of global probes. Nearest one covers entire scene.

2. On top of that blend your local probes.

I mentioned that approach, what I dislike about it is that far away objects will have a very wrong reflection, and also when the closest envmap changes the entire scene gets re-lighted. But I will probably go with this one as this method can be implemented with no hard popping when a new envmap gets closest.

12 hours ago, JoeJ said:

But the probe positions are still static data i guess? What limitations could you expect from an offline method? 

You could use a voxelization of the scene, flag voxels inside probe OOBs and use remaining unlit voxels to extend closest probe OOB. You already have voxelization, and it could be realtime or progressively updated if really needed.

The probe locations are mostly static, but they can be grabbed in the editor and moved, and be refreshed instantly. About the voxelization, that is an interesting idea. Though I would rather go the Remedy way then, which is placing a bunch of probes in relevant spots automatically with the help of a voxel grid. Will think about your idea, maybe try to implement it as it sounds like an easy extension to voxel GI which I played around with recently.

On ‎30‎/‎01‎/‎2018 at 9:06 PM, Hodgman said:

Yeah as above you can make it into an art/content problem :)

No :D (In this case I would be delegating the problem to myself as probably only I will ever use this engine :) )

On ‎30‎/‎01‎/‎2018 at 9:06 PM, Hodgman said:

You can make the OBB/parallax correction feature optional, to allow "localised global" probes. You might bathe a whole building in a non-parallax probe, then add a few parallax probes to important rooms only. 

I want to do that, basically I just wondered how to blend between the global probes once "I leave the building". Because say that when I exit the door, we want to switch probes, now the outside environment will have the inside envmap for some time and the whole scene will blend abruptly. Btw which game did you bake lightmaps at runtime?

Share this post


Link to post
Share on other sites

Remedy also had voxelized pointers towards which probes are relevant where. Heck you could go a step further (or does Remedy do this already) and store a SH probe, with channels pointing towards the relevant probes to blend. It'd be great for windows and the like, blending relevant outdoor probes would be great there.

You could even make the entire system realtime, or near to it. Infinite Warfare used deferred probe rendering for realtime GI, and Shadow Warrior 2 had procedurally generated levels lit at creation time. I seriously hope those are the right links, I'm on a slow public wifi at the moment so...

Regardless a nice trick is to use SH probes with say, ambient occlusion info, or static lighting info or something, to correct cubemap lighting. This way you can use cubemaps for both spec and diffuse, and then at least somewhat correct it later.

Share this post


Link to post
Share on other sites
4 hours ago, turanszkij said:

I mentioned that approach, what I dislike about it is that far away objects will have a very wrong reflection, and also when the closest envmap changes the entire scene gets re-lighted. But I will probably go with this one as this method can be implemented with no hard popping when a new envmap gets closest.

In my solution global env maps are separated from the local ones. Global ones should capture mostly sky and generic features, and be used very sparsely (few per entire level). This way it's enough to blend just two of those to have perfect transitions and far away reflections will look fine. I actually used this system for Shadow Warrior 2, just with a small twist - probes were generated and cached in real-time. If you are interested you can check out some slides with notes: “Rendering of Shadow Warrior 2”.

Share this post


Link to post
Share on other sites
22 hours ago, turanszkij said:

No :D (In this case I would be delegating the problem to myself as probably only I will ever use this engine :) )

You can still make it into a problem of manual labour per scene (a level editor task) instead of an algorithmic challenge :)

Either way you're going to find lighting bugs in the level editor so the difference is whether that prompts you to go and fix the code or massage the lighting data in the editor to hide the bug! 

22 hours ago, turanszkij said:

I want to do that, basically I just wondered how to blend between the global probes once "I leave the building". Because say that when I exit the door, we want to switch probes, now the outside environment will have the inside envmap for some time and the whole scene will blend abruptly. Btw which game did you bake lightmaps at runtime?

I was suggesting to have one truly global probe, but then use large non-parallax local probes to override it in areas (such as a whole building), and then even smaller local probes to override rooms within the buildings. You'd define a soft falloff at the edge of each local probe region for blending, and results aren't based on the camera position. 

I did lightmap baking on Don Bradman Cricket 14 (PC, PS3, 360 edition, not used in the PS4/Xbone edition though). Bakes were budgeted 1ms of GPU per frame during gameplay or 30ms of GPU per frame on loading screens. A bake took about 3 minutes during gameplay, though we also had a low quality setting if we needed one quicker. So, not useful for dynamic lights from explosions/etc, but perfectly fine for dynamic time of day. Sports games also feature lots of camera cuts (e.g. After a football player is tackled,  or a goal is scored, or before a bowler bowls in cricket) so we would wait for one of these camera cuts before switching out the old lightmap with the newest bake, so the change couldn't be noticed :)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Advertisement
  • Advertisement
  • Popular Tags

  • Popular Now

  • Advertisement
  • Similar Content

    • By AzeAngelicOne
      Real talk, This game, this game right here:

      It sucks. Immensely. It's good playing with friends, but it's AWFUL playing alone.
      But what if it didn't?
      Anybody know this game?

      A strong community behind it, and patches every month, just to show that the devs are doing everything in their power to make sure that the game is still supported, and improved to the point where everyone is happy. Well, most people.
      What if someone made a Shonen Jump crossover game in the style of DBFZ/UMvC3? What would you do?
    • By Mailbox
      Hello,
      I'm in the midst of developing a simple 3d physics engine, but have been stuck on a problem for quite a while now. I would really appreciate it if someone could have a look at it, because I'm at a loss.
      The issue I'm having concerns the collision response code, which seems to fail spectacularly in some cases, generating wrong impulse magnitudes. Specifically, something with the angular part of it is wrong, since it works as expected when only dealing with the linear part.
      I'm still not sure if I calculate the inertia tensor in world coordinates the right way, so that could be one possible culprit (though I've tried a few different ways). Impulses seem to be applied correctly to the bodies, so I think the error lies in the calculation of the impulse magnitude denominator.
      I've checked it many times with the way the Bullet physics engine does it, with the formulas available at Wikipedia, and some other sources, to no avail.
      The bodies simply keep bouncing off eachother (even though there is supposed to be no restitution) or spinning wildly on collisions.
      Here is the code that deals with the collision response:
      Vector3 originA = contactOrigin - bodyA.Position; Vector3 originB = contactOrigin - bodyB.Position; Vector3 velocityA = bodyA.Velocity + Vector3::Cross(bodyA.AngularVelocity, originA); Vector3 velocityB = bodyB.Velocity + Vector3::Cross(bodyB.AngularVelocity, originB); Vector3 velocity = velocityA - velocityB; Matrix3x3 inverseWorldInertiaTensorA = Matrix3x3::Scale(bodyA.InverseInertia) * Matrix3x3::Rotate(bodyA.Orientation); Matrix3x3 inverseWorldInertiaTensorB = Matrix3x3::Scale(bodyB.InverseInertia) * Matrix3x3::Rotate(bodyB.Orientation); float normalVelocity = Vector3::Dot(velocity, contactNormal); if (normalVelocity > 0.f) { float impulseDenominator = bodyA.InverseMass + bodyB.InverseMass + Vector3::Dot(Vector3::Cross(inverseWorldInertiaTensorA * Vector3::Cross(originA, contactNormal), originA), contactNormal) + Vector3::Dot(Vector3::Cross(inverseWorldInertiaTensorB * Vector3::Cross(originB, contactNormal), originB), contactNormal); float j = -normalVelocity / impulseDenominator; bodyA.Velocity += j * bodyA.InverseMass * contactNormal; bodyB.Velocity -= j * bodyB.InverseMass * contactNormal; bodyA.AngularVelocity += j * (inverseWorldInertiaTensorA * Vector3::Cross(originA, contactNormal)); bodyB.AngularVelocity -= j * (inverseWorldInertiaTensorB * Vector3::Cross(originB, contactNormal)); } Thanks in advance!
    • By Luca Falco
      https://www.youtube.com/watch?v=UMoc52DEoC8     ABOUT THE GAME: Room54 is a first-person Horror/adventure  Videogame for pc,mac and linux users. the game is currently under development by a very small team, we put so much effort in this project and we are keep doing our best for that we are going to launch a kickstarter campaing in order to reach our ideal budget to complete the project, we hope the GameDev community can help us a lot                      STORY:   Daniel is a family father like many other, one day he decide with your wife and his daughter  to spend her winter holidays at their mountain house that they have recently buy in  mountain ,a  wonderful place surrounded by the nature of the woods,  Completely far away from the caotic city life. During their holidays Daniel and his family will understand that they are not welcome there and they will discover an  hided and disturbing part of the valley that they have never seen before                       GAMEPLAY:   You will play as Daniel, a father that will try to save his family, your gameplay will be focused on discovering secrets places around valley,investigating and trying to survive, you will find object that will help you solve enigma and to stay alive. The immersive audio and  environment will make you feel constantly follow by an high anxiety dose  during the game.                         SOCIAL PAGES:     Follow us to get the latest development news and insights     IndieDB:     http://www.indiedb.com/games/room54   Twitter:       https://twitter.com/Room54Thegame   facebook:   https://www.facebook.com/Room54/
    • By Gnollrunner
      Hi again,  After some looking around I have decided to base my game directly on Direct X rather than using an existing game engine.  Because of the nature of the stuff I'm doing it just didn't seem to fit very well and I kept running into road blocks.  At this point I have a big blob of code for doing fractal world generation and some collision code,  and I'm trying to put it into some form that resembles a game engine.  Since I've never used one before It's a bit alien to me ..... so can someone direct me to a book, website, article, whatever... that covers this?  I'm mainly looking for stuff that covers C++ library design. I'm not adverse to using 3rd party tools for stuff I can used them for.
    • By mmmax3d
      Hi everyone,
      I would need some assistance from anyone who has a similar experience
      or a nice idea!
      I have created a skybox (as cube) and now I need to add a floor/ground.
      The skybox is created from cubemap and initially it was infinite.
      Now it is finite with a specific size. The floor is a quad in the middle
      of the skybox, like a horizon.
      I have two problems:
      When moving the skybox upwards or downwards, I need to
      sample from points even above the horizon while sampling
      from the botton at the same time.  I am trying to create a seamless blending of the texture
      at the points of the horizon, when the quad is connected
      to the skybox. However, I get skew effects. Does anybody has done sth similar?
      Is there any good practice?
      Thanks everyone!
  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

Participate in the game development conversation and more when you create an account on GameDev.net!

Sign me up!