Sign in to follow this  
spek

Realtime cubemap reflections (GTA IV)

Recommended Posts

spek    1240
Dear graphics friends, I was enjoying GTA IV, and wondered how they did the reflections. The cars, but also the environment (buildings, pavement, etc.) reflect. I guess they do it with cubeMaps, where the player carries one cubeMap with him which is updated every frame. This way the car will always have a proper realtime reflection. Now I was thinking, would they use the very same "player cubeMap" as well for the (nearby) environment? In the past I did reflections like Halflife2 did, by placing (pre-rendered) cubeMaps all over the scene. Each surface would pick the most nearby/efficient probe. Works ok, but its not a realtime solution of course. And I also had problems sometimes where 2 neighbour polygons would use a different probe; a seam became noticable. GTA IV doesn't seem to have that "seam" problems, and the reflection are dynamic (the lighting changes all the time). I doubt if they use a lot of probes that are updated each frame. So maybe the (nearby) environment simply uses the same cubeMap as the player? Of course this won't give super accurate results, but with the help of low-res and blurring, you won't see it that much anyway. If this is the case, there is one thing I still wonder though. How are the reflections working for more distant/larger objects? For example, a cubeMap for the player that stands in a narrow dark alley won't be working very well for the skyscraper in the background... Maybe the objects further away simply don't reflect in GTA IV, I haven't really checked that out yet. Greetings, Rick

Share this post


Link to post
Share on other sites
spek    1240
Maybe... But I still doubt if they use multiple cubeMaps. I mean, updating those damn things could cost quite some time. The XBox and PS3 are equiped with some graphical power which allows to do this, but I wonder how much probes they use then (nearby). Maybe the player not carries 1, but a whole grid of probes surrounding him.

Could be, but I still doubt it. For a start, the environment in GTA IV doesn't show seams between 2 different cubeMaps (imagine 2 walls next to each other, where 1 wall uses probe A, and the other probe B). This makes me think either 1 probe is used, or they are blending between probes instead of assigning only 1 probe per face/polygon/chunk. Since you can drive pretty fast in this game, it would become quite noticable if the underground would be switching between a lot of probes.

On the other hand, using only 1 probe isn't really accurate of course. They mask this by only using very blurred and vague reflections (and even then you can see errors). But maybe they use some other tricks as well?

I was just wondering how they did it. One of the things I need to implement in my new hobby engine, are reflections. Normally I would have created pre-rendered probes and assign them to polygons, but maybe this works as well. At least its easy to maintain, its realtime, and it doesn't require the designer to place these stupid probes everywhere :)

Greetings,
Rick

Share this post


Link to post
Share on other sites
Jason Z    6434
I haven't played GTA IV yet, but I am pretty sure it is a direct3d based game. This means that you can fire up PIX from the DX SDK and take a single frame capture at an appropriate time. Then go back and you can see every API call that was made and what resources were used. You can even view the contents of the various textures and/or cubemaps at various points in the frame rendering.

It takes some time to dig through everything, but I actually ran a few frames from Crysis through the same process and it proves to be very educational. Check it out - you won't be disappointed!

Share this post


Link to post
Share on other sites
Ging    181
Jason - that won't work unless you happen to work at Rockstar, with GTA 4 currently only being a console title I think it might prove difficult to run it through PIX.

Share this post


Link to post
Share on other sites
Jason Z    6434
Whoops! I didn't realize that there isn't a PC version yet! I wonder why they didn't launch on multiple platforms at the same time...

Share this post


Link to post
Share on other sites
snisarenko    526
GTA IV is not the first game to do this. What other games have done, was render a low poly/detail version of the environment(including other cars etc.) into a low resolution cube map every frame. Since GTA IV is running on next generation hardware, I am assuming its able to do this with more detail and higher resolution.

Share this post


Link to post
Share on other sites
spek    1240
@Jason Z
Thanks for that tip anyway! Never heard of it, but it sounds usefull. I saw GTA IV indeed on the Playstation 3. BTW, I thought the PS3=OpenGL, and XBox=DirectX?


I've seen these dynamic reflections in a lot of (race)games as well, but GTA IV was the first game where I noticed these (realtime) reflections on the surrounding geometry as well. I have no doubt that current hardware can easily render a couple of cubeMaps and even blur them every frame. But still, the question is where/how many probes do games like GTA IV use? Only 1 probe (the one inside the player/car), a grid of probes surrounding the player, or probes at fixed positions like Halflife 2 did?

I'm hoping to hear they indeed only use 1 probe inside the player. Because it doesn't require you to place these probes when designing the world, and updating only 1 cubeMap every frame won't be much of a problem. But of course, it has some accuresy issues though...

Greetings,
Rick

Share this post


Link to post
Share on other sites
mattnewport    1038
Quote:
Original post by spek
I thought the PS3=OpenGL, and XBox=DirectX?

The Xbox 360 uses a somewhat customized version of DirectX 9, it's very similar but not identical to the PC version. The PS3 does have an OpenGL implementation as part of the SDK but very few games use it - most use an alternative API called libGCM that gives much more direct access to the hardware.

Share this post


Link to post
Share on other sites
remigius    1172
Quote:
I'm hoping to hear they indeed only use 1 probe inside the player. Because it doesn't require you to place these probes when designing the world, and updating only 1 cubeMap every frame won't be much of a problem. But of course, it has some accuresy issues though...


If you want the definite word on it, Wolf would be the one to ask. FWIW I did notice the accuracy is indeed somewhat off with the water reflections for example and it seems increasingly worse the further reflected objects are away from the player.

Share this post


Link to post
Share on other sites
spek    1240
Quote:

FWIW I did notice the accuracy is indeed somewhat off with the water reflections for example and it seems increasingly worse the further reflected objects are away from the player.


That is a known problem with cubeMaps, since the reflections are only 100% correct for exactly 1 point in space. However, reflections on the environment are ussually very vague. Concrete, wood, pavement, asphalt, bricks. All these materials won't reflect much. In GTA IV you only see a low-res reflected world, with blurred streaks from lightsources. Darker stuff is almost not visible in the reflection. Furthermore, you usually only see reflections nearby. And yes, if you look carefully, you will notice that the reflected stuff has an offset.

For very reflective surfaces such as water, I always mirror the world to get much better/accurate results. But this only works if the surface is flat (which is the case with water or a mirror). But environments like the city in GTA IV, have much more complex shapes. You can't use a mirrored texture on a floor with many height differences, unless you could do some depth/height correction maybe.


Correct reflections are one of those graphic thingies that won't be correct for the next years, just like realtime GI tricks. Nevertheless, GTA IV showed an effective way of using the available techniques. I'm just wondering what they exactly did though...

Greetings,
Rick

Share this post


Link to post
Share on other sites
mre    124
I've been looking at this closely on my PS3 lately, and what I've been able to determine:

- The reflections are low resolution. If rendered to a cube map we're looking at probably 256x256 each face.
- Dynamic objects such as cars, trees and people, do not appear in the reflections. It's pretty much just buildings and the telephone poles you can't knock over.
- Shadows do not appear on things in the reflections.
- The reflected scene being rendered is very limited in distance. Looking at certain angles I was able to see buildings appear out of the depth fog a half a block (or so) up the street.

One problem with cube maps is that things tend to look as though they're infinitely far away in the reflection. Things that were close end up looking more like they're enormous. Although with it being updated every frame, the effect of motion might overcome that. It's possible also that they're using some sort of depth field for parallax mapping the environment map.

Aside from a cube map generated for every frame, another possibility might be a hemi-sphere map or some other custom solution.

[Edited by - mre on May 16, 2008 5:06:18 AM]

Share this post


Link to post
Share on other sites
spek    1240
Sorry for the late reply. I'm moving to my new house. No time to hobby with game programming and packing stuff at the same time unfortunately :)

@Wolf
I looked at your blog... you actually did the graphics of GTA IV?! Excellent job you guys did there! And the same for the graphics. I'll have to admit GTA San Andreas is still my favourite (more humor if you ask me), but the visuals have been improved big time (and the phsyics as well!).

You mentioned "paraboloid reflection maps". I guess its somewhat the same as dual paraboloid shadow maps, where you'll have to use alot of vertices and "bend" them into the shape of a half sphere. Ifso, you probably won't need that many vertices for reflections, since its a lot more harder to tell if a reflection is not 100% correct.

Maybe it's a company secret, but could you explain how many reflection maps are used to capture the environment, besided of the one inside the player/car? Are you using maps at fixed (pre-designed) points, or are there 1 or more maps moving with the player?

Greetings,
Rick

Share this post


Link to post
Share on other sites
Jason Z    6434
I wrote an article on this very topic a while ago (see the link in the signature). Even though my implementation is not quite as high quality as the rockstar version, it will still give you the overall concept.

Share this post


Link to post
Share on other sites
spek    1240
Thanks, this will become usefull some day!

However, the real question is not how to capture the reflections at a certain point. I wondered how many points they use in GTA IV, and/or if those points are placed at fixed points or move along with the player. Halflife2 used fixed points, placed by the designer, to apply reflections on the (static)environment.

Maybe they simply do this in GTA IV as well. I wondered though, since the environment is *huge* and changes all the time >> how to update all these reflections for such a large environment realtime when speeding through the city with 100 miles per hour? I also wondered because I never saw any "seams" between the probes. I mean, if the a skyscraper uses 6 reflection points stacked on each other, a seam will become visible between them. This will be even more noticable on the pavement I guess.

This made the suggestion for me that GTA IV isn't using reflection maps at fixed points, but uses a (low-res) grid of probes or maybe even only 1 point that moves along with the camera/player.

Greetings,
Rick

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this