Sky-rendering techniques

Started by
137 comments, last by Filip 20 years, 2 months ago
Those clouds are absolutely awesome. Probably the best out of any game I''ve ever played (and I''ve played quite a few..) I''m still just stunned at that picture man.. amazing!! You even have kind of streaks of sun where the clouds block sunlight and where they don''t.. Your sky engine is very well written! I can imagine why it took you so long.
However, how would your clouds perform in-game? I read someone talking about sacrificing in-game complexity/activity to have better clouds, and vice-versa, I can''t remember if that was you.. but if you created, say, a town that you could stroll through, and other non-player characters are walking around.. would you have to tone down your sky? Or does it take not much processing power.. I read most of the posts, sorry if you already addressed this, but I can''t believe how awesome that is, and the only reasons I can think of why other games don''t look that good is either you are a much better programmer or the sky takes too much power.. which one is it?
Advertisement
Yann L: your replies are very helpful. My crappy clouds already look a little less crapier then they did before

Currently I''m having a problem with perspective. My clouds look a little flat, as if there is a polygon facing the camera. I increased the size of my skyplane (I use a "plane" on top of the viewer, pretty much a tesselated quad with its corner vertices "pulled down". This way I get the benefit of the sphere and don''t run into texture mapping artifacts) but the clouds looked too low. It''s hard to explain the effect, but I''m sure anyone who wrote a sky class ran into this problem. Can someone give me some basic values of how high above the terrain their sphere/plane is, what''s the radius, how large is it relative to the terrain. I just need some basic values to start off with, to make experimentation a little easier.

Another major problem I ran into is shading the clouds. Right now my clouds have constat color, and it doesn''t look too good. Yann L, I looked at the shading article u posted. The approximation is pretty good, however I don''t think implementing the algorithm they describe is necessary. The integral over all their parameters is a good evaluation for nonrealtime systems, but we''re not doing Toy Story 2 here. I''m wondering if perlin noise could help out again. I''ll try to get some noise again over the clouds and specify the color based on the noise. I''ll post here about how it looks. If anyone knows other techniques for shading clouds, please post links/ideas here.

Concerning the tradeoffs, I don''t think they''re such a big thing. There are few polygons required to render the skybox, the fill rate isn''t that huge either. To make the clouds dynamic the perlin noise has to be evaluated in 3 dimensions over many frames, so it shouldn''t be a problem either. Implementing the procedure in another thread or a loop that breaks off should work fine. The shading is the same thing, it''s evaluated over many frames. So in the case of clouds it actually turns out that you could have a beautiful skybox and not pay a lot for it. Or am I mistaken?
quote:
coelurus wrote:
Some cool ideas for you all:
Thunder and Lightning? Haven''t seen any good yet, would be interesting (and coupled with awesome sound effects, yay!).

Was on my todo list Here is a very interesting paper about doing amazingly realistic lightning. I am highly motivated to implement that stuff in realtime !

quote:
kill wrote:
This way I get the benefit of the sphere and don''t run into texture mapping artifacts) but the clouds looked too low. It''s hard to explain the effect, but I''m sure anyone who wrote a sky class ran into this problem.

Yep, I know the effect.

quote:
Can someone give me some basic values of how high above the terrain their sphere/plane is, what''s the radius, how large is it relative to the terrain. I just need some basic values to start off with, to make experimentation a little easier

Well, the height depends on your unit scale. It normally isn''t relevant, since you can always adjust the frequency of the perlin noise to compensate. There is no real scale in sky-renderers anyway, all is relative. Try out various heights, and keep the lowest one that still gives a large scale feeling.

The x,y extends of your cloud plane depend on the quality you want, and on the technique you use to compute and display clouds. In my system the extends are infinite: it''s a virtual plane that is procedurally textured, so there are no limits.

quote:
Another major problem I ran into is shading the clouds. Right now my clouds have constat color, and it doesn''t look too good. Yann L, I looked at the shading article u posted. The approximation is pretty good, however I don''t think implementing the algorithm they describe is necessary. The integral over all their parameters is a good evaluation for nonrealtime systems, but we''re not doing Toy Story 2 here.

Hmm, this article actually described a realtime system... And it is very efficient.

quote:
So in the case of clouds it actually turns out that you could have a beautiful skybox and not pay a lot for it. Or am I mistaken?

As long as the clouds are static, that is correct.

About the performance issue: the impact of rendering the sky itself is negligible. It''s nothing but a few hundred faces in the view. It takes an important hit on fillrate, though, but that is not so much a concern on modern HW (my sky system uses 2 to 3 fullscreen passes, depending on the view height).

You''ll always have the sky at the quality as shown above, even with very crowded 3D geometry beneath it. The problem is animating the clouds. Simple movement is no problem, it''s just shifting texcoords and a bit of data around, nothing critical. But having the clouds change shape and billowing imposes a very serious burden on the CPU. Basically, you need to recalculate the whole perlin noise set every frame. That''s 8 octaves plus the exponentiation pass, for every texel in the virtual cloud texture set (around 1,770,000 texels !). And interpolating isn''t enough: you could interpolate the perlin noise, but you still need the exponentiation after interpolation. Otherwise the clouds will behave strangely. Shading, while not being very expensive, also adds to the cost equation.

quote:
okonomiyaki wrote:
I read most of the posts, sorry if you already addressed this, but I can''t believe how awesome that is, and the only reasons I can think of why other games don''t look that good is either you are a much better programmer or the sky takes too much power.. which one is it?

Hehe, my ego would surely tend to say the former

But seriously: neither one. Big game development companies just don''t have the time/resources to invest months of work just into a simple (and unimportant) subsystem such as the sky. We are a small startup gamedev studio, so we can do those kind of things. That''s the whole difference.

OK, that''s it for today, I''m going to bed. Gd''night all.

/ Yann
Very awesome and informative thread. I hope to start on a sky system soon.
That''s a good point, Yann You know what they say.. the homemade cookies are always better then the massive-produced store bought ones..
Yann L, you''re mistaken about the costs of animation of the clouds. As you mentioned above, simple movement is easy, you just shift texture coordinates. In fact, you don''t even have to do that, you can simply translate the texture matrix so you don''t have to update the vertex buffers. Recalculating the perlin noise is also not expensive. If you do it every frame, sure, you''ll bring any CPU to its knees. But what I was trying to tell you, you don''t need to do it every frame. If you look outside, the clouds may move fast but the way they change their shape is relatively slow. Instead of recalculating the change every frame you can recalculate it once per second, even more. Of course doing that kind of calculation once per second will be noticable to the user, so you might wanna put your calculating in a thread. The problem with that approach is that with Win32 you can''t control the exact timeslices your threads get. Win32 has "fibers" API, which was created for people porting unix application. In Unix you can have exact control of the way your threads are executed and fibers API gives developers the same features for ease of portability. You can use that, or your loop can simply break off once its done with 1/60th of the texels (assuming u have 60fps) and when you call the function on the next frame you continue with the calculation. That approach is unintuitive so you might wanna look at the fibers solution. In any case, if you choose to animate your clouds the point is that it doesn''t have to be expensive because the clouds deform over minutes, sometimes even hours (depending on the wind) so you can stretch the calculation over many frames and the user will not see a quality decrease.
quote:
As you mentioned above, simple movement is easy, you just shift texture coordinates. In fact, you don''t even have to do that, you can simply translate the texture matrix so you don''t have to update the vertex buffers

Well, it''s not as easy as that either. This only works if you directly render your sky plane. I project the plane onto a cloud box using procedural textures, and upload the cloud box cubemap. This gives better resolution using less texture memory. So, I can''t just shift the texcoords. But you''re right, it''s not very expensive to make them simply move.

quote:
But what I was trying to tell you, you don''t need to do it every frame. If you look outside, the clouds may move fast but the way they change their shape is relatively slow. Instead of recalculating the change every frame you can recalculate it once per second, even more.

I''m not sure, if that is worth the effort. As you said, in reality shape-changing is a slow process, much slower than cloud movement. So, since moving the clouds automatically updates the perlin noise set over time (the data scrolling in is newly generated), I already have a slow continuous shape change, distributed over several frames.

The main idea behind realtime billowing are special effects: sudden weather changes (due to a magical spell, for example), where cloud suddenly contract to form a thunderstorm or similar. All this has to be done every frame, since shape morphing is only really impressive, if you can witness it''s effects in realtime.

A theoretical next step would be to put the exponentiation pass onto the GPU, using a pixelshader. That way, the whole shape morphing, including trivial interpolation, could be done by the GPU. All the CPU would have to do, is to supply a new perlin noise set every 30 seconds or so. That would be a great improvement over the current system.

/ Yann
I didn''t think of the magical spells and sudden changes in weather. Actually, now that I think a little more about this, I think even sudden changes are possible.

I''m not sure how you go about generating the perlin noise, but this step doesn''t take any CPU resources. All perlin noise is, is a blurred white noise. So you can generate it entirely on hardware. Depending on what frequency you want, generate a small texture of white noise (16x16) and then just stretch that texture over a larger one (512x512). You''re done, the larger texture contains perlin noise done completely in hardware. No speed hit there.

The exponent is a bit more of a problem. I can''t really think of a standard way to do it in hardware. As you''ve mentioned above, if the graphics hardware you aim for supports pixel shaders, then there''s no problem (although in my case I can''t use that, since I aim for geforce2 and above). Without the pixel shaders we''d have to read from the large texture, apply exponent and write back. The actual exponentual function can be put into a lookup table that will easily fit into the CPU cache. Still, a large texture (1024x1024) is A LOT of cycles even with the lookup table. If you handoptimize the assembly you can take advantage of pipelining, so your code will run about 20 times faster.

This is a lot of code though... May be in version 2 of the game?
quote:
Depending on what frequency you want, generate a small texture of white noise (16x16) and then just stretch that texture over a larger one (512x512). You're done, the larger texture contains perlin noise done completely in hardware. No speed hit there.

No, that's not full perlin noise. That's *one* single octave of perlin noise. You'd have to add up to 8 octaves to get a good result (low frequencies but with lots of small fractal details). To do that on hardware, you'd need 8 texture units. *)

quote:
The actual exponentual function can be put into a lookup table that will easily fit into the CPU cache. Still, a large texture (1024x1024) is A LOT of cycles even with the lookup table. If you handoptimize the assembly you can take advantage of pipelining, so your code will run about 20 times faster.

Two things to keep in mind: a 1024² texture is not going to be enough for high resolution clouds on a plane. If you don't use the virtual cloud box trick, then aim for minimum 2048², if not 4096² for high quality clouds.

And then there is AGP bus reading performance. If you want to read back a 1024² texture from your 3D card every frame (to perform exponentiation), then that operation alone will totally kill your framerate, regardless of your actual exponentiation speed. And it creates a 'bubble' in the GPU command stream: the stream is stalled for the time reading the texture.

*) Edit: That's not entirely true. With tricky use of texture shaders, you could actually do it in 4 units. But then, there is no more room for the exponentiation. Perhaps by cutting down to 6 octaves, it would work. But then I'd loose all the nice high-frequency details of my clouds So that's not an option...

/ Yann


[edited by - Yann L on May 21, 2002 10:30:40 AM]
OK, I just thought a little about this. It could actually be possible to do the clouds 100% on the GPU, in full quality. But you''d need a rather powerful 3D card (GF3+).

Your CPU would need to precompute 3 perlin noise textures, each one with 3 octaves pre-encoded. The textures need to be signed, in the -1 to 1 range. You''d load them onto texture unit 0 to 2. The texcoords will need to be adjusted by a vertexshader, so that they match for adding the individual noise octaves. 2 regcom stages would be enough to add them all up. Now the exponent pass: the cover substraction + clamping can be done in a regcom. The pow() operation can be simulated by a 1D lookup texture on texture unit 3. The result needs to be range mapped in the final combiner, and can be directly textured onto a cloud plane. The shading can be encoded into the rgb part of the 3 noise textures, although it won''t be as accurate.

Any major flaws in there ?

/ Yann
1. A 1024x1024x1 texture is one meg. One byte per texel should give enough precision for the clouds. Now, if you read one meg every frame that''s about 70 megs per second, write it back, another 70. So let''s say 140 megs per second. They claim you could push 20 gigs per second on the AGP bus. Now, keeping the texture in AGP memory is very slow for reading, it will probably cause a huge stall. So instead, just keep it in system memory and send it back to the card every frame. That''s 70 megs per second without reading stalls, so on AGP 2x it should be fine.

2. If you look into Fourier theory, a sampled function has a Nyquist limit. That means if you use more then a certain number of octaves you actually LOSE quality due to aliasing artifacts (the screen resolution doesn''t have enough pixels to represent higher frequencies and they introduce errors in lower frequencies). According to "Texturing and Modelling: Procedural Approch" non realtime guys usually use 6 - 10 octaves depending on the screen resolution. I don''t remember the exact equation to figure out how many octaves you should use for a specific screen resolution, but I''d say if you use 4-5 octaves you won''t notice a considerable loss of quality. In my tests 5 octaves look just as good as 8.

3. I have a GeForce2 MX on my development machine, so unfortunately I don''t have access to hardware shaders Since vertex shaders are emulated quiet well, I did a couple of things with them, so I''m familiar with working with them and what you could accomplish with them. With no efficient software support for pixel shaders, I am not very familiar with them, so I don''t know exactly what you could do. As you said, you could represent many octaves in one texture (RGBA - 4 octaves, one byte is enough precision), that''s two texture units. However, I don''t know if you could combine different channels of the same texture with pixel shaders, so I''m not sure this approach will work.

Anyway, we''re getting into very complex tricks here. Getting this to work and to look good will probably take a LONG time. Next generation cards are coming up with branching instructions, loops, subroutines etc in hardware. So once GeForce5 or GeForce3000 or whatever it will be, making realtime clouds will probably be possible fully in hardware without major tricks. I guess we just have to wait

On a different note, here''s what I read on vterrain.org about shading the clouds: "Hugo writes: ''The secret behind Terragen''s beauty: rather than going for all out brute force and mathematical accuracy, Matt just writes algorithms that produce good results, rather than trying to exactly model the physics. The clouds are essentially a beautiful bodge.''" Do you have any idea which algorithms they''re talking about? I tried to find more details, but couldn''t

This topic is closed to new replies.

Advertisement