Sign in to follow this  
Cybrosys

Detail texture techniques

Recommended Posts

Hi all! Recently i descided to analyze the techniques used in the CryTek engine (Far Cry). As far as i've been able to tell the entire landscape is built up out of 32x32 33x33 tiles using about 4 lods. Pretty standard heightmap landscape which at first was quite a shock, i'd thought that they were using more advanced techniques when it came to lod, but in their case, trees and such cover up the poping etc. I realised how they texture their landscapes. They use layers, each layer consists of a texture and an alpha mask for that texture, defining where on the landscape it will be blended. The alpha mask can be generated by defining that the texture will be blended on polygons between certain height values aswell as steepness of the polygon. (Ie horisontal to vertical) All these layers and masks are blended onto one huge 1024x1024 texture. That texture is dragged out on this huge landscape giving a very low blurred textured look. This is fine for longdistance terrain, you'll get one texel per quad and it's just the underlying base texture. If you've played the game you'll notice that the ground looks nothing like that up close. Hell, if you'd look down you'll see leaves. Well that's the detail texturing technique that i'm wondering how they've implemented in realtime. As far as i can tell the editor has no limits as to how many texture layers there can be, i mean that's all fine and dandy since the base texture is rendered once and saved in the editor. But how does their detail texturing work? You can't send more than 8 textures through the pipe during a draw call. So how are they able to in realtime blend all those textures that were used to create the huge base texture? What polygons gets detail mapped and which don't are determined on a quad basis with a distance from the camera.. and for some reason that's not all, it seems as if extra polygons gets drawn when detail texturing is used. I'd realy love to know how they do their detail texture mapping and how it can be done otherwise. It's driving me nuts! =) It could be such as that each tile gets it's own personal high res texture that gets activated, but that's 1024 extra textures in total... And to get that kind of resolution you'd need atleast 512x512 texture res on each tile texture... the memory usage just shot through the roof on that theory. Old detail texuring techniques only involed using perlin noise or such to generate noise on a rather plain surface, causing the illusion that there's more "detail" than there realy is.

Share this post


Link to post
Share on other sites
Aren't you in luck. I recently also looked at the way Farcry did its detail texturing and decided to do something similar in my own engine.

Basically this is the way you do it (or the way I did it):

- Farcry uses the concepts of layers and surfaces. They use the layers to essentially color the base texture of the terrain and surfaces (or detail textures) to blend with the base texture to represent detail up close. Each layer uses a surface, surfaces can be used by more than one layer. I've limited myself to a maximum of 8 surfaces as its very very rare that youll need more than that because the base texture adds enough variety. I believe farcry gets around this by doing multiple passes of 4 detail textures, but you lose performance, and its rare for a map creator to use more.

- You'll need to create layer masks for each layer, generally around 1024x1024 is enough detail. (You can use the farcry sandbox to do this). Then youll need to generate a surface mask for each surface. Creation of a surface mask is done via the following steps:

1. Fill the surface that is used by the default/base layer of the terrain with white (or 0xFF). This is the default surface.

2. Make the other surfaces black (0x00).

3. Iterate through each pixel in each layer. For each pixel update the corresponding pixel in all surfaces. Update using the rule; if a layer uses the surface then add the pixel value, otherwise subtract the pixel value.

4. You should now have 1-8 surface masks.

5. Save them into what I call a diffuse and specular map. The diffuse and specular maps are essentially heightmaps for vertex colors. Save the first surface into the red channel of the diffuse map, second in the green channel and so on, finally the 8th surface mask into the alpha channel of the specular map.

- Now you've got your maps, load them into your vertices when you load your terrain patches.

- The next part involves calculating which patch is closest to the camera. Once you find that, find the patches neighbouring it also. This gives you 9 patches and is generally enough of a distance for detail textures to blend off into. You could grab the next neighbouring patches also which would give you 25 patches, but it's usually not necessary.

- Render those patches again and use the following vertex and pixel shaders:


//
// Structures
//
struct VS_OUTPUT
{
float4 pos : POSITION;
float4 diffuse : COLOR0;
float4 specular : COLOR1;
float2 uv0 : TEXCOORD0;
float2 uv1 : TEXCOORD1;
float blendRange : TEXCOORD2;
};

//
// Vertex Shader
//
VS_OUTPUT DetailTextureVS(float3 pos : POSITION,
float4 diffuse : COLOR0,
float4 specular : COLOR1,
float2 uv0 : TEXCOORD0,
float2 uv1 : TEXCOORD1)
{
VS_OUTPUT o;

//
// Calculate the distance from the vertex to the camera
//
float cameraDistance = distance(g_cameraPosition, float4(pos, 1.0));

//
// Divide the camera distance by the blend distance and clamp it to 0..1
//
float blendRange = saturate(cameraDistance / g_blendDistance);

//
// Translate the vertex upwards depending on the distance to the camera
//
float3 newPos = pos;
newPos.y += lerp(0.001, 0.1, blendRange);

//
// Translate to screen space
//
o.pos = mul(float4(newPos, 1.0), g_worldViewProjection);

//
// Copy the diffuse and specular colours
//
o.diffuse = diffuse;
o.specular = specular;

//
// Copy the texture coordinates
//
o.uv0 = uv0;
o.uv1 = uv1;

//
// Calculate the exponential blend factor and store it in the diffuse value
//
o.blendRange = blendRange * blendRange * blendRange;

return o;
}





PS_OUTPUT DetailTexture8PS(VS_OUTPUT vs)
{
PS_OUTPUT o;

//
// Multiply each detail texture by its surface mask and add the result to the resultant colour
//
o.color = vs.diffuse.r * tex2D(DetailTexture0Sampler, vs.uv1 * g_detailTexture0Scale);
o.color += vs.diffuse.g * tex2D(DetailTexture1Sampler, vs.uv1 * g_detailTexture1Scale);
o.color += vs.diffuse.b * tex2D(DetailTexture2Sampler, vs.uv1 * g_detailTexture2Scale);
o.color += vs.diffuse.a * tex2D(DetailTexture3Sampler, vs.uv1 * g_detailTexture3Scale);
o.color += vs.specular.r * tex2D(DetailTexture4Sampler, vs.uv1 * g_detailTexture4Scale);
o.color += vs.specular.g * tex2D(DetailTexture5Sampler, vs.uv1 * g_detailTexture5Scale);
o.color += vs.specular.b * tex2D(DetailTexture6Sampler, vs.uv1 * g_detailTexture6Scale);
o.color += vs.specular.a * tex2D(DetailTexture7Sampler, vs.uv1 * g_detailTexture7Scale);

//
// Blend between the texture colour and grey based on the exponential blend
//
o.color.rgb = lerp(o.color.rgb, float3(0.5, 0.5, 0.5), vs.blendRange);

return o;
}




- You'll need a different pixel shader according to the number of surfaces. The above does 8 surfaces.

- As you can see the vertex shader translates the patch upwards depending on the distance between the camera and the patch. This is needed to prevent z fighting.

- Finally, enable alpha blending with SrcBlend set as Destination Color and DestBlend set as Source Color.

There you have it. :) Hope that helps.

Share this post


Link to post
Share on other sites
Nice.

I can't understand step 3. I know you're altering pixel values, but by which value?

The detail textures that you're passing in to the shaders are the ones that were used when you created the base texture?

Which means that all that you're doing is texture blending with per-vertex alpha masks? If so why go through the process of creating them again, why not use the ones you used when you blended together all the layers to form the base texture?

That's pretty much what i figured out but what got me pondering was the fact that crytek never stopped at 8 layers and doing multiple rendering passes is a not so welcome performance hit so i've been pondering on a better solution.

Am i missing something? =)

[Edited by - Cybrosys on March 29, 2005 8:26:06 AM]

Share this post


Link to post
Share on other sites
Quote:
I can't understand step 3. I know you're altering pixel values, but by which value?


The value of the layer mask and the value of the surface mask at that point.

eg.



//
// Reduce or increase the mask pixel of the surface
//
int color;

if (layersSurface)
{
color = static_cast<int>(pSurfacePixels[surfaceOffset]) + (pLayerPixels[layerOffset] & 0xFF);

if (color > 255)
color = 255;
}
else
{
color = static_cast<int>(pSurfacePixels[surfaceOffset]) - (pLayerPixels[layerOffset] & 0xFF);

if (color < 0)
color = 0;
}

//
// Copy the pixel color to the mask
//
pSurfacePixels[surfaceOffset] = static_cast<byte>(color);



Quote:
Are you forgetting to mention that you're passing 8 textures in aswell, and the surface masks is pretty much a per vertex alpha mask?


Sorry I thought that by reading the shader code that would be a given. Yes you need to pass the 8 (or however many) detail textures to the shader.

Yes in the end the surface masks pretty much act as a per vertex alpha mask to each detail texture.

Quote:
The detail textures that you're passing in to the shaders are the ones that were used when you created the base texture?


No the detail textures are a lot different to the base textures. The base textures are stretched across the entire terrain and basically colour the terrain. The detail textures are 'de-saturated' so that they change the base color as little as possible and only add detail. At the moment I'm using Farcry's textures but will be replacing them with my own soon.

Quote:
Which means that all that you're doing is texture blending with per-vertex alpha masks? That's pretty much what i figured out but what got me pondering was the fact that crytek never stopped at 8 layers and doing multiple rendering passes is a not so welcome performance hit.


Do you really need more than 8 detail textures? Like I said, its rare that you'd need more because the base texture adds enough variety that you dont need anymore. Remember its a game, not a real world simulation.

One way I could think of that you could get around this is to store the surface masks as seperate textures instead of storing them in the vertices. You'd then pass the surface masks and detail textures to the shader, 4 at a time, over multiple passes.

eg.


PS_OUTPUT DetailTexture4PS(VS_OUTPUT vs)
{
PS_OUTPUT o;

//
// Multiply each detail texture by its surface mask and add the result to the resultant colour
//
o.color = tex2D(SurfaceMask0Sampler, vs.uv0) * tex2D(DetailTexture0Sampler, vs.uv1 * g_detailTexture0Scale);
o.color += tex2D(SurfaceMask1Sampler, vs.uv0) * tex2D(DetailTexture1Sampler, vs.uv1 * g_detailTexture1Scale);
o.color += tex2D(SurfaceMask2Sampler, vs.uv0) * tex2D(DetailTexture2Sampler, vs.uv1 * g_detailTexture2Scale);
o.color += tex2D(SurfaceMask3Sampler, vs.uv0) * tex2D(DetailTexture3Sampler, vs.uv1 * g_detailTexture3Scale);

//
// Blend between the texture colour and grey based on the exponential blend
//
o.color.rgb = lerp(o.color.rgb, float3(0.5, 0.5, 0.5), vs.diffuse.a);

return o;
}



I did this at the start and it wasn't much slower (if at all) over the vertex based version I posted before. I was sending in 6 surface masks and 6 detail textures.

Good luck.

Share this post


Link to post
Share on other sites
I understand the algorithm for the detail texture, however i still have some questions if it's ok.

To recap, the detail texture is blended ontop of the base texture? I.e. you don't want to move the detail polygons to far off the ground because that would move the detail "pattern" off target a bit, am i correct? How exactly do the de-saturated textures get created from the base texture? And if all that you need to do is blend the "pattern" texture on, why the complex layer creation formula, which i don't understand because i don't know the basis for Layer + Surface = color . Are the Layer and Surface just grayscale alpha values or do they contain the actual color values? Why do you alter the alpha value and not just set it to 0xFF if it's supposed to be blended there?

(What's confusing me the most is that i'm having a hard time knowing which layer you're referring to when you write Layer/Surface etc, if it's the textures' pixelcolor or the masks' alpha values.. and which, detail or base component layers heh)

Thanks for the help, it's apreciated.

[Edited by - Cybrosys on March 29, 2005 9:37:06 AM]

Share this post


Link to post
Share on other sites
Just an extension to what was already suggested:
You dont need to limit yourself to 8 layers max, and you dont need to render the whole terrain with 8 layers. My renderer decides this on a per chunk basis:
Go through all layer-alpha-maps for a certain chunk, and only store the alpha of a maximum of 4 layers in the color array ( which is then passed on a per-vertex basis to the vertexshader ). This technique ensures that you dont need to render a single chunk multiple times. The speed hit is also not very hard because shader switching ( between 1,2,3,4 layers ) allows you to only render what is really needed.

Share this post


Link to post
Share on other sites
Started playing with photoshop when i got back to understand what you meant with de-saturate, and it pretty much mean make it grayscale, which helps the blending onto the base texture so that there won't be any color variations.

However i unpacked all the Far cry textures and went to check out their detailmaps and i couldn't find a single one that was de-saturated, maybe i missed it or maybe it's something they do when you "compile" the map.

Share this post


Link to post
Share on other sites
Quote:
To recap, the detail texture is blended ontop of the base texture? I.e. you don't want to move the detail polygons to far off the ground because that would move the detail "pattern" off target a bit, am i correct?


Correct. The artifacts get worse the higher the patch with the detail texture is a above the original patch.

Quote:
How exactly do the de-saturated textures get created from the base texture? And if all that you need to do is blend the "pattern" texture on, why the complex layer creation formula, which i don't understand because i don't know the basis for Layer + Surface = color.


The detail textures are totally different to the base texture and the textures used to create the base texture. Take a look at these diagrams I drew up for you.

Creation of the base texture:

Free Image Hosting at www.ImageShack.us

Creation of the surface masks:

Free Image Hosting at www.ImageShack.us

Example of the detail textures:

Image Hosted by ImageShack.us

Quote:
Started playing with photoshop when i got back to understand what you meant with de-saturate, and it pretty much mean make it grayscale, which helps the blending onto the base texture so that there won't be any color variations.


Not fully greyscale, but heading towards greyscale. Take a look at the two above.

Hope thats enough to get you well underway :)

[Edited by - Buttza on April 16, 2005 1:35:29 AM]

Share this post


Link to post
Share on other sites
Quote:
Original post by Dtag
Just an extension to what was already suggested:
You dont need to limit yourself to 8 layers max, and you dont need to render the whole terrain with 8 layers. My renderer decides this on a per chunk basis:
Go through all layer-alpha-maps for a certain chunk, and only store the alpha of a maximum of 4 layers in the color array ( which is then passed on a per-vertex basis to the vertexshader ). This technique ensures that you dont need to render a single chunk multiple times. The speed hit is also not very hard because shader switching ( between 1,2,3,4 layers ) allows you to only render what is really needed.


Good suggestion, though what happens if there are more than 4 surfaces on a patch?
I know its rare, but possible. Do you apply this restriction via your level editor?

What do you perform to determine which 4 surfaces should be placed in the diffuse map?

Share this post


Link to post
Share on other sites
Buttza:

Hi there, I looked at your screenshot with interest. However I find your method a bit hihg-powered for the results it gives. It's pretty but when you take away the (very nice) water it's very similar to what I have, using two textures only in a single pass on a TNT2 with no shaders. Maybe that's just because you aren't fully utilising the capabilities of your engine with the map shown in that shot - it's basically just sand and grass. I didn't follow all of your description since I don't know shaders - how many different groundtypes can you use? I mean how many different detail textures for grass, sand etc are supported in a level? Did you say 4/8, and does just rely on the GPU having 4/8 texture units? When you mention multiple passes is that so you can handle more detail textures?
Finally, you have 'layer 1 mask', 'layer 2 mask' etc which tells how much of each groundtype to use, are these masks stetched over the whole map or tiles of some kind?

Feel free to PM me if you fancy a chat about these things and don't want to derail the thread...

Share this post


Link to post
Share on other sites
Thanks alot Buttza, you've been very helpful and the images realy made the difference in understanding the DT layer mask creation algorithm.

Btw, what water technique is it that you're using on the first screenshot you posted? Just interrested because it resembles the Far cry water alot.

Share this post


Link to post
Share on other sites
Well my algorithm may be a little "lame" because it doesnt take the overall intensity of the layers into account, but all in all, i have only found 1 small error in the whole farcry demo map. This is how it works:


// This shows the assignment of different DetailLayers to the color register
int iLayers[4] = {-1,-1,-1,-1};
// Chunksize=33x33 vertices for me ( same as farcry )
unsigned char* pSpace=new unsigned char[m_iChunkSize*m_iChunkSize];
// Initialize pSpace to 255 here! ( i took some part of the code out )


int v=0;
// make sure youve got the right traversal direction here!
for (int m=m_DetailLayers.size()-1;m>=0;m--)
{
// this variable determines to which color the next found layer is assigned
int iWriteToColor=-1;
// Check if the layer is already assigned to this chunk, if so, get the index
for (int lay=0;lay<iLayerCount;lay++)
{
if (!stricmp(m_DetailLayers[iLayers[lay]]->chDetailTexture,m_DetailLayers[m]->chDetailTexture))
{
iWriteToColor=lay;
break;
}
}

v=0;
tDetailLayer* detlay=m_DetailLayers[m];
for (int r=0;r<m_iChunkSize;r++)
{
for (int c=0;c<m_iChunkSize;c++)
{
// these are the actual xy positions on the alpha map of the layer
int x=i+c;
int y=j+r;
unsigned char intensity=GetIntensity(detlay->pMask,x,y);
if (intensity==0) continue;

unsigned char& iSpace=pSpace[r*m_iChunkSize+c];
// this means we have used up our space... since the total intensity ( r+g+b+a should be =255 )
if (iSpace<=0) continue;

// If the layer is not assigned to the chunk yet, check if we are below the layer limit (4)
// and exclude very low values
if (iWriteToColor==-1)
{
if (iLayerCount<4)
{
// this makes sure that low intensities at a point dont insert a whole layer into the current chunk
if (intensity<10) continue;
// now we've found the color where we want to write to
iWriteToColor=iLayerCount;
iLayers[iLayerCount++]=m;
}
// This means exactly that there are not enough layers (4 max ) to hold the current layer as well. This may result in visual problems
//else
//CLog::Inst()->Error("CTerrain::LoadHeightmap Layer overflow!");
}
if (iWriteToColor!=-1)
{
// make sure the intensity is not bigger than the space we have ( 255 max because r+g+b+a=255 )
if (iSpace-intensity<0) intensity=iSpace;
// this actually stores the intensity in the color of the current vertex.
pColors[((r*m_iChunkSize+c)*4)+iWriteToColor]+=intensity;
iSpace-=intensity;
}
}
}

}


Share this post


Link to post
Share on other sites
Quote:

However I find your method a bit hihg-powered for the results it gives. It's pretty but when you take away the (very nice) water it's very similar to what I have, using two textures only in a single pass on a TNT2 with no shaders. Maybe that's just because you aren't fully utilising the capabilities of your engine with the map shown in that shot - it's basically just sand and grass.


By using one single base terrain texture you get variety in the terrain that you just don’t get with multiple textures blended using alpha maps. It also allows you to apply static lighting and shadows, eg. from the sun, on the base texture instead of in a separate lightmap.

This allows you to render the terrain in one pass using one texture. Of course unless the base texture is of ultra high resolution it will appear blurry. This is why we use detail textures to 'simulate' detail up close. Take a look at the game FarCry for a demonstration of this technique in action because as you pointed out my screenshot doesn't show it off fully.

Quote:

I didn't follow all of your description since I don't know shaders - how many different groundtypes can you use? I mean how many different detail textures for grass, sand etc are supported in a level? Did you say 4/8, and does just rely on the GPU having 4/8 texture units? When you mention multiple passes is that so you can handle more detail textures?
Finally, you have 'layer 1 mask', 'layer 2 mask' etc which tells how much of each groundtype to use, are these masks stetched over the whole map or tiles of some kind?


Unlimited number of layers. Fixed number of surfaces. Read my descriptions above on the differences between the two. FarCry limits the number of surfaces to 7. In my engine I limit it to 8, though theoretically it could be unlimited.

Quote:

Thanks alot Buttza, you've been very helpful and the images realy made the difference in understanding the DT layer mask creation algorithm.

Btw, what water technique is it that you're using on the first screenshot you posted? Just interrested because it resembles the Far cry water alot.


No problem Cybrosys. The water looks similar because it’s using the same technique as FarCry. Read up on the many threads here on gamedev regarding water or read Yann L’s article on Realistic Natural Effect Rendering: Water I. My water is based on that article.

Quote:
Well my algorithm may be a little "lame" because it doesnt take the overall intensity of the layers into account, but all in all, i have only found 1 small error in the whole farcry demo map.


Nice. As I thought you’re checking each surface mask for an intensity > 10. I'll remember that if I need to optimize. Thanks.

Share this post


Link to post
Share on other sites
Oh no sorry my definition of passes is wrong. I meant setting two texture units, then doing one pass for each ground type. So I draw the base colour map first as you do. Then blend the detail maps in, but without needing blend textures or shaders.

Share this post


Link to post
Share on other sites

Hello!
All your explanations are usefull, this technique produces really nice screen shot!
I'm working on such technic, and i constantly search the way far cry works.

And i've understood (thanks) the difference between layers and surfaces.
In fact if i'm right the only final difference between the two will be the base texture, which is alpha blended, who produces more or less noise, and more or less colour variation.
Simple.
Thanks for your explanations because i didn't understand how they can fit so much alpha textures in memory (i try to fit in 64 Mb of graphic card memory). With 7 or 8 surfaces, it's much more simple.
Currently i use 1024*1024 alpha maps, but it lacks precision from far (it's beautiful when detail textured). So i'll change these maps' resolution.


I really think far cry doesn't use a per-vertex alpha variation, because if you look attentively in sand box, there are roads, and the roads have on their center a little bit of grass (and you can add/substract grass manually). The roads aren't decals, so i you alpha blend per vertex you will not have suffisent precision.


I saw recently that far cry uses fog to fade between detail texture to base texture, it's really visible when you disable fog in sand box.

To make transition between the two, i think it's possible they have used vertex
shaders to change alpha per vertex (0 or 1) according to the distance from the camera.
I never done shaders so i could just imagine.

When we approach ground we see that the spheric zone where we are shows detail texture, the limit isn't chunk-based, but vertex-based (see in wireframe with no fog).
Currently i'm rendering the entire chunk with a global-per-chunk-alpha to make the detail texture appearing softly, but when the chunk near it isn't detail textured (bounding box too
far), it's really ugly.
I would need to know the nearest vertex of a chunk from the camera,
but i don't know how to do that for the moment.
When the rest will be nice (lot of work), i'll look for the vertex shaders.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this