Jump to content
  • Advertisement
csisy

UE4-like shaders from materials (theory)

This topic is 1289 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi,

 

I've just checked out the new UE4 (Unreal Eninge 4) and its material editor and I'm thinking how it works.

Is a new shader compiled for each material? If yes, isn't the lots of shader switches affecting the performance? How can it be optimized?

 

Edit:

According to this paper, the material is compiled into shader code.

 

This node graph specifies inputs (textures, constants), operations, and outputs, which are compiled into shader code

Edited by csisy

Share this post


Link to post
Share on other sites
Advertisement

You definitely don't need to compile to a new shader if just some data of the material differs (e.g. two materials share everything except for one diffuse albedo texture), but I don't know the UE4 source so I can't say what they do. But it's not at all atypical to compile new shaders when certain things in a material or related rendering parameters change and end up with systems that have thousands, tens of thousands (or much more) shader combinations. In this blog post, aras_p of Unity mentions 1 million variations for a single shader, which seems kind of ludicrous to me but apparently it happens. 

 

The reason this is doable at runtime is because you only use very few of those resulting variations at the same time in a scene. Undoubtedly if you assembled a scene that needed to go through 1M shader switches every frame, your CPU wouldn't react kindly. 

Share this post


Link to post
Share on other sites

Wow, that sounds crazy! mellow.png

 


The reason this is doable at runtime is because you only use very few of those resulting variations at the same time in a scene. Undoubtedly if you assembled a scene that needed to go through 1M shader switches every frame, your CPU wouldn't react kindly. 

That's that makes me disturbed. I thought switching to a new shader is expensive; I have to set up new attributes, uniforms and everything (I'm targeting OGL 2) which is related to the new shader even if only 1 object is drawn by that. And it could happen: in this tutorial series a single wood material is created and maybe it's used by only 1 object in the whole scene.

Share this post


Link to post
Share on other sites

The Unreal engine does optimize the shaders for you when you apply them.

 

Each material consist of small shaders doing something simple, these small shaders combined to form a large shader for a material, this is why the unreal will build multiple shaders when you add a new kind of material.

This way you can have two materials that share a single part of a shader, lets say Wood and Metal, thy share a vertex shader but each has a unique pixel shader. Now instead of using four shaders you use three.

 

Now if you have a red chair and you copy it's material to make a green chair, thy will share most of there shaders. However this doesn't change the fact that you added more materials.

 

If you wan't to make say a large swarm of a hundred aliens who very in color, you will need hundreds of materials and this will cause a large performance hit.

Luckily Unreal has a simple and effective solution, material instancing: https://docs.unrealengine.com/latest/INT/Engine/Rendering/Materials/MaterialInstances/index.html

 

If you are familiar with animation or mesh instancing this works the same. 

A over simplified explanation is that you render the same thing with slight changes.

Share this post


Link to post
Share on other sites

Oh yea, I didn't write the original post accurate enough. I meant new shader for each material which differs not just the values but the "structure" as well. So the same as you wrote here. :) BTW, this is a pretty flexible system, I love it. Has anyone tried to implement similar system?

 

Thanks for your replies! :)

Share this post


Link to post
Share on other sites

If by "structure" you mean the shader as a text file, like:

 

void main() {			
	gl_TexCoord[0] = gl_MultiTexCoord0;
	
	// Set the position of the current vertex 
	gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;
}

and

void main() {			
	// Set the position of the current vertex 
	gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;

	gl_TexCoord[0] = gl_MultiTexCoord0;
}

are two files, yet thy are the same shader.(This is GLSL not for Unreal)

 

Unreal's material has no problem understanding that it's the same shader.

Unreal understands shaders as a group of instructions so both these are the same vertex shader with two instructions.

 

When unreal turn materials into shaders it doesn't add instructions that exists, instead it refers to the ones that exist.

However if you have some thing like:

UV = (vec3(0,0,0)) it's a instruction.

and !=

UV = (vec3(0,0,1)) so both instructions will be saved.

All shaders using UV = (vec3(0,0,0)) will refer to the first instruction, so no unnecessary data is stored.

 

In the end if you have a two vertex shader that use UV = (vec3(0,0,0)) and thy each uses other instructions that differ then you will still have two shaders.

 

For two shaders to be considered the same thy must have the exact same instructions.

Edited by Scouting Ninja

Share this post


Link to post
Share on other sites

 

Has anyone tried to implement similar system?

 

I have. It's a very intuitive way to work with shaders. The beauty of this is that you can design your nodes and shader structure such that the artists/users can customize exactly what you want them to customize, while maintaining control over the underlying structure of the shader, as opposed to generating the entire shader code only from nodes and requiring users to hook certain nodes in certain ways.

 

I am planning to work more on the system and release it as middleware for game developers so they can integrate it in their editors/pipeline/engines. Unfortunately I'm busy with another project currently. Here is a screenshot:

 

shader_editor.png

 

Some shader parameters can be dynamic and some can be "static". Changing a static parameter will require recompiling the shader, thus resulting in a new permutation of the shader. Another beauty of this system is makes it easier to create a platform-independent shader solution.

Share this post


Link to post
Share on other sites


Has anyone tried to implement similar system?

I built one a while back.

 

It basically starts chunks of pre-written shader code, fills in a bunch of variables in the text, and sticks them all together in a single file - the only tricky part is mapping inputs to outputs, and you can create a GUI interface to let the user manage those.

Share this post


Link to post
Share on other sites


I have. It's a very intuitive way to work with shaders

Agree. Oh, and your tool looks pretty cool. happy.png

 


I built one a while back.

Do you have any reference? My main problem is just how I can identify that two materials should use the same shader with different uniform values. Or let it be controlled by the user? Like in UE4, they have material instances which uses the same "base material" aka shaders.

Share this post


Link to post
Share on other sites

My main problem is just how I can identify that two materials should use the same shader with different uniform values. Or let it be controlled by the user? Like in UE4, they have material instances which uses the same "base material" aka shaders.

A shader is any unique (legal) combination of nodes. A material is that shader plus all the instance data required to render it (uniforms, textures, etc). i.e. you need strong separation between the functionality of a given block, and the instance data it requires at runtime.

Things get fuzzy if you start trying to hard code variables as constants in the output, and all that, so I'd advise not doing that in the first cut of your system.

Share this post


Link to post
Share on other sites

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!