Sign in to follow this  
gjaegy

strategy for automatic shader compilation caching

Recommended Posts

Hi,

I am trying to reduce the load time of my game.

Right now, I compile all the shader at runtime, when the game is started, which takes quite some time.

I use D3DX10CompileFromMemory to compile my shaders.

I am thinking about writing the content of the blob returned by this function on disk, the first time the shader is being compiled, and re-use it next time the game is started.

This is very straightforward.

However, what I also would like to achieve, is to automatically detect any change in the shader source, and re-compile the shader if any change occured.

So, my initial idea was to save two files, the source code version used for compilation, and the resulting compiled shader blob.
At runtime, I would load the shader source file, and compare it with the source version used for compilation. If there is no difference, i can use the pre-compiled blob, otherwise I compile the shader again.

However, this approach gets a bit more complicated when considering "#include" directives, along with the usage of a "ID3D10Include" object.

The only way I see would be to manually resolve all the #include" directives, in order to get a "#include"-free HLSL source code I could use to compare the current version with the pre-compiled cache version.

Is there any other option I haven't seen ? How do you guys handle this problem ?

Share this post


Link to post
Share on other sites
You could use a hierarchical cache - a tree (or more accurately a forest) of the dependencies. If any of the roots (included files) of any leaf (source files that include other files) change, you need to invalidate all the leafs of that root. The tree structure could be precalculated, and it is not very difficult to parse include directives to build the tree to begin with.

That said, if you are using custom ID3D10Include objects, the hierarchy determination could become unreasonably complex.

Share this post


Link to post
Share on other sites
Hi Nick,

thanks for your answer. I think I have found a good trade off solution.

Basically, I am resolving all includes by myself. This results in a big HLSL source code text with no #include directive anymore.

I then check the shader cache (which contains a pair of "hlsl source code" / "compiled byte code" files for each original shader).

If an entry already exists in the cache, I compare the current source code text with the "hlsl source code" file. If no change could be found between the two strings, I assume I can safely load the existing "compiled byte code" file and use it.

Otherwise the shader gets compiled, and the result saved on disk ("hlsl source code" / "compiled byte code" files).

It seems to work pretty well.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this