Jump to content

  • Log In with Google      Sign In   
  • Create Account

We're offering banner ads on our site from just $5!

1. Details HERE. 2. GDNet+ Subscriptions HERE. 3. Ad upload HERE.


strategy for automatic shader compilation caching


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
3 replies to this topic

#1 gjaegy   Members   -  Reputation: 104

Like
0Likes
Like

Posted 04 April 2012 - 05:29 AM

Hi,

I am trying to reduce the load time of my game.

Right now, I compile all the shader at runtime, when the game is started, which takes quite some time.

I use D3DX10CompileFromMemory to compile my shaders.

I am thinking about writing the content of the blob returned by this function on disk, the first time the shader is being compiled, and re-use it next time the game is started.

This is very straightforward.

However, what I also would like to achieve, is to automatically detect any change in the shader source, and re-compile the shader if any change occured.

So, my initial idea was to save two files, the source code version used for compilation, and the resulting compiled shader blob.
At runtime, I would load the shader source file, and compare it with the source version used for compilation. If there is no difference, i can use the pre-compiled blob, otherwise I compile the shader again.

However, this approach gets a bit more complicated when considering "#include" directives, along with the usage of a "ID3D10Include" object.

The only way I see would be to manually resolve all the #include" directives, in order to get a "#include"-free HLSL source code I could use to compare the current version with the pre-compiled cache version.

Is there any other option I haven't seen ? How do you guys handle this problem ?
Gregory Jaegy[Homepage]

Sponsor:

#2 Nik02   Crossbones+   -  Reputation: 2883

Like
0Likes
Like

Posted 04 April 2012 - 06:41 AM

You could use a hierarchical cache - a tree (or more accurately a forest) of the dependencies. If any of the roots (included files) of any leaf (source files that include other files) change, you need to invalidate all the leafs of that root. The tree structure could be precalculated, and it is not very difficult to parse include directives to build the tree to begin with.

That said, if you are using custom ID3D10Include objects, the hierarchy determination could become unreasonably complex.

Niko Suni


#3 gjaegy   Members   -  Reputation: 104

Like
0Likes
Like

Posted 05 April 2012 - 07:51 AM

Hi Nick,

thanks for your answer. I think I have found a good trade off solution.

Basically, I am resolving all includes by myself. This results in a big HLSL source code text with no #include directive anymore.

I then check the shader cache (which contains a pair of "hlsl source code" / "compiled byte code" files for each original shader).

If an entry already exists in the cache, I compare the current source code text with the "hlsl source code" file. If no change could be found between the two strings, I assume I can safely load the existing "compiled byte code" file and use it.

Otherwise the shader gets compiled, and the result saved on disk ("hlsl source code" / "compiled byte code" files).

It seems to work pretty well.
Gregory Jaegy[Homepage]

#4 Nik02   Crossbones+   -  Reputation: 2883

Like
0Likes
Like

Posted 06 April 2012 - 12:42 AM

Yea, the preprocessing is usually faster than the actual compiling.

Niko Suni





Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS