Shader cache

Started by
13 comments, last by Prune 13 years, 9 months ago
Hello,

I would like to implement a shader cache which will store all compiled shaders in a file. Shader will be re-compiled if source file has been changed - otherwise compiled shader will be used directly from the cache file.

I am not very familiar with OpenGL and I am looking for a function which will return compiled shader and another function which I could use to set compiled shader which was loaded from the cache.

Best regards,
Tomasz
Advertisement
just one question why?
My application uses many variations of several shaders and I need to access them at any time. For example - if I enable a point light, then several shaders need to be re-compiled to include point light computations. Currently it stops the whole application for a short while, because OpenGL driver needs to re-compile these shaders. Instead of re-compiling shaders, I would like to directly use them from a cache in a binary (compiled) format.
This is not possible under OpenGL.
I am a bit surprised that it's not possible to load a shader in a binary format, because as I know it's not a problem under DirectX.

Anyway, thanks for the answer.
Quote:Original post by Volgut
I am a bit surprised that it's not possible to load a shader in a binary format, because as I know it's not a problem under DirectX.

Well, tell that the ARB. People have been bitching for years to finally get binary blobs, and the ARB keeps coming up with idiotic "reasons" why we still don't have them.

OpenGL ES has them, although in somewhat limited way and on a purely optional basis.
Can you not do this with CG? I thought with CG you could get the compiled code back and reload it straight as the binary? I may be talking out my arse though, so probably best to ignore me.
This is not possible under any graphics library, including both OpenGL and DirectX. Different GPUs have completely different native instruction sets, and they change every few generations. You can't compile straight to binary and expect the code to work on more than one generation of GPUs from one vendor. The best you can do is compile to some generic intermediate format, and this is what DirectX provides. This saves the compiler from having to parse the input program, run syntactic analysis, and perform high-level optimizations (like inlining and dead code elimination), but the it will always still have to generate the final native code and perform low-level optimizations (like instruction scheduling and register allocation).
Quote:Original post by Yann L
Quote:Original post by Volgut
I am a bit surprised that it's not possible to load a shader in a binary format, because as I know it's not a problem under DirectX.

Well, tell that the ARB. People have been bitching for years to finally get binary blobs, and the ARB keeps coming up with idiotic "reasons" why we still don't have them.

OpenGL ES has them, although in somewhat limited way and on a purely optional basis.


Without being too much into the topic, I guess ppl cried for it because they hoped for protection. But then, OpenGL seems to be a non-proprietary standard: How should their content be more safe in a binary blob if that binary blobs codec's open? Thus I'd conclude that OpenGL is simply not for them.

If blobbing was more about performance, i.e. about readily tokenized but non-compiled code, then ignore my post.

And in case of a compiled blob: I think that would be in the way of future compatibility and code-scalability w.r.t. improving hardware, as compilation in this case would also be information loss.
Quote:Original post by Eric Lengyel
This is not possible under any graphics library, including both OpenGL and DirectX. Different GPUs have completely different native instruction sets, and they change every few generations.

Well, a popular idea is a local binary shader cache. You ship your product with uncompiled high level shaders. On the first run, these shaders are compiled to a native binary, which are put into a cache tied to the GPU ID. On subsequent runs, the time intensive compilation can be skipped. If the GPU is swapped, the cache is rebuilt.

Quote:Original post by phresnel
Without being too much into the topic, I guess ppl cried for it because they hoped for protection. But then, OpenGL seems to be a non-proprietary standard: How should their content be more safe in a binary blob if that binary blobs codec's open? Thus I'd conclude that OpenGL is simply not for them.

Yes, IP protection is a part of the issue. Performance is the other. OpenGL is not open. It is however standarized, just as D3D is. Even a standarized intermediate language can improve IP protection by a lot, compared to shipping uncompiled GLSL. At least it's much harder to reverse engineer than being presented plain source.

I for myself would go a lot further, advocating asymmetric on-die decryption and DRM. I firmly believe that more and more IP lies in the shaders, and that IP needs to be protected. But an IL would already be a first step. At this point, obfuscating GLSL (where you replace all variable/attribute/uniform/etc names with GUIDs) is the only, highly unsatisfactory way.

About OpenGL not being for them. I'd like to remind you for whom OpenGL was originally designed for and still is: the CAD/CAM and defense industries.

Quote:Original post by phresnel
If blobbing was more about performance, i.e. about readily tokenized but non-compiled code, then ignore my post.

Of course it's also about performance. The idea of a native binary blob is exclusively about performance, since shader source is still initially required.

This topic is closed to new replies.

Advertisement