• Advertisement
Sign in to follow this  

C++ Embeding shader source to code or use external file

Recommended Posts

I am shipping my app. I am facing the issue of shaders and their representation. Should I embed shaders to code directly or use external file and load them via "fopen"?

For the first approach - in binary

1) faster load time

2) shaders are "less easy to copy" (but only for beginners, the text is visible in hex binary)

Against the first approach - in binary

1) Larger binary - more memory needed

2) Shader is basically loaded only once at startup

 

--

For the second approach - external file

1) Easier shader update - just edit file, no rebuild needed

2) Smaller binary - less memory needed

3) Unified Virtual File System - load everything with one logic

Against the scond approach - external file

1) Slower load (especially on mobile devices, where files are in resource - unpacking them may be slow)

 

What are your thought? For me, I am more inclined to the second approach - extrenal file, but maybe I am missing something?

 

Edited by Martin Perry

Share this post


Link to post
Share on other sites
Advertisement

In the professional world shaders are one of many assets, and shaders are in their own files.

When built for shipping shaders get processed and bundled together with all the other assets, meaning they are small pieces of a very large file.

I thought it was unexpected that you mention slow load on mobile devices. Are you working on a mobile game?

Share this post


Link to post
Share on other sites

I did both of what you described while I added "normal" shaders to a resource package, I also added them to code as plain text for some little build in feature (Debug-Drawing in GL).

I would recommend to ship them as external resource because packages could be maintained easier, you otherwise would need to rebuild your source when changing a shader and that may propably take longer compile times, also for bigger projects, your assembly file will be less in size as when you add the shader source to code.

Yes, GLSL takes a "relative" long time to compile because it is plain text so your devices graphics chip needs first to get all the text transferred before it is able to split, tokenize and parse it before it is able to translate the source into a GPU ready assembly program. You could not do anything very straight forward here because GLSL takes plain text source but you could cache shaders as binary output on disk and reuse it if possible. This is a lot faster because anything is already processed but the binary is platform and vendor dependent so you wont be able to ship that without a guarantee to work.

Access time is also something you should optimize. Make your assets a package and align that to the disk cache so you will reduce I/O-Buffer usage. In C++ you are also able to reserve page memory for your files (Memory Mapped Files) using CreateFileMapping on Windows and mmap on Linux/Unix (Android is Unix). This will enable your program to tell the OS to load a specific part of the file into a memory page (thats why you should align/padd your file) and increase access time by several milliseconds. You could also access two or more files (as long as they fit into the "chunks" you loaded) at once, even in an asynchronous multithreaded way. (This are the tricks that AAA games use on console)

Share this post


Link to post
Share on other sites

Presently I compile my shaders (HLSL) straight to a header file and include them in my projects. The additional compile time is negligible (in my case), with the entire project taking eight seconds to build an exe.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this  

  • Advertisement
  • Advertisement
  • Popular Tags

  • Advertisement
  • Popular Now

  • Similar Content

    • By sergio2k18
      Hi all
      this is my first post on this forum.
      First of all i want to say you that i've searched many posts on this forum about this specific argument, without success, so i write another one....
      Im a beginner.
      I want use GPU geometry clipmaps algorithm to visualize virtual inifinte terrains. 
      I already tried to use vertex texture fetch with a single sampler2D with success.
       
      Readed many papers about the argument and all speak about the fact that EVERY level of a geometry clipmap, has its own texture. What means this exactly? i have to 
      upload on graphic card a sampler2DArray?
      With a single sampler2D is conceptually simple. Creating a vbo and ibo on cpu (the vbo contains only the positions on X-Z plane, not the heights)
      and upload on GPU the texture containing the elevations. In vertex shader i sample, for every vertex, the relative height to te uv coordinate.
      But i can't imagine how can i reproduce various 2d footprint for every level of the clipmap. The only way i can imagine is follow:
      Upload the finer texture on GPU (entire heightmap). Create on CPU, and for each level of clipmap, the 2D footprints of entire clipmap.
      So in CPU i create all clipmap levels in terms of X-Z plane. In vertex shader sampling these values is simple using vertex texture fetch.
      So, how can i to sample a sampler2DArray in vertex shader, instead of upload a sampler2D of entire clipmap?
       
       
      Sorry for my VERY bad english, i hope i have been clear.
       
    • By mangine
      Hello. I am developing a civ 6 clone set in space and I have a few issues. I am using Lua for the logic and UI of the game and c++ directx 12 for the graphics. I need a way to send information between Lua and c++ occasionally and was wondering what is the best and most flexible (and hopefully fast) way to do this. Don't forget that I also need to send things from c++ back to Lua. I know of a lua extension called "LuaBridge" on github but it is a little old and I am worried that it will not work with directx 12. Has anybody done something similar and knows a good method of sending data back and forth? I am aware that Lua is used more and more in the industry and surely plenty of AAA game programmers know the answer to this. I want a good solution that will hopefully still be viable code in a couple of years...
    • By owenjr
      Hi there.
      I'm pretty new to this and I don't know if it has been asked before, but here I go.
      I'm developing a game using SFML and C++.
      I would like to use the "Tiled" tool to load maps into my game but I don't actually find any tutorial or guide on how to exaclty use it (I know that I have to read an XML file and stuff). I just step into diverse projects that make all a mess. 
      Anyone knows where can I find good information to make my map loader by myself?
      Thanks in advantage!!
    • By MHG OstryTM
      Hello guys,
      I've released my game for the first time. I'm very excited about it and I hope you'll enjoy the game - Beer Ranger. It's a retro-like puzzle-platfromer which makes you think a lot or die trying. You have a squad of skilled dwarfs with special powers and your goal is tasty beer. There is a lot of traps as well as many solutions how to endure them - it is up to your choice how to complete the level! 
      Link to the project: Project site
      Link to the Steam site with video: Beer Ranger
      Have fun and please write feedback if you feel up to.
      Some screens: 




    • By francoisdiy
      So I wrote a programming language called C-Lesh to program games for my game maker Platformisis. It is a scripting language which tiles into the JavaScript game engine via a memory mapper using memory mapped I/O. Currently, I am porting the language as a standalone interpreter to be able to run on the PC and possibly other devices excluding the phone. The interpreter is being written in C++ so for those of you who are C++ fans you can see the different components implemented. Some background of the language and how to program in C-Lesh can be found here:

      http://www.codeloader.net/readme.html
      As I program this thing I will post code from different components and explain.
  • Advertisement