Sign in to follow this  
Followers 0
Vilem Otte

Do you extend GLSL?

18 posts in this topic

I would like to know if anyone else out there extends GLSL by some pre-processing by C++ code? If you do, how much preprocessing do you use?

Do you implement includes? Do you implement "target" specification in shader (e.g. which code path to use with "shader model 3.0", with "shader model 4.0", etc.) ... e.g. somewhat extending GLSL to allow as much stuff as HLSL allows us to do.
0

Share this post


Link to post
Share on other sites

Yes, I extend it (mostly includes, switches, global constants etc.) using a pre-processor (in my case mcpp). Still you need to remember, that GLSL itself contains some pre-processing powers to handle e.g. different shader model support.

2

Share this post


Link to post
Share on other sites

I have a hack job include/prefix system that is just barely workable. Personally I find it absurd that GLSL supports linking but not file includes. If I had time, I'd probably retool with a full parser/effect type system. I've also toyed with simply jumping to hlsl2glsl as a potentially more sane alternative.

Edited by Promit
1

Share this post


Link to post
Share on other sites

Like Promit, I have a somewhat hacky preprocessor which manages #includes but not much else. For experimental and small scale projects you don't really need anything more. A serious renderer would probably require a more complete system for things like generating shader permutations. External #defines are a good mechanism for that sort of thing. 

1

Share this post


Link to post
Share on other sites

You don't even need file includes in GLSL; glShaderSource has a pretty good "include" mechanism built-in as is.

const char *globalShaderIncludes = "stuff"; // read this from a file or resource if you want
const char *thisShader = "more stuff"; // load this as normal for the current shader

const char *shaderSources[] = {globalShaderIncludes, thisShader};

glShaderSource (shaderObject, 2, shaderSources, NULL);
3

Share this post


Link to post
Share on other sites

I am currently working on simplified GLSL effects, kinda like http://gleffect.sourceforge.net/ but much simpler and focused on GLSL 330 and higher. I am writing it myself to plug it into my engine, I didn't want the external dependencies that GLeffect! includes. I also wanted something that works perfectly with my engine.

1

Share this post


Link to post
Share on other sites

Tsk.
 

You don't even need file includes in GLSL; glShaderSource has a pretty good "include" mechanism built-in as is.
const char *globalShaderIncludes = "stuff"; // read this from a file or resource if you want
const char *thisShader = "more stuff"; // load this as normal for the current shader

const char *shaderSources[] = {globalShaderIncludes, thisShader};

glShaderSource (shaderObject, 2, shaderSources, NULL);

That is what I'm doing and described as a "hack job". It has a variety of practical issues. Here's a hint: where does #version go, particularly when using heterogeneous shader versions? Also, it requires you to remember to do things like add terminating newlines (like a C preprocessor from twenty years ago). It does work, barely.
 

How could GLSL even have file includes? GLSL doesn't even know about source files. You would either need a feedback mechanism telling the host application which include file is needed

D3DX has done this for many years and many titles have shipped like that. It's a very reasonable approach. As far as I know, there's no OpenGL method of any sort that accepts and uses a callback though. I think there's possibly some reason they've resisted them, because it's the only way to explain...

or the host application would have to upload each and every file that might possibly be required by the current source file. To get that right, you would have to parse your GLSL source files anyway to find out which includes you need.

And that is what they actually did. Seriously, how is such a significant API designed this poorly? It's beyond belief.

Edited by Promit
3

Share this post


Link to post
Share on other sites

Thanks for the input... I'm glad to know that more people think about limitations of GLSL (as opposed to F.e. HLSL).

 

So far I'm deciding whether I need some more preprocessing than just includes and running specific code for specific shader models (both is already implemented).

0

Share this post


Link to post
Share on other sites

I'm gradually coming round to the idea that for anything more than simple demos, writing shaders by hand is for the birds.

 

A decent graph-based material designer, and a runtime component that generates full shaders from the material definition, is really very pleasant...

2

Share this post


Link to post
Share on other sites

opengl can't read files, ever, because it's not built for a specific OS, architecture, etc.

it's got wide use in medical equipment, and honestly

reading line by line, trimming it (removing whitespace on each side) and testing for "#include " is hardly much work

 

really, the only gripe is with C++ SL itself which has a faulty getline() method that doesn't work the same way on linux and windows

1

Share this post


Link to post
Share on other sites

Thanks for the input... I'm glad to know that more people think about limitations of GLSL (as opposed to F.e. HLSL).

 

So far I'm deciding whether I need some more preprocessing than just includes and running specific code for specific shader models (both is already implemented).

You have a problem: GLSL is kind of stupid. You can a) work around this problem as needed b) devise a new block of code to solve this problem. You're using a and it already works. If you choose b, you now have two problems. That's why most of us are using hackjob include mechanisms; writing a more elaborate parser is a pain in the ass and most of us don't want to do it despite the quirks that are there now. It requires more code that is difficult to write correctly and robustly.

 

At some point your shader will crash or somehow malfunction. Do you really want to be wondering whether the problem is the shader, the driver, or your intermediate compiler?

Edited by Promit
1

Share this post


Link to post
Share on other sites

#Promit You exactly summed the things I've been thinking about. So far I came up with a bit creative solution for the game engine - as it heavily utilizes modularity and system internally compiles just GLSL shader code, I decided to "off-load" this work to plug-in, and different plug-ins will take care of F.e. just loading the files (and doing no preprocessing), or F.e. doing some pre-processing ... 

 

E.g. you have set of plugins, through one of these you load shader file, their output is GLSL shader code (which I can dump to some text file), from there the engine compiles this shader code and links it (possible errors are dumped to log files).

0

Share this post


Link to post
Share on other sites

I thought about making a play on words of "extend opengl" (keyword in java) but meh, no ideas.

 

I'm gradually coming round to the idea that for anything more than simple demos, writing shaders by hand is for the birds.

 

A decent graph-based material designer, and a runtime component that generates full shaders from the material definition, is really very pleasant...

 

I've read that there are some people against those, saying that if the visual designer is too high level you can't do interesting things with it, and that if its too low level its better to write the shader and be done with it.

 

Anyway I've seen a few shader editors where you can define new nodes made from other low-level'ish nodes and have all the functionality in one place, a library of shader nodes. It does seems pretty handy.

0

Share this post


Link to post
Share on other sites

I'm redoing my GL support in the near future, but my plan is to parse my shader code offline, converting it into an intermediate representation, and then using that to generate streamlined GLSL code for the engine to use (with all includes/preprocessor statements resolved, basic optimisations applied, whitespace/variable names trimmed, etc).

 

Screw using raw GLSL.

1

Share this post


Link to post
Share on other sites

I'm using pure GLSL with the built-in preprocessor, plus I process #includes manually in cpp, but that's it. With this I've implemented an uber-shader system, I personally didn't feel the need for much more.

0

Share this post


Link to post
Share on other sites

 

As far as I know, there's no OpenGL method of any sort that accepts and uses a callback though.

 

See GL_ARB_debug_output.

 

Ah of course. And I added debug output support recently, too. Silly of me to forget about it. But it just makes it that much harder to explain the bizarre include extension.

1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0