Jump to content

  • Log In with Google      Sign In   
  • Create Account

Do you extend GLSL?


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
18 replies to this topic

#1 Vilem Otte   Crossbones+   -  Reputation: 1561

Like
0Likes
Like

Posted 18 July 2013 - 06:17 PM

I would like to know if anyone else out there extends GLSL by some pre-processing by C++ code? If you do, how much preprocessing do you use?

Do you implement includes? Do you implement "target" specification in shader (e.g. which code path to use with "shader model 3.0", with "shader model 4.0", etc.) ... e.g. somewhat extending GLSL to allow as much stuff as HLSL allows us to do.

My current blog on programming, linux and stuff - http://gameprogrammerdiary.blogspot.com


Sponsor:

#2 Ashaman73   Crossbones+   -  Reputation: 7991

Like
2Likes
Like

Posted 18 July 2013 - 11:06 PM

Yes, I extend it (mostly includes, switches, global constants etc.) using a pre-processor (in my case mcpp). Still you need to remember, that GLSL itself contains some pre-processing powers to handle e.g. different shader model support.



#3 Promit   Moderators   -  Reputation: 7621

Like
1Likes
Like

Posted 18 July 2013 - 11:20 PM

I have a hack job include/prefix system that is just barely workable. Personally I find it absurd that GLSL supports linking but not file includes. If I had time, I'd probably retool with a full parser/effect type system. I've also toyed with simply jumping to hlsl2glsl as a potentially more sane alternative.


Edited by Promit, 18 July 2013 - 11:27 PM.


#4 rnlf   Members   -  Reputation: 1185

Like
5Likes
Like

Posted 19 July 2013 - 03:51 AM

How could GLSL even have file includes? GLSL doesn't even know about source files. You would either need a feedback mechanism telling the host application which include file is needed or the host application would have to upload each and every file that might possibly be required by the current source file. To get that right, you would have to parse your GLSL source files anyway to find out which includes you need. It's only a small step towards handling the includes yourself from there.

 

Linking is a completely different story though, the system knows all symbols generated by its source files and does not need any two-way communication with the host application.

 

So, I think the way it is done is the only feasible one.


my blog (German)


#5 johnchapman   Members   -  Reputation: 550

Like
1Likes
Like

Posted 19 July 2013 - 05:01 AM

Like Promit, I have a somewhat hacky preprocessor which manages #includes but not much else. For experimental and small scale projects you don't really need anything more. A serious renderer would probably require a more complete system for things like generating shader permutations. External #defines are a good mechanism for that sort of thing. 



#6 mhagain   Crossbones+   -  Reputation: 8278

Like
3Likes
Like

Posted 19 July 2013 - 05:17 AM

You don't even need file includes in GLSL; glShaderSource has a pretty good "include" mechanism built-in as is.

const char *globalShaderIncludes = "stuff"; // read this from a file or resource if you want
const char *thisShader = "more stuff"; // load this as normal for the current shader

const char *shaderSources[] = {globalShaderIncludes, thisShader};

glShaderSource (shaderObject, 2, shaderSources, NULL);

It appears that the gentleman thought C++ was extremely difficult and he was overjoyed that the machine was absorbing it; he understood that good C++ is difficult but the best C++ is well-nigh unintelligible.


#7 michalferko   Members   -  Reputation: 670

Like
1Likes
Like

Posted 19 July 2013 - 06:55 AM

I am currently working on simplified GLSL effects, kinda like http://gleffect.sourceforge.net/ but much simpler and focused on GLSL 330 and higher. I am writing it myself to plug it into my engine, I didn't want the external dependencies that GLeffect! includes. I also wanted something that works perfectly with my engine.



#8 Promit   Moderators   -  Reputation: 7621

Like
3Likes
Like

Posted 19 July 2013 - 10:20 AM

Tsk.
 

You don't even need file includes in GLSL; glShaderSource has a pretty good "include" mechanism built-in as is.
const char *globalShaderIncludes = "stuff"; // read this from a file or resource if you want
const char *thisShader = "more stuff"; // load this as normal for the current shader

const char *shaderSources[] = {globalShaderIncludes, thisShader};

glShaderSource (shaderObject, 2, shaderSources, NULL);

That is what I'm doing and described as a "hack job". It has a variety of practical issues. Here's a hint: where does #version go, particularly when using heterogeneous shader versions? Also, it requires you to remember to do things like add terminating newlines (like a C preprocessor from twenty years ago). It does work, barely.
 

How could GLSL even have file includes? GLSL doesn't even know about source files. You would either need a feedback mechanism telling the host application which include file is needed

D3DX has done this for many years and many titles have shipped like that. It's a very reasonable approach. As far as I know, there's no OpenGL method of any sort that accepts and uses a callback though. I think there's possibly some reason they've resisted them, because it's the only way to explain...

or the host application would have to upload each and every file that might possibly be required by the current source file. To get that right, you would have to parse your GLSL source files anyway to find out which includes you need.

And that is what they actually did. Seriously, how is such a significant API designed this poorly? It's beyond belief.


Edited by Promit, 19 July 2013 - 10:21 AM.


#9 Vilem Otte   Crossbones+   -  Reputation: 1561

Like
0Likes
Like

Posted 19 July 2013 - 02:58 PM

Thanks for the input... I'm glad to know that more people think about limitations of GLSL (as opposed to F.e. HLSL).

 

So far I'm deciding whether I need some more preprocessing than just includes and running specific code for specific shader models (both is already implemented).


My current blog on programming, linux and stuff - http://gameprogrammerdiary.blogspot.com


#10 swiftcoder   Senior Moderators   -  Reputation: 10367

Like
2Likes
Like

Posted 19 July 2013 - 03:02 PM

I'm gradually coming round to the idea that for anything more than simple demos, writing shaders by hand is for the birds.

 

A decent graph-based material designer, and a runtime component that generates full shaders from the material definition, is really very pleasant...


Tristam MacDonald - Software Engineer @Amazon - [swiftcoding]


#11 Kaptein   Prime Members   -  Reputation: 2180

Like
1Likes
Like

Posted 19 July 2013 - 03:20 PM

opengl can't read files, ever, because it's not built for a specific OS, architecture, etc.

it's got wide use in medical equipment, and honestly

reading line by line, trimming it (removing whitespace on each side) and testing for "#include " is hardly much work

 

really, the only gripe is with C++ SL itself which has a faulty getline() method that doesn't work the same way on linux and windows



#12 Promit   Moderators   -  Reputation: 7621

Like
1Likes
Like

Posted 19 July 2013 - 03:35 PM

Thanks for the input... I'm glad to know that more people think about limitations of GLSL (as opposed to F.e. HLSL).

 

So far I'm deciding whether I need some more preprocessing than just includes and running specific code for specific shader models (both is already implemented).

You have a problem: GLSL is kind of stupid. You can a) work around this problem as needed b) devise a new block of code to solve this problem. You're using a and it already works. If you choose b, you now have two problems. That's why most of us are using hackjob include mechanisms; writing a more elaborate parser is a pain in the ass and most of us don't want to do it despite the quirks that are there now. It requires more code that is difficult to write correctly and robustly.

 

At some point your shader will crash or somehow malfunction. Do you really want to be wondering whether the problem is the shader, the driver, or your intermediate compiler?


Edited by Promit, 19 July 2013 - 03:38 PM.


#13 Vilem Otte   Crossbones+   -  Reputation: 1561

Like
0Likes
Like

Posted 19 July 2013 - 04:25 PM

#Promit You exactly summed the things I've been thinking about. So far I came up with a bit creative solution for the game engine - as it heavily utilizes modularity and system internally compiles just GLSL shader code, I decided to "off-load" this work to plug-in, and different plug-ins will take care of F.e. just loading the files (and doing no preprocessing), or F.e. doing some pre-processing ... 

 

E.g. you have set of plugins, through one of these you load shader file, their output is GLSL shader code (which I can dump to some text file), from there the engine compiles this shader code and links it (possible errors are dumped to log files).


My current blog on programming, linux and stuff - http://gameprogrammerdiary.blogspot.com


#14 TheChubu   Crossbones+   -  Reputation: 4766

Like
0Likes
Like

Posted 19 July 2013 - 08:27 PM

I thought about making a play on words of "extend opengl" (keyword in java) but meh, no ideas.

 

I'm gradually coming round to the idea that for anything more than simple demos, writing shaders by hand is for the birds.

 

A decent graph-based material designer, and a runtime component that generates full shaders from the material definition, is really very pleasant...

 

I've read that there are some people against those, saying that if the visual designer is too high level you can't do interesting things with it, and that if its too low level its better to write the shader and be done with it.

 

Anyway I've seen a few shader editors where you can define new nodes made from other low-level'ish nodes and have all the functionality in one place, a library of shader nodes. It does seems pretty handy.


"I AM ZE EMPRAH OPENGL 3.3 THE CORE, I DEMAND FROM THEE ZE SHADERZ AND MATRIXEZ"

 

My journals: dustArtemis ECS framework and Making a Terrain Generator


#15 Hodgman   Moderators   -  Reputation: 31843

Like
1Likes
Like

Posted 19 July 2013 - 10:11 PM

I'm redoing my GL support in the near future, but my plan is to parse my shader code offline, converting it into an intermediate representation, and then using that to generate streamlined GLSL code for the engine to use (with all includes/preprocessor statements resolved, basic optimisations applied, whitespace/variable names trimmed, etc).

 

Screw using raw GLSL.



#16 Eric Lengyel   Crossbones+   -  Reputation: 2492

Like
0Likes
Like

Posted 20 July 2013 - 04:15 AM

As far as I know, there's no OpenGL method of any sort that accepts and uses a callback though.

 

See GL_ARB_debug_output.



#17 Yours3!f   Members   -  Reputation: 1396

Like
0Likes
Like

Posted 20 July 2013 - 07:57 AM

I'm using pure GLSL with the built-in preprocessor, plus I process #includes manually in cpp, but that's it. With this I've implemented an uber-shader system, I personally didn't feel the need for much more.



#18 Promit   Moderators   -  Reputation: 7621

Like
1Likes
Like

Posted 22 July 2013 - 12:07 AM

 

As far as I know, there's no OpenGL method of any sort that accepts and uses a callback though.

 

See GL_ARB_debug_output.

 

Ah of course. And I added debug output support recently, too. Silly of me to forget about it. But it just makes it that much harder to explain the bizarre include extension.



#19 rnlf   Members   -  Reputation: 1185

Like
1Likes
Like

Posted 22 July 2013 - 02:32 AM

And that is what they actually did. Seriously, how is such a significant API designed this poorly? It's beyond belief.

Oh my... I was totally unaware of this extension. Ouch! My Brain!


my blog (German)





Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS