HLSL questions

Started by
13 comments, last by hupsilardee 13 years, 6 months ago
Hi all,

Firstly, I'm new around here, this is my "hello" thread:

http://www.gamedev.net/community/forums/topic.asp?topic_id=583769

So I hope I'm in the right place for this kind of stuff but I've spent the last week or so diving into the world of shaders, which, despite being and relatively experienced software guy, are making me feel very newbish! In an effort to better understand things I've compiled a short list of questions I have regarding HLSL (I'll move on to GLSL next). I hope you don't mind answering them if you can:

1: Can you run multiple shaders/techniques on one vertex buffer, in that a DrawPrimitives() call in a for loop which is looping through all the shaders/techniques/passes, or does this have to be done as seperate passes in the same technique and therefore I need one global shader? I spose you could say, can you 'stack' different shaders on a single mesh?

2: What is the difference, and pros/cons, between D3DXCreateEffectFromFile and D3DXCompileShader?

3: Since I am writing a game engine that I intend to use in a production environment eventually, is it worth getting used to compiling the shaders externally now? Or shall I just program for ASCII files now and then abstract ASCII/binary state later?

4: How similar are GLSL and HLSL, not in syntax, but in concept? What architectural differences exist between the two 'worlds'?

I think they're my main questions right now, I'm sure they'll be more.

Any help/steering would be appreciated...

Thanks

Tim
Advertisement
1. You can only have one vertex shader and one pixel shader active at time for a draw call. No exceptions. If you need combine two bits of shader code, you essentially have 3 options:

A. Combine the shader code into a single shader, either manually or with an automated tool or framework

B. Combine the results of the two shaders using fixed-function blending states by rendering multiple passes of the same geometry (for instance, using additive blending to sum the contributions from two light sources)

C. Render one pass with one shader to a render target texture, then sample that texture in the second pass using the second shader which combines the results.

A is most ideal in terms of performance, while B is probably easiest in terms of implementing it. C isn't really that great for either.

2. Well one creates an effect, and one just creates a raw shader. The effects system is a framework built on top of the raw D3D9 shader API. It encapsulates multiple shaders and details about them, in order to make it easier to use them and build shaders with more complex functionality. Generally if you don't use effects, then you implement your own framework for doing what it does.

3. It probably won't matter unless you have many shaders, in which case it may take a while to compile the shaders every time you run the game.

4. I'm not really a GLSL expert, but in my experience the fundamental concepts are the same. One thing to watch out for is that in many cases GLSL can't be compiled offline, and is JIT compiled by the driver when you load it. This is really bad on some mobile platforms where the compiler sucks at optimizing, which can result in some really bad shader performance.
Quote:Original post by MJP
4. I'm not really a GLSL expert, but in my experience the fundamental concepts are the same. One thing to watch out for is that in many cases GLSL can't be compiled offline, and is JIT compiled by the driver when you load it. This is really bad on some mobile platforms where the compiler sucks at optimizing, which can result in some really bad shader performance.


On mobile platforms, you have OpenGL ES 2.0 which has on offline compiler.
Also, JIT is used on desktops where OpenGL 2.1 or 3.3 will be available and JIT gives better results since compilers are highly tuned up.
Sig: http://glhlib.sourceforge.net
an open source GLU replacement library. Much more modern than GLU.
float matrix[16], inverse_matrix[16];
glhLoadIdentityf2(matrix);
glhTranslatef2(matrix, 0.0, 0.0, 5.0);
glhRotateAboutXf2(matrix, angleInRadians);
glhScalef2(matrix, 1.0, 1.0, -1.0);
glhQuickInvertMatrixf2(matrix, inverse_matrix);
glUniformMatrix4fv(uniformLocation1, 1, FALSE, matrix);
glUniformMatrix4fv(uniformLocation2, 1, FALSE, inverse_matrix);
Hi MJP,

Thanks for the info, its a fascinating subject.

So I have some follow ups then (sorry!)...

1 and 2. Writing your own effect framework... Is it worth it? I'm thinking I could implement my own framework and abstract it so parts of the engine are agnostic to GLSL/HLSL (obviously not the renderer). As well as provide a solution to combining different parts of shaders...

3. I see. I was also thinking about keeping my shader source private...

4. Is GLSL really used for OpenGL, or should I consider Cg (as its supposed to be so similar to HLSL)?

Cheers!

Tim

Quote:Original post by tgjones
1 and 2. Writing your own effect framework... Is it worth it? I'm thinking I could implement my own framework and abstract it so parts of the engine are agnostic to GLSL/HLSL (obviously not the renderer). As well as provide a solution to combining different parts of shaders...


Well, if you have to ask it probably isn't worth it [smile] The effects framework is rather flexible, complete, efficient, easy to work with etc, so unless you have any very specific and clear needs for your shader handling framework, you should probably be using the default one. As for making parts of the engine agnostic (crossplatform?) from my limited exprience I think it's more typical to create completely seperate renderers for multiple platforms.

Quote:3. I see. I was also thinking about keeping my shader source private...


I'm not sure you can 'pre-compile' all your shaders to try and keep them safe, depending on your target platform of course. In any case, I think this is more pertinent to the general concern of keeping game assets private so you could try some of those techniques (obfuscation, encoding etc). On the other hand, most complex shaders worth protecting require specific inputs from the application code, so it might not be useful to try and steal them.

In various high-profile commercial games some snooping turned up the shaders in .txt files in their program directory, so if big studios aren't bothered by this, I wouldn't worry too much about it either.

Quote:4. Is GLSL really used for OpenGL, or should I consider Cg (as its supposed to be so similar to HLSL)?


No clue, sorry [smile]
Rim van Wersch [ MDXInfo ] [ XNAInfo ] [ YouTube ] - Do yourself a favor and bookmark this excellent free online D3D/shader book!
A large conceptual difference between GLSL and HLSL, is one of the "pillars of hatred" I have for OpenGL generally.

In a GLSL shader, you can't specify contstants manually (ie there are no register semantics).

This means you can't write your shader expecting to get its ambient colour from constant 3 for example, whereas in HLSL you can. To handle constants, you declare a global variable in your shader and then after compiling it you need to ask where the ambient should go and remember it. (You can also do this in HLSL but you don't need to).

Sounds subtle, but it cause carnage in my engine that makes a lot of shaders via cpu code (strcat'ing lines of code together to build functions to be compiled). It also means you can't reserved some killer importance constants and just leave em alone.


------------------------------Great Little War Game
Quote:Original post by V-man
On mobile platforms, you have OpenGL ES 2.0 which has on offline compiler.


OES_shader_binary is an optional extension.

Quote:Original post by V-man
Also, JIT is used on desktops where OpenGL 2.1 or 3.3 will be available and JIT gives better results since compilers are highly tuned up.


Better results than what? A standarized offline compiler? I know there's no way I would ever prefer having to deal with the quirks and bugs of multiple shader compilers rather than just one.

Hi remegius, Rubicon,

Thanks for the info, just beginning to get my head around this. So I take it from the discussion so far that writing my own framework seems somewhat redundant, for HLSL at least, as the Microsoft one seems to do a perfectly good job.

Additionally, as only one vs and ps can be active during a draw I'm guessing I can use techniques or passes to combine effects? I'm thinking all vertices need WVP transforming, but not need to flap like a flag for example, so I could have options associated with a mesh telling the renderer 'behaviour' and then combine the relevant passes at the renderer level. Does this sound like a reasonable approach?

Additionally, it seems trying too hard to keep shader source private is a bit pointless, and that I should really get my head around HLSL before I start trying to learn GLSL/Cg!!

Thanks again.

Tim
Quote:Original post by tgjones
Additionally, as only one vs and ps can be active during a draw I'm guessing I can use techniques or passes to combine effects? I'm thinking all vertices need WVP transforming, but not need to flap like a flag for example, so I could have options associated with a mesh telling the renderer 'behaviour' and then combine the relevant passes at the renderer level. Does this sound like a reasonable approach?


It's not unreasonable, but it's probably a can of worms and inefficient to boot. Shader passes are, in my opinion, a bit outdated and not really that useful for much anymore. The main usage was, as MJP pointed out, rendering the scene multiple times and blending the results. Since you cannot read back the results from previous passes (not without swithing render targets, which defeats the purpose of having multiple passes) it's only really suited for rendering the scene with multiple lights and additively blending these.

Even so, this multi-passing was only really needed with the limited instruction count allowed in pixel shaders on older hardware. These days, you can calculate numerous lights in one pass or use alternative techniques like deferred rendering, if you really want *a lot* of lights.

As a good rule of thumb, it's typically the best option to create a specific shader for each specific rendering technique. This may lead to some code redundancy, but in general it will give you better performance and the system as a whole will be easier to work with. Finally it might help to consider that you're probably only going to use one or two 'main' shaders for your game, since you'll want to have the lighting to look consistent across all objects anyway.

Rim van Wersch [ MDXInfo ] [ XNAInfo ] [ YouTube ] - Do yourself a favor and bookmark this excellent free online D3D/shader book!
Interesting.

So I'd sort of come to that conclusion actually myself too with my experimentation, duplicate code that is. A single FX file for each vertex format (declaration) seems a more reasonable (and easier to maintain) approach, #including external shader functions to minimize duplication...? Or is #including in shaders frowned upon?

Its quite a learning curve, although I think (hope) I'm getting over the hump now!

Tim

This topic is closed to new replies.

Advertisement