Jump to content

  • Log In with Google      Sign In   
  • Create Account

Tocs' Blog



Shader Templates - Simple But Useful

Posted by , 25 February 2014 - - - - - - · 2,543 views

For the longest time I've struggled with how I wanted to handle materials in my graphics framework. When searching around for existing solutions I found basically two things.

A: Shaders with strict inputs:
A single shader that had specific inputs that were textures, floats, etc etc.

B: Node based shaders:
Crazy flexible graphical editors for materials constructed from graphs of various building block elements.

'A' wasn't flexible enough for me, and 'B' seemed like something way to big and time consuming to properly create myself.

So I decided on something somewhat inbetween...

I built a GLSL preprocessor templated shader generator. It takes in GLSL code with extra markup. Instead of specifying inputs as strictly typed GLSL uniform variables. I tell the generator the input names and the desired final data type (float, vec2, vec3, vec4). Then a set of inputs is given to the generator and it creates a shader that samples the textures, looks up values, and makes sure there's a variable with the requested name that contains the appropriate value.

It's easier to show what I mean... Here's my Blinn-Phong material shader template.

Templates
#version 140
#include "shaders/brdf/blinnphong.hglsl"
#include "shaders/brdf/NormalMapping.hglsl"

<%input vec4 DiffuseColor%>
<%input vec4 SpecularColor%>
<%input float SpecularPower%>
<%input float SpecularIntensity%>
<%input vec3 NormalMap optional%>
<%input vec4 Emissive optional%>

in vec2 TextureCoordinate;
in vec3 GeometryNormal;
in vec3 GeometryTangent;

<%definitions%>

void ShadePrep ()
{
	<%init%>
}

vec4 ConstantShading(vec4 AmbientLight)
{
	vec4 result = AmbientLight * DiffuseColor;
	<%ifis not Emissive: None%>
		result += Emissive;
	<%endif%>
	return result;
}

vec4 Shade (vec3 LightDir, vec3 ViewDir, vec3 LightColor, float Attenuation)
{
	vec3 normal = normalize(GeometryNormal);
	<%ifis NormalMap: Texture %>
		normal = NormalMapping(normal,normalize(GeometryTangent),NormalMap);
	<%endif%>
	return BlinnPhong (LightDir, ViewDir, LightColor, Attenuation, normal, DiffuseColor, SpecularColor, SpecularPower, SpecularIntensity);
}

<%input ... %> : Specifies input names and desired types to the generator. Some inputs can be optional, the generator won't raise an error if these aren't supplied.

<%definitions%> : This tells the generator where to define extra variables it may need. While technically redundant because the extra variables could be placed where the <%input%> tag is, I wrote it with the <%definitions%> tag and didn't bother to remove it.

<%init%> : This tells the generator where to put the code that's needed to get the proper final value, such as sampling a texture.

<%ifis ...%> : This lets the generator do different things based on the types of input you supply. In the above shader I add an extra line of code to transform the normal if a NormalMap texture is supplied. I also apply an Emissive term if the shader is supplied one.

Input Sets

Sets can be constructed in code, or loaded from file. The file contains text like this:
DiffuseColor tex("sword/diffuse.png")
SpecularPower 6.0
SpecularIntensity tex("sword/specular.png").r
SpecularColor color(255,255,255)
NormalMap tex("sword/normal.png")
Emissive tex("sword/glow.png")
It's pretty easy to tell what this does. The generator takes the input set, and generates a shader which can utilize it. You might notice I specify a swizzle for SpecularIntensity. You can pick different channels out of a texture for a certain input, if you specify the same texture twice it's smart enough to only sample it once and swizzle the sample in the shader.

When I plug those inputs in, this is what it generates (I fixed the whitespace up though...)
#version 140
#include "shaders/brdf/blinnphong.hglsl"
#include "shaders/brdf/NormalMapping.hglsl"

uniform sampler2D Texture_0;
uniform vec4 SpecularColor;
uniform float SpecularPower;
uniform sampler2D Texture_4;
uniform sampler2D Texture_2;
uniform sampler2D Texture_1;

in vec2 TextureCoordinate;
in vec3 GeometryNormal;
in vec3 GeometryTangent;

vec4 DiffuseColor;
vec4 Emissive;
vec3 NormalMap;
float SpecularIntensity;


void ShadePrep ()
{
	vec4 Sample0 = texture2D(Texture_0, TextureCoordinate);
	DiffuseColor = Sample0.xyzw;
	vec4 Sample1 = texture2D(Texture_1, TextureCoordinate);
	Emissive = Sample1.xyzw;
	vec4 Sample2 = texture2D(Texture_2, TextureCoordinate);
	NormalMap = Sample2.xyz;
	vec4 Sample4 = texture2D(Texture_4, TextureCoordinate);
	SpecularIntensity = Sample4.x;
}

vec4 ConstantShading(vec4 AmbientLight)
{
	vec4 result = AmbientLight * DiffuseColor;
	result += Emissive;
	return result;
}

vec4 Shade (vec3 LightDir, vec3 ViewDir, vec3 LightColor, float Attenuation)
{
	vec3 normal = normalize(GeometryNormal);
	normal = NormalMapping(normal,normalize(GeometryTangent),NormalMap);
	return BlinnPhong (LightDir, ViewDir, LightColor, Attenuation, normal, DiffuseColor, SpecularColor, SpecularPower, SpecularIntensity);
}

These are just simple inputs, but you can do more interesting things with it as well. For particle systems I can connect the material inputs to values passed in from the particle data. For instance getting a index for an array of different particle textures. If you wanted an animated texture you can have an input type represent all the frames and switch between them.

Additionally the input sets generate a hash code (probably)unique to the generated shader, so if you have similar input sets that use the same generated shader, it's only created once. I was also hoping to cache the compiled shader binaries. However, even in the latest OpenGL the spec says that glShaderBinary must be supported but there doesn't have to exist a format to save them in. So it's pretty useless and disappointing, it's possible to cache the linked programs though.

It was fairly easy to implement, allows decent flexibility, and cuts down on the time I have to spend writing little differences in shaders. There's obviously a lot of improvements I could make (as with anything) but I'm getting a lot of mileage out of it's current state.

What do you think? I'd love some feedback.

Also, Obligatory Progress Screenshot:
Posted Image


Single Pass Order Independent Transparency

Posted by , 06 January 2014 - - - - - - · 1,671 views
transparency, OIT, OpenGL
Over the past couple of days I wanted to try out order independent transparency. AMD showed it off in their "Mecha Demo". And with a little help from here I was able to get it functioning in my own graphics framework.

In Cyril's blog he uses some bindless buffer extensions from NVidia. In my implementation I use Image Load Store. (Cyril also had an implementation for this). I also swapped out an Atomic Counter for one of Cyril's bindless buffers. Allowing me to be independent of NVidia extensions (despite how cool bindless buffers are).

While I was basically re-implementing what Cyril had in his blog I learned a lot and it was a lot of fun. I don't know how efficient it is, my 770 GTX sometimes stutters when I fill up the screen with too many layers. But I am running a debug build with lots of checks and outputs so it's hard to tell what the source of lag is.

I paired the transparency stuff with tiled forward shading so I can evaluate many lights in a scene with transparent geometry. I haven't done much bench marking or optimization work. However, I was happy it functioned and thought it was worth a share.

Posted Image

Posted Image

However glass just doesn't look quite right without some refraction. And the table's texture is horrendous that close to the camera.


Two things I wish C++ had.

Posted by , 28 August 2013 - - - - - - · 982 views
C++
I thought I'd share some thoughts on two features I wished C++ had, tell me what you think.


Identifier Template Parameters

I'd really like to be able to take an identifier token as a template parameter, similar to a macro but it could only accept a single identifier.

You could use it somewhat like this.
template <class T, identifier variablename>
void AddOne (T &value)
{
    ++value.variablename;
}

or
template <class T1, identifier first, class T2, identifier second>
class NamedPair
{
public:
    T1 first;
    T2 second;

    ...pair like functions...
};

NamedPair<int,A,int,B> pair;

pair.A = 10;
pair.B = 20;
You may be wondering why I would want such a feature. While there are more uses the one that prompted me to wish for this was a sort of "Compile Time Reflection" set of helpers. Similar to those in type_traits, but to check if a type has certain functions or variables.

You wouldn't need a macro to create SFINAE member checking classes anymore.
template <class T, identifier member>
class has_member
{ ... };

class A
{
public:
    int foo;
};

class B
{
};

has_member<A,foo>::value; //true
has_member<B,foo>::value; //false
Which means you could use std::enable_if to turn on and off functions based of the contents of a template parameter. Which would be really nifty.

You would probably need the equivalent of typedef for identifiers (alias?) if you were to use this in a more complex template situation something like...
identdef foo bar;

class A
{
public:
     int foo;
};

A.bar = 10;
Which could introduce a whole host of issues... maybe I haven't thought it all the way through.

Perfect Forwarding Without Templates

C++11 added R-Value references and move semantics which ease a lot of the pain from C++03. However to take advantage of this you need to handle forwarding R-Value and L-Value references properly. Which either means using a template or dealing with the permutation explosion of arguments.
void A (const int &value)
{ ... }

void A (int &&value)
{ ... }

//or

template <class T>
void A (T &&value)
{ ... }
Writing out the cases is ok if you have just one argument, but if you have more than one its time to use a template. But with a template you lose the ability to assert which type you would like to accept.
template <class T1, class T2>
void B (T1 &&one, T2 &&two)
{ ... }
If I wanted to make sure B only excepted type "Foo" I could only use static asserts to inform the programmer. Instead I'd love to be able to write something like
void B (Foo &@one, Foo &@two)
{ ... }
And have the compiler generate the permutations, just as the compiler does for template type && arguments. (Except enforcing type Foo). I chose an @ sign at random, its kind of ugly substitute what you wish.


Lost in Asset Management

Posted by , 22 July 2013 - - - - - - · 996 views
C++, assets for sale
So I've been contemplating a good way to manage assets used by my game and so far I've been losing the mental battle.

My current idea is a custom smart pointer much like shared_ptr, in addition to reference counting it keeps track of which files have been loaded and simply returns a reference if its already loaded.

This makes loading files worry free as it looks like this.
Asset<Texture2D> tex = Asset<Texture2D>::Load("UglyTexture.png");
And if you load it twice.
Asset<Texture2D> tex1 = Asset<Texture2D>::Load("UglyTexture.png");
Asset<Texture2D> tex2 = Asset<Texture2D>::Load("UglyTexture.png");
//UglyTexture.png is only loaded once.
It won't waste CPU or GPU memory.

And preloading commonly used things is easy too, it just adds the texture to the table with an extra reference so it never gets unloaded.
Asset<Texture2D>::PreLoad("UglyTexture.png");
Things get tricky though if I generate some sort of texture at runtime, it becomes a question of who should own the Texture object.

If an object is dynamically allocated, it's rather simple to say Asset should determine when it's deleted with something like.
Texture2D *proceduraltex= new Texture2D (100,100,...);
//Do something to proceduraltex
Asset<Texture2D> tex = Asset<Texture2D>::Manage(proceduraltex,"ProceduralTex");


//Somewhere else
Asset<Texture2D> texagain = Asset<Texture2D>::Load("ProceduralTex");
But than I began to wonder about the case of the Texture2D object belonging to something else, and how i would use it.
void SomeAssetFunction (const Asset<Texture2D> &asset) {}

class Foo
{
    Texture2D footex;
    Foo ()
    {
        SomeAssetFunction(Asset<Texture2D>::Wrap (footex));
    }
};
But since the "Wrapped" asset doesn't own the texture it could go out of scope and suddenly the Asset<>'s promise of the texture existing is meaningless.

So my thought was to only use Asset<>'s as member variables, it could easily cause a problem of loading a texture every frame if the Asset<> is used as a local variable.
void SomeFunctionCalledEveryFrame ()
{
    Asset<Texture2D> tex = Asset<Texture2D>::Load("NonPreloadedTex.png");
    //use tex
}//NonPredloadedTex.png gets destroyed.
And that any function that needed a Texture2D simply took a Texture2D reference.
void SomeFunctionThatTakesATexture (const Texture2D &);

Asset<Texture2D> tex = Asset<Texture2D>::Load("UglyTexture.png");
SomeFunctionThatTakesATexture(tex.Get ());
Overall I'm just mildly happy with the system. On one hand its nice to have the separation between the resource object (Texture2D, Mesh, Shader, etc) and the asset handling it seems to reduce cruft and redundancy in each file. On the other hand the disconnect between generated assets and file loaded assets is a tad annoying.

Its also expandable I plan to add live reload of assets behind the scenes. Since Asset<> controls the storage of each asset, its possible to just flop out the asset behind the scenes and anything using the Asset<> will instantly have the new resource to work with.

Downsides include:
  • Thread Unsafety: I might be able to work this out if I was super clever but as it stands the storage container for keying filenames to assets is thread unsafe. Also live reload would be tougher to add with thread safety.
  • Global state: Currently I'm not bothered by the fact it assumes there's only one OpenGL context and will put all resources there. But I eventually want to be able to use my graphics code in an editor where I might have multiple contexts.
  • No special parameters: Sometimes its nice to specify additional info for how a file should be loaded such as filtering for textures. Perhaps some variardic templates and a little SFINAE wizardry could produce usable results. (static_assert when type T of Asset<T> doesn't supply the proper functions. Otherwise cryptic compile errors appear)
This was my first ever journal post, and I just kinda vomited thoughts all over the page. If you have suggestions on how to handle assets leave me a comment.





August 2016 »

S M T W T F S
 123456
78910111213
14151617181920
21222324252627
2829 30 31   

Recent Comments



PARTNERS