Hi Guys,

I am having a problem witha vignette shader in Game Maker: Studio. The shader works perfectly in Codea on the iPad, but I don't get any fall-off in GM:S. All I get is a slightly darker screen.

I made the background whit to ease troubleshooting and the background turns to a perfectly even grey (checked in photoshop). So, I know there is not fall-off. It is like smoothstep() isn't happening on the PC.

I have even added a sepia effect to verify that the shader is actually being applied. So with that addition I get an even sepia screen.

// Fragment shader
 
//uniform sampler2D u_texture;
varying vec4 v_vColor;          //"in" attributes from our vertex shader
varying vec2 v_vTexcoord;
const float RADIUS=0.5;
const float SOFTNESS=0.45;
uniform vec2 resolution;
void main()
{
    vec4 texColor = texture2D(gm_BaseTexture,v_vTexcoord);
    // *** VIGNETTE ***
    vec2 pos = (gl_FragCoord.xy/resolution.xy)-vec2(0.25);
    float len = length(pos);
    float vignette = smoothstep(RADIUS, RADIUS-SOFTNESS, len);
    texColor.rgb = mix(texColor.rgb, texColor.rgb * vignette, 0.5);
   
    // *** SEPIA ***
    vec3 lum = vec3(0.299, 0.587, 0.114);
    float gray = dot(texColor.xyz, vec3(0.299, 0.587, 0.114));
    texColor = vec4(gray, gray, gray, texColor.w);
    texColor.rgb *= vec3(1.15,1.1,0.8);
   
    gl_FragColor = texColor * v_vColor;
}

Any help on this would be awesome. I am completely out of ideas.

vec2 pos = (gl_FragCoord.xy/resolution.xy)-vec2(0.25);

0.5 instead of 0.25 ?

Just tried 0.5 as well, but absolutely not difference, unfortunately.

The whole screen is just 147,140,102 (RGB) - sepia. Instead of sepia falling off to dark sepia (which my Codea project correctly does on the iPad).

This is what my shader was derived from - https://github.com/mattdesl/lwjgl-basics/wiki/ShaderLesson3

It may be that you have transformed the coordinates.

When displayed the pixel coordinates are in the range -1 to 1, you are assuming 0-resolution

I would try changing this

vec2 position = (gl_FragCoord.xy / resolution.xy) - vec2(0.5);

to this

   vec2 position = (gl_FragCoord.xy  * 0.5f);

It is worth a try at least

The GLSL specifications say:

genType smoothstep (genType edge0, genType edge1, genType x)
...
Results are undefined if edge0 >= edge1.

In your case edge0 = 0.5 and edge1 = 0.5 - 0.45 = 0.05, so you get a undefined result! Even it it works on one platform, the result may be totally different on another platform, and so happens here. Try to exchange the arguments RADIUS and RADIUS-SOFTNESS when invoking smoothstep.

When displayed the pixel coordinates are in the range -1 to 1, you are assuming 0-resolution

The x and y values of gl_FragCoord are window relative co-ordnates. The bottom left pixel has co-ordinates (0.5,0.5) and the upper right has (w-0.5,h-0.5) where (w,h) denotes the resolution in pixels. The range [-1,+1] instead is in normalized device co-ordinates (NDC).

Just tried 0.5 as well, but absolutely not difference, unfortunately.

That is okay w.r.t. the fact that the problem is probably based in the use of smoothstep, but Ashaman73 is right: A symmetric effect would require the subtraction of (0.5,0.5) there.

Thanks Guys, lots of replies here so I'll go through the suggestions one by one

Stainless - I just tried vec2 position = (gl_FragCoord.xy * 0.5f); No joy, I am afraid

haegarr - I just tried float vignette=smoothstep(RADIUS-SOFTNESS,RADIUS,len); so then egde0<edge1, still no good.

Ashaman73 - Tried float vignette = smoothstep(-RADIUS, -(RADIUS-SOFTNESS), -len); But still no luck

I am told that GameMaker user GLES, does this make a difference? All compiles without a problem though.

There is a slight difference in the vertex shaders though.

Although I don't know enough about shaders to know what is going on.

From the working shader on Codea

uniform mat4 modelViewProjection;
attribute vec4 position;
attribute vec4 color;
attribute vec2 texCoord;
 
varying highp vec2 v_vTexCoord;
varying lowp vec4 v_vColor;
 
void main()
{
    //Pass the mesh color to the fragment shader
    v_vColor = color;
    v_vTexCoord = texCoord;
    //Multiply the vertex position by our combined transform
    gl_Position = modelViewProjection * position;
}

and the non-working Game Maker shader

attribute vec4 in_Position;
attribute vec4 in_Colour;
attribute vec2 in_TextureCoord;
varying vec2 v_vTexCoord;
varying vec4 v_vColor;
void main()
{
    vec4 object_space_pos=vec4(in_Position.x,in_Position.y,in_Position.z,1.0);
    //Pass the mesh color to the fragment shader
    v_vColor = in_Colour;
    v_vTexCoord = in_TextureCoord;
 
    //Multiply the vertex position by our combined transform
    gl_Position =gm_Matrices[MATRIX_WORLD_VIEW_PROJECTION]* object_space_pos;
}

GM has a few of its own variables like in_Position etc. I am a bit dubios of the last line though as it was copied from another shader.

Any direction would be awesome!

...

But the above code breaks the shader on the iPad (Codea) as well. So, obviously something not right with it.
...

"Breaks the shader" is somewhat unspecific. However, have you checked that the uniform resolution is up to date?

What do you see if you debug output the shape of the vignette like so:

// Fragment shader

//uniform sampler2D u_texture;
varying vec4 v_vColor;          //"in" attributes from our vertex shader
varying vec2 v_vTexcoord;
const float RADIUS=0.3; // <<< changed here for debug output
const float SOFTNESS=0.1; // <<< changed here for debug output
uniform vec2 resolution;
void main()
{
    vec4 texColor = texture2D(gm_BaseTexture,v_vTexcoord);
    // *** VIGNETTE ***
    vec2 pos = (gl_FragCoord.xy/resolution.xy)-vec2(0.5); // <<< changed here for symmetry
    float len = length(pos);
    float vignette = 1.0 - smoothstep(RADIUS-SOFTNESS, RADIUS, len); // <<< changed here for correctness with spec.
//    texColor.rgb = mix(texColor.rgb, texColor.rgb * vignette, 0.5);
    texColor.rgb = vec3(vignette, vignette, vignette); // <<< debug output
    gl_FragColor = texColor;
}

If I made no mistake, the shader should show you an inner white disk, a relatively sharp white-to-black transition, and a black surrounding.

Thanks again haegarr for your help.

With your code I get a completely black screen. If I swap the smoothstep() function back to the old one I had, we the get a completely white screen.

A bit frustrating - LOL.

Even at the point where I am now downloading the latest driver for my video card.

[edit]

Nope - drivers didn't give a miracle fix, unfortunately.