Problem with vignette shader on PC

Started by
16 comments, last by DividedByZero 10 years ago

Maybe try using gl_TexCoord[0].xy instead of gl_FragCoord.xy. I remember having a similar issue where gl_FragCoord would return the correct value in GLES, but in GL it would always be 0.

"I would try to find halo source code by bungie best fps engine ever created, u see why call of duty loses speed due to its detail." -- GettingNifty
Advertisement

Maybe try using gl_TexCoord[0].xy instead of gl_FragCoord.xy. I remember having a similar issue where gl_FragCoord would return the correct value in GLES, but in GL it would always be 0.

Thanks for the suggestion.

I just tried that but it says that it is an undeclared identifier sad.png

Just saw this in the GLES manual. Does this mean it was removed from GLSL ES?

10.17 Unsized Array Declarations

Desktop GLSL allows arrays to be declared without a size and these can then be accessed with constant

integral expressions. The size never needs to be declared. This was to support gl_Texcoord e.g.

varying vec4 gl_TexCoord[];

...

gl_FragColor = texture2D (tex, gl_TexCoord[0].xy);

This allows gl_TexCoord to be used without having to declare the number of texture units.

gl_TexCoord is part of the fixed functionality so unsized arrays should be removed for GLSL ES

RESOLUTION: Remove unsized array declarations.

Getting pretty close now. Had a suggestion on another forum and this is pretty close to working.



precision highp float;   
uniform vec2 resolution;
varying vec4 v_vColor;
varying vec2 v_vTexCoord;
   
const float RADIUS=0.75;        // 0.75 is the good setting
const float SOFTNESS=0.45;
 
void main()
{
    vec4 texColor=texture2D(gm_BaseTexture,v_vTexCoord);
   
    vec2 center=vec2(0.5,0.5);
    vec2 aspect_center=vec2(0.0,0.0);
    aspect_center.x=(v_vTexCoord.x-center.x)*(resolution.x/resolution.y);
    aspect_center.y=v_vTexCoord.y-center.y;
    float len=length(aspect_center)*2.0;
       
    float vignette=smoothstep(RADIUS,RADIUS-SOFTNESS,len);  //use smoothstep to create a smooth vignette
    texColor.rgb=mix(texColor.rgb,texColor.rgb*vignette,0.50); //apply the vignette with 50% opacity
   
    gl_FragColor=texColor*v_vColor;
}

The only thing now is that it is positioned a bit too far across and down the screen. And I dont have the nice oval shape I had in Codea anymore. The Codea shader was centred and the vignette edge had an even amount of pixels from top, bottom, left, & right (oval shaped).

Codea original example (ignore the harsh black/white settings)

codea01.png

Current image from withing GameMaker (using above code). More the position and shape that I am looking to correct.

shader00.png

Thanks again for everyone's awesome help!

[edit]
I did some more testing. I think (using this method) it is positioning the vignette at the center of the texture page. As, the centre of the vignette is positioned at 1024, 512 (texture page would likely be 2048,1024 in this case).

[edit2]
Taking screen resolution 1600 x 900 and page size 2048 x 1024 into account, I have worked out that the centre should be
vec2 center=vec2(0.390625,0.439453125);


Had a suggestion on another forum and this is pretty close to working.

So, what is wrong with using gl_FragCoord in the end? I've used it in a test implementation on a MacBook and it worked well. You also mentioned it is working on an iPad. And from a logical point of view it should work anyway...


The only thing now is that it is positioned a bit too far across and down the screen.

The shader script shown in the post above expects a full-screen quad rendered with texture co-ordinates [0,1] from left to right and [0,1] from bottom to top. If you supply the drawing with such a quad, then there should be no need to adapt the center variable.


And I dont have the nice oval shape I had in Codea anymore.

This is due to the aspect correction factor in the shader code. If you want the oval back, then remove the stroked part in the following line:

aspect_center.x = (v_vTexCoord.x-center.x) * (resolution.x/resolution.y) ;

So, what is wrong with using gl_FragCoord in the end? I've used it in a test implementation on a MacBook and it worked well. You also mentioned it is working on an iPad. And from a logical point of view it should work anyway...


No idea why that doesn't work. Maybe it is a Game Maker: Studio bug or something. As GML is an 'interpreted' language and is based on DirectX9 when running under windows and GLSL ES is OpenGL based. So, maybe something is being lost in the translation?
Turns out that 'straight from the horses mouth', GLSL ES is not fully implemented in Game Maker: Studio.

This explains why things are inconsistent. Turns out my shader was right but GM, well, isn't.

gl_FragCoord, while is compiles ok in GM has been removed from GM's interpretation on GLSL ES in favour of their own 'broken' implementation. So, using their own implementation is returning coordinates for the entire texture page not just the texture.

Scary stuff...

I hate chopping and changing between engines, but I am getting to the point to where I figure I am safer with my own C++ framework.

I just want to get this project into the next IGF but, geez, even with another 6-8 months until submissions, the way things are going, I'll run out of time...

Good job on finding the core of the issue. I hope all goes well!

"I would try to find halo source code by bungie best fps engine ever created, u see why call of duty loses speed due to its detail." -- GettingNifty
Thanks for that.

Now trying to convince the GM devs that their shader support is broken.

Ah, us devs love challenges don't we? LOL

And thanks to everyone who helped out in this topic. It must have been pretty frustrating for you guys as well I'd imagine smile.png

This topic is closed to new replies.

Advertisement