Color key with OpenGL

Started by
9 comments, last by benang00 16 years, 8 months ago
Hi, I'm trying to rewrite my 2D rendering engine from pure SDL to using textured quads with SDL and OpenGL. I've tried to fiddle with the code but stumbled on how to color key a sprite with a textured quad. FYI, I'm using BMP images (thus no alpha channel). And every sprites that need colorkeying can have its own color key / transparent pixel (eg. black, white, etc). So the question is pretty obvious, how do I implement color key to a textured quad? Thanks in advance.
------------------ERROR: Brain not found. Please insert a new brain!“Do nothing which is of no use.” - Miyamoto Musashi.
Advertisement
The most direct way would probably be to apply the color key to the image manually, turning it from an RGBA image into an RGBA image, before handing it over to OpenGL.

You can also do it during rendering if using fragment shaders.
Well, I've tried that but it didn't work. FYI, I'm using SDL to load the image and then I generate the texture from the SDL surface that was created. Here's my texture loading function:

	if ( (surface = SDL_LoadBMP("test.bmp")) ) 	{  		// Check that the image's width is a power of 2		if ( (surface->w & (surface->w - 1)) != 0 ) 		{			printf("warning: test.bmp's width is not a power of 2\n");		}				// Also check if the height is a power of 2		if ( (surface->h & (surface->h - 1)) != 0 ) 		{			printf("warning: test.bmp's height is not a power of 2\n");		}				// Set the color key to black		SDL_SetColorKey(surface, SDL_SRCCOLORKEY, SDL_MapRGB(surface->format, 0, 0, 0));					// Have OpenGL generate a texture object handle for us		glGenTextures( 1, &texture );			// Bind the texture object		glBindTexture( GL_TEXTURE_2D, texture );			// Set the texture's stretching properties		glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR );		glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR );			// Edit the texture object's image data using the information SDL_Surface gives us		glTexImage2D( GL_TEXTURE_2D, 0, 3, surface->w, surface->h, 0,					  GL_BGR, GL_UNSIGNED_BYTE, surface->pixels );				return 0;	} 	else 	{		printf("SDL could not load test.bmp: %s\n", SDL_GetError());		return 1;	}	// Free the SDL_Surface only if it was successfully created	if ( surface ) { 		SDL_FreeSurface( surface );	}
------------------ERROR: Brain not found. Please insert a new brain!“Do nothing which is of no use.” - Miyamoto Musashi.
u need an alpha channel with a texture, create the alpha channel in a paint program.

u need to use the following
glTexImage2D( GL_TEXTURE_2D, 0, GL_RGBA8, surface->w, surface->h, 0, GL_BGRA, GL_UNSIGNED_BYTE, surface->pixels );

with glEnable( GL_ALPHA_TEST );
glAlphaFunc( GL_GREATER, 0.0 );
Quote:Original post by benang00
Well, I've tried that but it didn't work. FYI, I'm using SDL to load the image and then I generate the texture from the SDL surface that was created.

It doesn't work because "SDL_SetColorKey" doesn't actually change the image, it just tells SDL what the color key will be when you use SDL to blit it to the framebuffer, but that never happens.

You actually need to copy the RGB image into a new RGBA image, setting color keyed pixels as transparent. Then you send the RGBA image to OpenGL. You can do that manually, or you should be able to blit the image with color keying into an RGBA surface; that should work as long as the RGBA surface is transparent before you blit, although I haven't tested.
Quote:Original post by zedz
u need an alpha channel with a texture, create the alpha channel in a paint program.

u need to use the following
glTexImage2D( GL_TEXTURE_2D, 0, GL_RGBA8, surface->w, surface->h, 0, GL_BGRA, GL_UNSIGNED_BYTE, surface->pixels );

with glEnable( GL_ALPHA_TEST );
glAlphaFunc( GL_GREATER, 0.0 );


Well, that means I've got to make non BMP (maybe a JPG or PNG) texture assets because AFAIK BMP images can't have alpha channel. Although I can do that by implementing SDL and SDL_image to load the image, I want to know how to load a BMP texture with color key.

Quote:Original post by Nathan Baum
It doesn't work because "SDL_SetColorKey" doesn't actually change the image, it just tells SDL what the color key will be when you use SDL to blit it to the framebuffer, but that never happens.

You actually need to copy the RGB image into a new RGBA image, setting color keyed pixels as transparent. Then you send the RGBA image to OpenGL. You can do that manually, or you should be able to blit the image with color keying into an RGBA surface; that should work as long as the RGBA surface is transparent before you blit, although I haven't tested.


You're right. I've read the SDL documentation and it only ignores the color key when blitting. D'oh! :( I've also tried to blit a 24 bit RGB surface with color key to a new 32 bit RGBA with per pixel alpha channel of 0 (transparent) but the color key still didn't become transparent. Even it makes the color wrong. :D

Maybe the only thing I can do is changing the pixel data directly.
------------------ERROR: Brain not found. Please insert a new brain!“Do nothing which is of no use.” - Miyamoto Musashi.
BMP support 32 bit. Just set bits value in the header to 32.
jpg also supports 32 bit but I think libjpeg can't work with them.
png, yes for sure.

There is no such thing as color keying in GL. You need to emulate it like the others have shown.
Sig: http://glhlib.sourceforge.net
an open source GLU replacement library. Much more modern than GLU.
float matrix[16], inverse_matrix[16];
glhLoadIdentityf2(matrix);
glhTranslatef2(matrix, 0.0, 0.0, 5.0);
glhRotateAboutXf2(matrix, angleInRadians);
glhScalef2(matrix, 1.0, 1.0, -1.0);
glhQuickInvertMatrixf2(matrix, inverse_matrix);
glUniformMatrix4fv(uniformLocation1, 1, FALSE, matrix);
glUniformMatrix4fv(uniformLocation2, 1, FALSE, inverse_matrix);
Quote:Original post by V-man
BMP support 32 bit. Just set bits value in the header to 32.
jpg also supports 32 bit but I think libjpeg can't work with them.
png, yes for sure.

There is no such thing as color keying in GL. You need to emulate it like the others have shown.


Yeah I know that there's no color keying in openGL. But there's alpha blending. So I need to know how to convert from a color key pixel to a pixel with a transparent alpha channel (while the other colors become opaque). And then make a texture out of it. If not, I'll just try to change the assets and add alpha blending to them.
------------------ERROR: Brain not found. Please insert a new brain!“Do nothing which is of no use.” - Miyamoto Musashi.
its easier/better/more flexible if u do it in a paint program eg photoshop/gimp

u can do it yourself with something like

read in rgb texture data
create a rgba texture (empty)
loop through all pixels in the rgb texture
copy the rgb -> rgba
with the a see if the r==255 && g==255 & b==0 if so set a = 0 else a = 255;
ie all yellow pixels will be transparent
you could also do this with "alpha testing" if you´re up to the challenge of providing an alpha channel to all your textures :)
"Game Maker For Life, probably never professional thou." =)

This topic is closed to new replies.

Advertisement