Before I start I will warn that I am extreamly new to fragment programming, this is actually my first time creating one. I am using the code from http://www.evl.uic.edu/rlk/howto/yuv/ to render a YCbCr texture. This works perfectly fine I only had to make a few modifications to make it work for my needs. Although there is still one problem, my YCbCr format is YUY2 (packed format). So i am unpacking the pixels using the processor then using the fragment program to display the texture. This would normally not be a problem but I am using this to render movies as a texture using DirectShow. The YUY2 format can be found here: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnwmt/html/YUVFormats.asp All I have left to do at this point is unpack the pixels on the hardware. The problem is that when the pixels are packed there are two pixels per pixel in the texture. So my question is how can I move through the texture in a way where 2 adjacent pixels use the same 4 bytes of the texture. For the fragment program my texture would be setup like this using GL_RGBA: Y1 = Texture.r Cb = Texture.g Y2 = Texture.b Cr = Texture.a Two pixels share the same Cb and Cr values with diffent Y values (YUY2 format), any idea how I can move through a texture like this?

Say you've got a 640x480 video feed. You pack two horizontally adjacent pixels into one to give a 320x480 RGBA texture and you render with this texture.

uniform vec2 TextureSize; // size of packed texture, eg. vec2(320, 240)

void main(void){

// get the packed YUY2 colour
vec4 packed = texture2D(myTexture, gl_TexCoord[0]);

// convert to a real pixel coord
float realTexCoordX = gl_TexCoord[0].x * TextureSize.x;

if( fract(realTexCoordX) < 0.5 ){
// do conversion using Y1
}
else{
// do conversion using Y2
}

gl_FragColor = myConvertedColor;
}


Note: texture must be GL_NEAREST filtering. I think this should work but I've pretty much made it up as I typed :-D

>>> Edit >>>>

There might be a single pixel shift introduced in the above code. I can't remember if GL_NEAREST does a floor() (as I've implicitly used in the code) or a round() on pixel coordinates.

[Edited by - coordz on August 11, 2006 3:20:50 AM]

I asked Robert, the author of the EVL doc, about pixel packing. He runs the pixels through a swizzle loop on the CPU to double up the U and V. The Y U V equal the R G B in a 24 bit BGR texture. This is obviously not the most efficient manner to handle YCbCr pixels though.

I do most of my work on OSX and Apple has had a 4:2:2 texture format for over 4 years now. This gives a large performance increase over handling video as RGB. All NV and ATI cards support YCbCr textures and hardware conversion to RGB, but for some reason the drivers do not expose this functionality on Windows.

coordz: Thanks, that did the trick.

LSStudio: I dont get it personaly why the YUY2 format is not supported in OpenGL although it is in DirectX. Perhaps there is just not enough demand for it in OpenGL.