Sign in to follow this  

LockRect and Video RGB

This topic is 3594 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi all, I'm having some trouble getting video to render to a texture in my engine. I'm using ffmpeg to open the video file then using it to convert the frames into RGB format. I'm then trying to put that data into a DX created texture. I can't seem to get it to work properly at all. Heres some of the code I currently have:

...

pSWSCtx = sws_getContext(pCodecCtx->width, pCodecCtx->height, pCodecCtx->pix_fmt, pCodecCtx->width, pCodecCtx->height, PIX_FMT_RGB24, SWS_FAST_BILINEAR, NULL, NULL, NULL);

if ( FAILED(D3DXCreateTexture(CDXMgr::Get().GetD3DDevice(), pCodecCtx->width, pCodecCtx->height, 1, D3DUSAGE_DYNAMIC, D3DFMT_R8G8B8, D3DPOOL_DEFAULT, &g_pTexture)))
return false;

...

// Convert the image from its native format to RGB
sws_scale(pSWSCtx, pFrame->data, pFrame->linesize, 0, pCodecCtx->height, pFrameRGB->data, pFrameRGB->linesize);

D3DLOCKED_RECT lockedRect;
HRESULT hResult = g_pTexture->LockRect(0, &lockedRect, NULL, D3DLOCK_DISCARD);
if(FAILED(hResult))
	return false;

unsigned int uHeight = pCodecCtx->height;
unsigned int uWidth = pCodecCtx->width;
unsigned char *pTextureMemory = static_cast<unsigned char*>(lockedRect.pBits);

for( unsigned int uRow = 0; uRow < uHeight; uRow++ )
{
	unsigned int uRowStart = uRow * lockedRect.Pitch;
	for( unsigned int uCol = 0; uCol < uWidth; uCol++ )
	{
		unsigned int uTexelStart = uRowStart + (uCol*3);
		pTextureMemory[uTexelStart + 0] = pFrame->data[0][uRow*uWidth + uCol*3 +0]; // Red
		pTextureMemory[uTexelStart + 1] = pFrame->data[1][uRow*uWidth + uCol*3 +1]; // Green
		pTextureMemory[uTexelStart + 2] = pFrame->data[2][uRow*uWidth + uCol*3 +2]; // Blue
	}
}
g_pTexture->UnlockRect(0);

Any help is appreciated, thanks.

Share this post


Link to post
Share on other sites
Nearly no cards currently in existance support R8G8B8. CreateTexture in the device will fail if you don't specify exactly valid values. D3DXCreateTexture on the other hand is a wrapper around that, which tries to manipulate your request into something that will succeed. It's probably making an X8R8G8B8 texture, but the only way to be sure is to ask D3D for the surface descriptor once the texture is made.

Also, because you don't know what format you're actually getting, "unsigned int uTexelStart = uRowStart + (uCol*3);" is not correct. The *3 will vary depending on the texture actually made. Since all cards support X8R8G8B8, it's best to just ask for that format directly, and change the *3 to *4. I think you'll also need to change the write order to blue, green, red, as Intel chips store the lowest byte first so the DWORD XXRRGGBB is stored as BB, GG, RR, XX in memory. You're using Pitch correctly.

If you like tons of overkill and want to do what's technically necessary to be 100% right all the time (remember you can't technically assume you'll get X8R8G8B8), you'll need to handle conversions to any format, of whatever bitdepth and byte order D3D happens to give you. I'd just assume X8R8G8B8 is valid though.

Share this post


Link to post
Share on other sites
Thanks for the replies. I dont want to use DirectShow because its seems so confusing and is apparantly not very good for use in a game engine.

Namethatnobodyelsetook:

I'm starting to think half of my problem is how to get the data from the ffmpeg frame and put it into a texture. I found an example of how to save frames as .PPM format. Here's how they do it:


void SaveFrame(AVFrame *pFrame, int width, int height, int iFrame)
{
FILE *pFile;
char szFilename[32];
int y;

// Open file
sprintf(szFilename, "frame%d.ppm", iFrame);
pFile=fopen(szFilename, "wb");
if(pFile==NULL)
return;

// Write header
fprintf(pFile, "P6\n%d %d\n255\n", width, height);

// Write pixel data
for(y=0; y<height; y++)
fwrite(pFrame->data[0]+y*pFrame->linesize[0], 1, width*3, pFile);

// Close file
fclose(pFile);
}



The output of that is fine and loads in Photoshop. So how would I use this way of accessing the data and copy it into my DirectX texture?

Cheers!

Share this post


Link to post
Share on other sites
You could do the reverse of the write pixel data loops to read back the information and write it into a d3d texture (using LockRect)

but loading each frame from a file is a bad way of doing it..is there any way to read frame by frame from the video file directly?

Share this post


Link to post
Share on other sites
Yeah you can read it frame by frame, thats what I'm doing at the moment. But my problem is, getting the image data of the frame and putting it into a DirectX texture. Using ffmpeg I can convert the video format from the video format YUV into RGB, which I've tested and works because that SaveFrame function I posted spits out valid .PPM files which contain RGB data that I can open in photoshop.

So my problem is, how do I read the RGB data of the frame correctly, and how do I copy it into a DirectX texture correctly?

I've been messing around with it for over a day now and can't seem to work it out. Maybe someone who has knowledge with LockRect and image formats could come save me from this torment hehe.

Cheers. :)

Share this post


Link to post
Share on other sites
In my original post theres a function called "sws_scale" which takes the original video frame that is in whatever format, usually YUV I think, and then converts it into another frame with a format I specify. As you can see I specified PIX_FMT_RGB24 as my format to convert to. Which according to the comments in the ffmpeg header it is this.. "// Packed RGB 8:8:8, 24bpp, RGBRGB..."

The raw RGB data is stored in pFrameRGB->data and.. pFrameRGB->linesize I think is the "pitch". Theres not much documentation unfortunately, only this shoddy doxygen:

http://cekirdek.pardus.org.tr/~ismail/ffmpeg-docs/structAVPicture.html#92fce3ff225b920a33c1ad866f4008cb

Btw I've tried doing a similar approach to the SaveFrame function but it doesnt seem to produce the right results.. just a random garbled image..


BYTE* pTextureData = (BYTE *)lockedRect.pBits;
for(int y=0; y < pCodecCtx->height; y++)
memcpy(pTextureData + y*lockedRect.Pitch, pFrameRGB->data[0]+y*pFrameRGB->linesize[0], pCodecCtx->width*4);




EDIT:

I've managed to get it working using the code above. It took a bit but I had to go through and get all the formats from ffmpeg matching the texture formats of directx. My only problem now is the image is upside down heh.

Cheers for your help guys.

[Edited by - Shael on February 14, 2008 8:30:52 PM]

Share this post


Link to post
Share on other sites

This topic is 3594 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this