Textures from AVI frames?
Author
Hi everyone,
I'm struggling to come up with a way to solve this, and I hope someone here might be able to help me. At the minute I have a application made in Direct3D that essentially creates a 2D world, using methods described in other parts of the site. My program is capable of drawing textured quads to the screen, and also some other basic stuff which is irrelevant here.
The program I am writing has an input file which gives a list of frames from an AVI file, and their co-ordinates in the 2D world. What I need to be able to do is to extract these frames from the video files, and use them as textures for the quads, and then position the quads appropriately in the world.
I have 2 vague ideas about how to approach this at the moment: 1) Using the Win32 API to get DIBs for each of the frames, and somehow creating the textures from these DIBs, or 2) Use DirectShow to extract these frames and draw them onto the surface of the quad.
Now, I've barely used DirectShow before, so I'm not fully sure of it's capabilities. I've gotten an app that can play a video stream, but is it possible to extract a given frame given the number and to map it as a texture?
Any thoughts or opinions about how to do this would be welcomed. If it seems both of these ways are impossible, does anyone have any other ideas about how to do this?
Thanks a lot,
Tarmin.
1. Open <<SDK Root>>\Samples\C++\DirectShow\BaseClasses and
build the solution (or workspace) in that directory. You'll get "strmbasd.lib" in "debug" folder (if you need a release library, set configuration to release).
2.Put all headers and the lib into your project folder (or create a subfolder).
Here's the code:
TextureAnim.h
AnimTexture.cpp:
Usage:
(in globals or where you want it to be):
(loading):
(frame update):
(drawing):
(shutdown):
!EDIT! and don't forget to link the strmbasd.lib! The source is taken from a book Anvanced Animation width Direct3D.
build the solution (or workspace) in that directory. You'll get "strmbasd.lib" in "debug" folder (if you need a release library, set configuration to release).
2.Put all headers and the lib into your project folder (or create a subfolder).
Here's the code:
TextureAnim.h
#ifndef _ANIMTEXTURE_H_#define _ANIMTEXTURE_H_#include <windows.h>#include <d3d9.h>#include <d3dx9.h>#include <dshow.h>#include "streams.h"// Define a macro to help release COM objects#ifndef ReleaseCOM#define ReleaseCOM(x) { if(x!=NULL) x->Release(); x=NULL; }#endifstruct __declspec(uuid("{61DA4980-0487-11d6-9089-00400536B95F}")) CLSID_AnimatedTexture;class CTextureFilter : public CBaseVideoRenderer{ public: IDirect3DDevice9 *m_pD3DDevice; // 3-D device IDirect3DTexture9 *m_pTexture; // Video texture storage D3DFORMAT m_Format; LONG m_lVideoWidth; // Width of video surface LONG m_lVideoHeight; // Height of video surface LONG m_lVideoPitch; // Pitch of video surface public: CTextureFilter(IDirect3DDevice9 *pD3DDevice, LPUNKNOWN pUnk = NULL, HRESULT *phr = NULL); HRESULT CheckMediaType(const CMediaType *pMediaType); HRESULT SetMediaType(const CMediaType *pMediaType); HRESULT DoRenderSample(IMediaSample *pMediaSample); IDirect3DTexture9 *GetTexture();};class CAnimatedTexture{ protected: IGraphBuilder *m_pGraph; // Filter graph IMediaControl *m_pMediaControl; // Playback control IMediaPosition *m_pMediaPosition; // Positioning control IMediaEvent *m_pMediaEvent; // Event control IDirect3DDevice9 *m_pD3DDevice; // 3-D device IDirect3DTexture9 *m_pTexture; // Texture object public: CAnimatedTexture(); ~CAnimatedTexture(); // Load and free an animated texture object BOOL Load(IDirect3DDevice9 *pDevice, char *Filename); BOOL Free(); // Update the texture and check for looping BOOL Update(); // Called at end of animation playback virtual BOOL EndOfAnimation(); // Play and stop functions BOOL Play(); BOOL Stop(); // Restart animation or go to specific time BOOL Restart(); BOOL GotoTime(REFTIME Time); // Return texture object pointer IDirect3DTexture9 *GetTexture();};#endifAnimTexture.cpp:
#include "AnimTexture.h"CTextureFilter::CTextureFilter(IDirect3DDevice9 *pD3DDevice, LPUNKNOWN pUnk, HRESULT *phr) : CBaseVideoRenderer(__uuidof(CLSID_AnimatedTexture), NAME("ANIMATEDTEXTURE"), pUnk, phr){ // Save device pointer m_pD3DDevice = pD3DDevice; // Return success *phr = S_OK;}HRESULT CTextureFilter::CheckMediaType(const CMediaType *pMediaType){ // Only accept video type media if(*pMediaType->FormatType() != FORMAT_VideoInfo) return E_INVALIDARG; // Make sure media data is using RGB 24-bit color format if(IsEqualGUID(*pMediaType->Type(), MEDIATYPE_Video) && IsEqualGUID(*pMediaType->Subtype(), MEDIASUBTYPE_RGB24)) return S_OK; return E_FAIL;}HRESULT CTextureFilter::SetMediaType(const CMediaType *pMediaType){ HRESULT hr; VIDEOINFO *pVideoInfo; D3DSURFACE_DESC ddsd; // Retrive the size of this media type pVideoInfo = (VIDEOINFO *)pMediaType->Format(); m_lVideoWidth = pVideoInfo->bmiHeader.biWidth; m_lVideoHeight = abs(pVideoInfo->bmiHeader.biHeight); m_lVideoPitch = (m_lVideoWidth * 3 + 3) & ~(3); // Create the texture that maps to this media type if(FAILED(hr = D3DXCreateTexture(m_pD3DDevice, m_lVideoWidth, m_lVideoHeight, 1, 0, D3DFMT_A8R8G8B8, D3DPOOL_MANAGED, &m_pTexture))) return hr; // Get texture description and verify settings if(FAILED(hr = m_pTexture->GetLevelDesc(0, &ddsd))) return hr; m_Format = ddsd.Format; if(m_Format != D3DFMT_A8R8G8B8 && m_Format != D3DFMT_A1R5G5B5) return VFW_E_TYPE_NOT_ACCEPTED; return S_OK;}HRESULT CTextureFilter::DoRenderSample(IMediaSample *pMediaSample){ // Get a pointer to video sample buffer BYTE *pSamplePtr; pMediaSample->GetPointer(&pSamplePtr); // Lock the texture surface D3DLOCKED_RECT d3dlr; if(FAILED(m_pTexture->LockRect(0, &d3dlr, 0, 0))) return E_FAIL; // Get texture pitch and pointer to texture data BYTE *pTexturePtr = (BYTE*)d3dlr.pBits; LONG lTexturePitch = d3dlr.Pitch; // Offset texture to bottom line, since video // is stored upside down in buffer pTexturePtr += (lTexturePitch * (m_lVideoHeight-1)); // Copy the bits using specified video format int x, y, SrcPos, DestPos; switch(m_Format) { case D3DFMT_A8R8G8B8: // 32-bit // Loop through each row, copying bytes as you go for(y=0;y<m_lVideoHeight;y++) { // Copy each column SrcPos = DestPos = 0; for(x=0;x<m_lVideoWidth;x++) { pTexturePtr[DestPos++] = pSamplePtr[SrcPos++]; pTexturePtr[DestPos++] = pSamplePtr[SrcPos++]; pTexturePtr[DestPos++] = pSamplePtr[SrcPos++]; pTexturePtr[DestPos++] = 0xff; } // Move pointers to next line pSamplePtr += m_lVideoPitch; pTexturePtr -= lTexturePitch; } break; case D3DFMT_A1R5G5B5: // 16-bit // Loop through each row, copying bytes as you go for(y=0;y<m_lVideoHeight;y++) { // Copy each column SrcPos = DestPos = 0; for(x=0;x<m_lVideoWidth;x++) { *(WORD*)pTexturePtr[DestPos++] = 0x8000 + ((pSamplePtr[SrcPos+2] & 0xF8) << 7) + ((pSamplePtr[SrcPos+1] & 0xF8) << 2) + (pSamplePtr[SrcPos] >> 3); SrcPos += 3; } // Move pointers to next line pSamplePtr += m_lVideoPitch; pTexturePtr -= lTexturePitch; } break; } // Unlock the Texture if(FAILED(m_pTexture->UnlockRect(0))) return E_FAIL; return S_OK;}IDirect3DTexture9 *CTextureFilter::GetTexture(){ return m_pTexture;}CAnimatedTexture::CAnimatedTexture(){ // Clear class data m_pGraph = NULL; m_pMediaControl = NULL; m_pMediaPosition = NULL; m_pMediaEvent = NULL; m_pD3DDevice = NULL; m_pTexture = NULL;}CAnimatedTexture::~CAnimatedTexture(){ Free();}BOOL CAnimatedTexture::Load(IDirect3DDevice9 *pDevice, char *Filename){ cTextureFilter *pTextureFilter = NULL; IBaseFilter *pFilter = NULL; IPin *pFilterPinIn = NULL; IBaseFilter *pSourceFilter = NULL; IPin *pSourcePinOut = NULL; WCHAR wFilename[MAX_PATH]; HRESULT hr; // Free prior instance Free(); // Store device if((m_pD3DDevice = pDevice) == NULL) return FALSE; // Create the filter graph manager CoCreateInstance(CLSID_FilterGraph, NULL, CLSCTX_INPROC_SERVER, IID_IGraphBuilder, (void **)&m_pGraph); // Add filter pTextureFilter = new cTextureFilter(m_pD3DDevice, NULL, &hr); pFilter = pTextureFilter; m_pGraph->AddFilter(pFilter, L"ANIMATEDTEXTURE"); // Add source filter mbstowcs(wFilename, Filename, MAX_PATH); m_pGraph->AddSourceFilter(wFilename, L"SOURCE", &pSourceFilter); // Find the input and out pins and connect together pFilter->FindPin(L"In", &pFilterPinIn); pSourceFilter->FindPin(L"Output", &pSourcePinOut); m_pGraph->Connect(pSourcePinOut, pFilterPinIn); // Query for interfaces m_pGraph->QueryInterface(IID_IMediaControl, (void **)&m_pMediaControl); m_pGraph->QueryInterface(IID_IMediaPosition, (void **)&m_pMediaPosition); m_pGraph->QueryInterface(IID_IMediaEvent, (void **)&m_pMediaEvent); // Get pointer to texture m_pTexture = pTextureFilter->GetTexture(); // Start the graph running Play(); // Free interfaces used here ReleaseCOM(pFilterPinIn); ReleaseCOM(pSourceFilter); ReleaseCOM(pSourcePinOut); return TRUE;}BOOL CAnimatedTexture::Free(){ // Stop any playback Stop(); // Release all COM objects ReleaseCOM(m_pMediaControl); ReleaseCOM(m_pMediaEvent); ReleaseCOM(m_pMediaPosition); ReleaseCOM(m_pGraph); // Release the texture ReleaseCOM(m_pTexture); // Clear Direct3D object pointer m_pD3DDevice = NULL; return TRUE;}BOOL CAnimatedTexture::Update(){ long lEventCode; long lParam1; long lParam2; // Error checking if(!m_pMediaEvent) return FALSE; // Process all waiting events while(1) { // Get the event if(FAILED(m_pMediaEvent->GetEvent(&lEventCode, &lParam1, &lParam2, 1))) break; // Call the end of animation function if playback complete if(lEventCode == EC_COMPLETE) EndOfAnimation(); // Free the event resources m_pMediaEvent->FreeEventParams(lEventCode, lParam1, lParam2); } return TRUE;}BOOL CAnimatedTexture::EndOfAnimation(){ return Restart();}BOOL CAnimatedTexture::Play(){ if(m_pMediaControl != NULL) m_pMediaControl->Run(); return TRUE;}BOOL CAnimatedTexture::Stop(){ if(m_pMediaControl != NULL) m_pMediaControl->Stop(); return TRUE;}BOOL CAnimatedTexture::Restart(){ return GotoTime(0);}BOOL CAnimatedTexture::GotoTime(REFTIME Time){ if(m_pMediaPosition != NULL) m_pMediaPosition->put_CurrentPosition(Time); return TRUE;}IDirect3DTexture9 *CAnimatedTexture::GetTexture(){ return m_pTexture;}Usage:
(in globals or where you want it to be):
// Animated water textureCAnimatedTexture animTexture;(loading):
// Load the streaming media fileif ( !animTexture.Load ( pD3DDevice, "file.avi" ) ) return FALSE;(frame update):
animTexture.Update();(drawing):
pD3DDevice->SetTexture ( 0, animTexture.GetTexture () );(shutdown):
animTexture.Free ();!EDIT! and don't forget to link the strmbasd.lib! The source is taken from a book Anvanced Animation width Direct3D.
Author
Thanks for the quick reply!
Unfortunately, the computer I'm working on (I'm at work) doesn't have the directShow samples on! But thanks for the guidance none-the-less, I'll have a look at what you posted and see how I get on.
Thanks again,
Tarmin.
Unfortunately, the computer I'm working on (I'm at work) doesn't have the directShow samples on! But thanks for the guidance none-the-less, I'll have a look at what you posted and see how I get on.
Thanks again,
Tarmin.
Thank you so much, I've been looking for a simplified version of the one in the DX Samples.
Now the only problem is that I need to understand what is happening here.
Firstly these functions don't seem like they are used, however they should be otherwise they won't be there:
HRESULT CheckMediaType(const CMediaType *pMediaType);
HRESULT SetMediaType(const CMediaType *pMediaType);
HRESULT DoRenderSample(IMediaSample *pMediaSample);
Secondly Update() only seems to check if the video is finished?
Again thank you for the great help
Now the only problem is that I need to understand what is happening here.
Firstly these functions don't seem like they are used, however they should be otherwise they won't be there:
HRESULT CheckMediaType(const CMediaType *pMediaType);
HRESULT SetMediaType(const CMediaType *pMediaType);
HRESULT DoRenderSample(IMediaSample *pMediaSample);
Secondly Update() only seems to check if the video is finished?
Again thank you for the great help
Author
Ok, I've had a look through and I think this is exactly what I've been looking for! Just a couple of quick questions:
1) I noticed the include "streams.h". I'm guessing from the fact you said to compile it in the DirectShow samples directory, that this file was originally there. Could you tell me what it contains the definitions for, and if it's important for me to get my hands on it?
2) I said previously that I had only the frame number to work off, and the function to navigate the stream seems to work in seconds. Is it possible to get the FPS of the source stream so that I can calculate the time at which each frame occurs?
Thanks very much!
(Btw, if anyone's wondering what I'm doing, I'm trying to build a panorama of a football stadium by taking key frames from a match video)
1) I noticed the include "streams.h". I'm guessing from the fact you said to compile it in the DirectShow samples directory, that this file was originally there. Could you tell me what it contains the definitions for, and if it's important for me to get my hands on it?
2) I said previously that I had only the frame number to work off, and the function to navigate the stream seems to work in seconds. Is it possible to get the FPS of the source stream so that I can calculate the time at which each frame occurs?
Thanks very much!
(Btw, if anyone's wondering what I'm doing, I'm trying to build a panorama of a football stadium by taking key frames from a match video)
You can download the BaseClasses from my site: http://www.gamedev3d.narod.ru/baseshow.zip.
You can get the average fps from CBaseVideoRenderer::get_AvgFrameRate ( int* frameRate ) (derived by CTextureFilter::get_AvgFrameRate ()). But there should be a way to get a specific frame.
You can get the average fps from CBaseVideoRenderer::get_AvgFrameRate ( int* frameRate ) (derived by CTextureFilter::get_AvgFrameRate ()). But there should be a way to get a specific frame.
About the functions: they are used in BaseClasses' implementation. You can have them virtual if you want to.
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement