Jump to content
  • Advertisement
Sign in to follow this  

Accessing and writting texture data

This topic is 4721 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hello, First of all, sorry if this topic has been already commented before on these forums, maybe i missed it out :) I'm writting an augmented-reality application, where 3D content should be displayed over a video layer, captured from a webcam. Im using Managed D3D for the rendering, and DirectShow managed wrapper for capturing the video. I'm displaying the video as a dynamic texture, using a Sprite. Since captured images come at about 20-30 fps and rendering occurs at a higher frequency, I'm using a separate thread for updating the sprite texture: It works like a double buffer; there are 2 textures for the sprite, a visible one and an offscreen texture. Each time a video-sample arrives from the webcam, I write the raw data into the offscreen texture, and swap visible/offscreen references, so the sprite displays it until a new sample is received. My question is about which would be the best (most efficient) method for updating textures dynamically. So far i am creating the textures like this:

bool CreateVideoTextures(Device device)
{
   ...
   VideoTexture_A = new Texture(device, VideoCaptureWidth, VideoCaptureHeight,  1, Usage.SoftwareProcessing, Format.X8R8G8B8, Pool.Managed);
   VideoTexture_B = new Texture(device, VideoCaptureWidth, VideoCaptureHeight, 1, Usage.SoftwareProcessing, Format.X8R8G8B8, Pool.Managed);
	
   VideoTexture_Displayed = VideoTexture_A;
   VideoTexture_Offscreen = VideoTexture_B;
}

----------------

private unsafe void FillTexture(System.IntPtr pVideoSampleBuffer, int BufferLen)
{
/* THIS CODE ONLY HAPPENS WHEN A NEW SAMPLE IS CAPTURED FROM THE WEBCAM */
    ...

    Surface s = VideoTexture_Offscreen.GetSurfaceLevel(0);
    GraphicsStream gs = s.LockRectangle(LockFlags.None, out pitch);

    byte* p = (byte*)gs.InternalDataPointer;			
    
    .... [ copy video image's buffer array into VideoTexture_Offscreen]

    [unlock and free resources]

    // Swap Displayed and Offscreen textures
    Texture refAux = this.VideoTexture_Displayed;
    this.VideoTexture_Displayed = this.VideoTexture_Offscreen;
    this.VideoTexture_Offscreen = refAux;
}

----------------------------

class CVideoPane
void Draw(Device device)
{
/* THIS CODE IS CALLED EACH FRAME */
sprite.Begin(SpriteFlags.None);
					
    device.RenderState.ZBufferEnable = false; // skip ZBuffer, we're drawing in 2D

    sprite.Draw2D(this.VideoTexture_Displayed, 
		  Rectangle.Empty, 
		  destRect, 
		  new Point(0,0), Color.White);			
					
    device.RenderState.ZBufferEnable = true;

    sprite.End();
}




I've tried it on an Ati Radeon 7000 card, and it works fairly well, but when I move to GeForce 5700FX and a GeForce 6600 cards the code doesn't work properly (the sprite is shown with a completely white texture, and if I draw a 3D mesh over it, the sprite seems to use the texture from the mesh!), despite i've make sure they're created and updated) Is there any better way of doing this? maybe creating the VideoTexture_Displayed texture in video memory, since it is displayed much more frequently, and VideoTexture_Offscreen in system memory and then copying them with UpdateSurface? In FillTexture method, the surface.LockRectangle call isn't very fast, so I wonder if i'm making the video card transferring the textures continuously from one memory to another... Any suggestion would be appreciated :) Thanks in advance, Jose [Edited by - jesteve on June 17, 2005 4:51:08 PM]

Share this post


Link to post
Share on other sites
Advertisement
Well, it seems i've managed to find a better solution. So i'm posting it here in case anyone is interested...

I've moved into the following scheme:

There are 2 textures on video memory: Displayed and Offset, and a third one in System memory, called VideoTexture, which receives the copy of the video capture. Since the render loop happens at a much higher frequency than the video capturing, having the Displayed texture into video memory assures a better performance in the inbetween frames. When a new video frame is received, a separate thread copies it into VideoTexture, and when its done activates a flag, indicating there's a new texture to be uploaded to video memory. When that flag is on, before rendering a new frame, a call to UpdateTexture is performed, so VideoTexture (allocated in System memory) is copied into Offscreen texture (video memory). Then Offscreen and Displayed textures are swapped.


[SYSTEM MEMORY] [ VIDEO MEMORY ]

Webcam -----> VideoTexture -- Updates ---> Offscreen
Frame |
swaps
|
V
Displayed <---renders-- Sprite



This way the data transfer is minimized by only updating the textures at the video-capturing frequency (which is relatively low, compared to rendering frequency), and meanwhile, the sprite displays a texture located always on video memory.

The code becomes as follows:



void CreateVideoTextures(Device device)
{
...
this.VideoTexture_SystemMemory = new Texture(device, VideoCaptureWidth, VideoCaptureHeight, 1, Usage.None, SurfaceFormat, Pool.SystemMemory);
this.VideoTexture_VideoMemory_Displayed = new Texture(device, VideoCaptureWidth, VideoCaptureHeight, 1, Usage.None, SurfaceFormat, Pool.Default);
this.VideoTexture_VideoMemory_Offscreen = new Texture(device, VideoCaptureWidth, VideoCaptureHeight, 1, Usage.None, SurfaceFormat, Pool.Default);
...
}

void FillVideoTexture(System.IntPtr pVideoSampleBuffer, int BufferLen)
{
/* THIS CODE ONLY HAPPENS WHEN A NEW SAMPLE IS CAPTURED FROM THE WEBCAM */

...

Surface s = this.VideoTexture_SystemMemory.GetSurfaceLevel(0);
GraphicsStream gs = s.LockRectangle(sourceRect, LockFlags.None, out pitch);
byte* p = (byte*)gs.InternalDataPointer;

.... [ copy pVideoSampleBuffer into VideoTexture_SystemMemory]

[unlock and free resources]

ReadyToSwapTextures = true;

}

void RenderLoop(Device device)
{
/* THIS CODE IS CALLED EACH FRAME */

if (this.ReadyToSwapTextures)
{
// Copy system memory texture onto video memory offscreen surface
device.UpdateTexture(this.VideoTexture_SystemMemory, this.VideoTexture_VideoMemory_Offscreen);

// Swap Displayed and Offscreen textures

Texture refAux = this.VideoTexture_VideoMemory_Displayed;
this.VideoTexture_VideoMemory_Displayed = this.VideoTexture_VideoMemory_Offscreen;
this.VideoTexture_VideoMemory_Offscreen = refAux;

this.ReadyToSwapTextures = false;
}

// Render the video texture

sprite.Begin(SpriteFlags.None);
device.RenderState.ZBufferEnable = false; // skip ZBuffer, we're drawing in 2D
sprite.Draw2D(this.VideoTexture_VideoMemory_Displayed,
Rectangle.Empty,
destRect,
new Point(0,0),
Color.White);

device.RenderState.ZBufferEnable = true;

sprite.End();
}




Cheers,
Jose.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

Participate in the game development conversation and more when you create an account on GameDev.net!

Sign me up!