Sign in to follow this  
begemot

transfer byte[] to effect

Recommended Posts

I have byte array of image's data (or pointer).
I have effect with pixel shader to fill result texture. I want to work with my image data in pixel shader without unnecessary transformations.
How I can transfer bytes array to my Effect and how I must define it in HLSL program?
I'm using SlimDX and C#.
Help me,please!!!

Share this post


Link to post
Share on other sites
Usually this kind of thing is done via textures.

A "texture" is essentially (and in somewhat simplified terms) "a handle to video memory area that contains texels for the purpose of sampling filtered image-type data by the gpu".

Typically, hardware supports several texture formats so "unnecessary transforms" are seldom needed. If you are targeting hardware with D3D10 and up, you can also load data from custom structured buffers in the pixel shader (but you have to do your own filtering).

Actual texture objects are usually faster than custom buffers since hardware tends to have dedicated circuits for the specific purpose of taking filtered samples from known common format textures.

Share this post


Link to post
Share on other sites
I need to use D3D9..

Now I create texture as

[font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"]new[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"] [/color][/size][/font][/size][/font][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"]Texture[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"](device, w[/color][/size][/font][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"], h[/color][/size][/font][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"], 1, [/color][/size][/font][/size][/font][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"]Usage[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"].Dynamic, [/color][/size][/font][/size][/font][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"]Format[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"].A8R8G8B8,[/color][/size][/font][/size][/font][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"]Pool[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"].Default);[/color][/size][/font][/size][/font]

[font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"]and then fill it [/color][/size][/font][/size][/font]

[font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"]DataRectangle[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"] a_ = t.LockRectangle(0, [/size][/font][/size][/font][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"]LockFlags[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"].None);[/size][/font][/size][/font][/color][/size][/font][/size][/font]
[font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"]int[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"] pitch = (([/size][/font][/size][/font][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"]int[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"])a_.Pitch / [/size][/font][/size][/font][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"]sizeof[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"]([/size][/font][/size][/font][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"]ushort[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"]));[/size][/font][/size][/font][/color][/size][/font][/size][/font]
[font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"]DataStream[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"] a = a_.Data;[/size][/font][/size][/font][/color][/size][/font][/size][/font]
[font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"]unsafe[/color][/size][/font][/color][/size][/font][/color][/size][/font][/color][/size][/font][/size][/font]
[font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"][font="Consolas"][size="2"][font="Consolas"][size="2"]{[/size][/font][/size][/font][/color][/size][/font][/size][/font]
[font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"]ushort[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"]* to = ([/size][/font][/size][/font][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"]ushort[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"]*)a.DataPointer;[/size][/font][/size][/font][/color][/size][/font][/size][/font]
[font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"]ushort[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"]* from = ([/size][/font][/size][/font][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"]ushort[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"]*)ptr.ToPointer();[/size][/font][/size][/font][/color][/size][/font][/size][/font]
[font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"]for[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"] ([/size][/font][/size][/font][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"]int[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"] j = 0; j < heightdef; j++)[/size][/font][/size][/font][/color][/size][/font][/size][/font]
[font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"]for[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"] ([/size][/font][/size][/font][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"]int[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"] i = 0; i < pit; i++)[/size][/font][/size][/font][/color][/size][/font][/size][/font]
[font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"][font="Consolas"][size="2"][font="Consolas"][size="2"]to[i + j * pitch] = from[i + j * pit];[/size][/font][/size][/font][/color][/size][/font][/size][/font]
[font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"][font="Consolas"][size="2"][font="Consolas"][size="2"]}[/size][/font][/size][/font][/color][/size][/font][/size][/font]
[font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"][font="Consolas"][size="2"][font="Consolas"][size="2"]t.UnlockRectangle(0);[/size][/font][/size][/font][/color][/size][/font][/size][/font]

[font="Consolas"][size="2"][font="Consolas"][size="2"] [/size][/font][/size][/font]
[font="Consolas"][size="2"][font="Consolas"][size="2"] [/size][/font][/size][/font]
[font="Consolas"][size="2"][font="Consolas"][size="2"]then I can set it to my effect to texture parameter: effect.SetTexture("Tex",t). [/size][/font][/size][/font]
[font="Consolas"][size="2"][font="Consolas"][size="2"] Is exist any faster way to update texture in effect?[/size][/font][/size][/font]

Share this post


Link to post
Share on other sites
It is faster to perform block copies of the data (CopyMemory or Marshal.Copy) than copying pixel per pixel.

You may also want to use double buffering; keep a pair of textures and update the other when you use the another in rendering the current frame. This helps to prevent stalling (condition where either CPU or GPU waits for access because other side has a resource locked).

Setting effect variables is not particularly slow; internally, when you apply an effect pass, the framework sets the texture for you using IDirect3DDevice9::SetTexture.

If you don't have to lock the texture each frame, then don't lock it each frame :) I assume that you have a video source or something like that here, though.

Share this post


Link to post
Share on other sites
It seems to me, that I can not use Marshal.Copy so as pitch of source image not equals texture's pitch at times..
I'm dreaming to remove this copying process..


Yes. I need get and display frames from video file in realtime mode)))

Share this post


Link to post
Share on other sites
May be, those copying code can be exclude in the following ways:
1. My idea to put bytes to shader/effect
2. Create texture from memory. In this case I have error in code

[font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"]Texture[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"].FromMemory(device, imageBytes,widthdef,heightdef,1,[/color][/size][/font][/size][/font][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"]Usage[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"].None,[/color][/size][/font][/size][/font][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"]Format[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"].A8R8G8B8,[/color][/size][/font][/size][/font][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"]Pool[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"].Default,[/color][/size][/font][/size][/font][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"]Filter[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"].None,[/color][/size][/font][/size][/font][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"]Filter[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"].None,[/color][/size][/font][/size][/font][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"]Color[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"].White.ToArgb())[/color][/size][/font][/size][/font]
[font="Consolas"][size="2"][font="Consolas"][size="2"][color="#000000"]In what I'm mistake?[/color][/size][/font][/size][/font]

3. Create one - dimension texture and pass it to shader/effect

Share this post


Link to post
Share on other sites
You can use Marshal.Copy to copy one line at a time. This is still far more efficient than doing byte-per-byte copies.

I believe that Texture.FromMemory expects to find image file contents from the source array, not just the image pixel data.

Share this post


Link to post
Share on other sites
Also pay attention to the lock flags. If you don't need to read back the texture, be sure to request write only access and discard behavior.

Share this post


Link to post
Share on other sites
One trick that can often be done with video data is to get the pixel shader to do the YUV->RGB conversion. You'll probably want to use three 8-bit textures for this (e.g. D3DFMT_A8). That method has several benefits:

- The GPU can do the conversion much quicker than the CPU.
- You need to upload less data (because U and V are a quarter of the size of Y).
- Saves a fair bit of memory too.

Ideally you would get your video codec to output directly to your texture too, but that's not always so easy.

Share this post


Link to post
Share on other sites
Which of ways of creating texture you suppose is the fastest?

Texture.FromMemory
Texture.FromStream
new Texture() and copy data

Share this post


Link to post
Share on other sites
It varies based on system architecture, and the source of the stream in FromStream (and a number of other things). A stream could be a disk file or a network socket, which are generally way slower than a local memory buffer.

FromMemory and FromStream differ from your third method in that they read and decode whole image file contents whereas in manual copying, you generally only handle the actual pixel data.

You are not supposed to create new textures each frame anyway; if you use a texture in more than one frame (which is very, very common), you allocate it outside the render loop when you initialize your objects and update it as needed.

Share this post


Link to post
Share on other sites
You might also want to look at [url="http://slimdx.org/docs/#Overload_SlimDX_Direct3D9_Surface_FromMemory"]Suface.FromMemory()[/url] which will do all the copying for you from a byte[] array.

Share this post


Link to post
Share on other sites
I tested creating texture from memory using Texture.FromMemory () (Nik02 was right in data format - it must contain BITMAPINFOHEADER),but it is very slow..
Now I do CopyMemory -this way a little bit faster when unsafe copying.

Working with video memory throw surfaces is faster then throw textures?

Share this post


Link to post
Share on other sites
Texture.FromMemory allocates a new texture each time. Allocations are always much slower than updates.

A surface is a block of video memory. A texture is a collection of 1...n surfaces. If you are updating one level of a texture, you are effectively updating a surface.

Share this post


Link to post
Share on other sites
But I don't understand how I pass my surface to shader..
I can create texture, Get surface as
[font="Consolas"][size="2"][font="Consolas"][size="2"]Texture.GetSurfaceLevel(0)[/size][/font][/size][/font],
[font="Consolas"][size="2"][font="Consolas"][size="2"]update surface using line per line CopyMemory and pass to shader texture. Is it right?[/size][/font][/size][/font]

Share this post


Link to post
Share on other sites
You pass textures to the shader.

You can create a texture with only one surface, by setting the mipLevels field of the texture descriptor to 1.

Share this post


Link to post
Share on other sites
I have texture and can do next operation

[font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"]using[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"] ([/size][/font][/size][/font][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"]Surface[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"] surface = t.GetSurfaceLevel(0))[/size][/font][/size][/font]
[font="Consolas"][size="2"][font="Consolas"][size="2"]{[/size][/font][/size][/font]
[font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"]DataRectangle[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"] dr = surface.LockRectangle([/size][/font][/size][/font][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"][font="Consolas"][size="2"][color="#2b91af"]LockFlags[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"].Discard);[/size][/font][/size][/font]
[font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"]int[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"] d_pit = dr.Pitch;[/size][/font][/size][/font]
[font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"]for[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"] ([/size][/font][/size][/font][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"][font="Consolas"][size="2"][color="#0000ff"]int[/color][/size][/font][/color][/size][/font][/color][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"] j = 0; j < heightdef; j++)[/size][/font][/size][/font]
[font="Consolas"][size="2"][font="Consolas"][size="2"]CopyMemory(dr.Data.DataPointer.ToInt32() + j*d_pit,[/size][/font][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"]ptr.ToInt32() + [/size][/font][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"]j*pit,[/size][/font][/size][/font][font="Consolas"][size="2"][font="Consolas"][size="2"]d_pit);[/size][/font][/size][/font]
[font="Consolas"][size="2"][font="Consolas"][size="2"]surface.UnlockRectangle();[/size][/font][/size][/font]
[font="Consolas"][size="2"][font="Consolas"][size="2"]}[/size][/font][/size][/font]

[font="Consolas"][size="2"][font="Consolas"][size="2"]then I pass it to shader.[/size][/font][/size][/font]

[font="Consolas"][size="2"][font="Consolas"][size="2"]Is it the fastest way? [/size][/font][/size][/font]

Share this post


Link to post
Share on other sites
In general, line by line copy is the fastest robust method with which you can copy stuff to surfaces. To optimize, you could check whether the destination pitch equals width*number of bytes per pixel, and if it happens to be the same, copy all the data in one call. If not, just copy line per line like you do now.

The texture usage flags and lock flags are very important for performance. Discard is often the desired behavior, when you want to overwrite the entire destination data.

I previously mentioned double buffering; have you tried it?

And finally, have you profiled your application? Do you have actual knowledge of where your performance bottleneck is?

Share this post


Link to post
Share on other sites
My problem is in copying a lot of memory in textures. I need execute operation of updating for 36 images with rate 25 times per second. If I have images 300x200 size I takes about 60 ms for copying and display. But if my pictures reach 700x500 size it takes just 150 ms (I tested with 16bpp format). I think to resize my images before copying, but I don't know how to do it. can ipp libs do it? Is it realy?

Share this post


Link to post
Share on other sites
Also YUV->RGB conversion was discussed. Can you help me in this? Can I get frames from my video files in YUV format or I must do any convertions between getting frame and copying steps?

Share this post


Link to post
Share on other sites
[quote name='myself' timestamp='1335526018' post='4935350']
have you profiled your application? Do you have actual knowledge of where your performance bottleneck is?
[/quote]

Share this post


Link to post
Share on other sites
The YUV to RGB conversion can be done in pixel shader, by using the conversion formula found here: [url="http://en.wikipedia.org/wiki/YUV"]http://en.wikipedia.org/wiki/YUV[/url]

Because of the simple logic, GPU can perform this operation very fast per pixel.

Some hardware is capable of using YUV format textures directly; they will perform the conversion to RGB automatically.

Whether or not you have YUV data available to begin with, depends entirely on your video codec of choice.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this