Jump to content

  • Log In with Google      Sign In   
  • Create Account

Like
8Likes
Dislike

GPU Texture Snapshot

By Jay Taoko | Published Sep 29 2013 07:36 PM in APIs and Tools
Peer Reviewed by (Servant of the Lord, Hodgman, Dave Hunt)

picopixel sdk texture gpu

As you know, 3D graphics engineering deals with a lot of textures. And often, you need to see what’s in those textures. Especially if they are the result of a rendering pass like your depth buffer, G-buffer… However, seeing what’s in the textures while your program is running is a bit of a challenge. Usually, you have to dump texture data into a file format that can be opened by an image viewer.

I have always found the task of viewing GPU generated textures to be complicated and unnecessary. What if I could just display the texture on the screen? Just like printf does with text!

Printf


Any programming language has the equivalent of C/C++ printf. For instance, every “Hello World!” program uses a print function to output text to a console, file or web page… I thought about what it would take to have a similar API that works for images, the same way printf works for text. The API would have to be:
  • Easy to use like printf.
  • Immediate. You should see your images (somewhere) as soon as the print function is called.
  • Unobtrusive. It should not alter a program’s flow or dramatically affect its performance.
  • It should rely on technology that is native to any system.
That being said, printf has a weakness that would be unsuitable for the goal of printing images. Every time code execution goes over a printf call, text is printed. This is often undesirable. Without any control mechanism, calling printf (for instance, in a loop) ends up flooding the text output outlet. Since images can be large in size, continuously printing them out of a running program could have a dramatic effect on performance.

Designing Printf For Images


What if you could tell a program to export raw image data at a moment of your choosing? This would be an improvement over the way C/C++ printf works. An image would be exported out of a program when the developer chooses to while the rest of the time the program would simply run unaffected.

The solution I designed, has three parts.
  • Just like C/C++ printf has a command line terminal as default output, I needed an outlet for images and I think Pico Pixel desktop application would be perfect for it. Pico Pixel is an image viewer we created. It is fast, simple, supports many pixel formats (DXT, BCn, ETC2…) and file formats such as OpenEXR, HDR, KTX and more.
  • A C/C++ API. Developers call functions of the API to export images out of programs. The API is written in simple C/C++ that any compiler can handle.
  • A communication system between a program and Pico Pixel. Network sockets are good for the job. Sockets libraries are everywhere. They work across hardware platforms and OSes. They are very well documented.
The result is Pico Pixel SDK. The SDK is made of a few files (open source) you drop in your build environment. They should compile without much trouble. To use the SDK you instantiate PicoPixelClient object and open a connection to Pico Pixel desktop application. The method you use to send pixel data to Pico Pixel is called PixelPrintf.

SDK Integration


As I said before, calling printf in C/C++ always outputs text in the console and this is not desirable for images. To prevent images from being sent everytime code execution goes over a call to PixelPrintf, we need something called a marker.

Markers are objects used by PixelPrintf to decide whether to send image data to Pico Pixel or not. Every marker has an associated counter of integer values. While a marker's counter value is greater than 0, any call to PixelPrintf with the marker will be carried through. Each time a marker is used, its counter value is decremented by one.

When a marker's counter value reaches zero, the marker can no longer be used to send image data to Pico Pixel. Its counter value has to be reloaded before it can be used again. Pico Pixel provides a user interface to reload counters. As a result, Pico Pixel acts as the trigger for image snapshot in a program.

Here is what the code looks like.

PicoPixelClient pico_pixel_client("GlxGears");

pico_pixel_client.StartConnection();

// the last parameter '0' means the marker use_count is 0.
int marker = pico_pixel_client.CreateMarker(std::string("Color Buffer"), 0);
int marker = pico_pixel_client.CreateMarker(std::string("Depth Buffer"), 0);

// Send the defined markers to PicoPixel desktop application
pico_pixel_client.SendMarkersToPicoPixel();

// Image data is sent if and only if the marker's use_count is greated than 0.
// A marker's use_count is decremented each time the marker is used.
pico_pixel_client.PixelPrintf(
    marker,                                 // data marker
    "color-framebuffer",                    // image name to appear in Pico Pixel desktop application
    PicoPixelClient::PIXEL_FORMAT_BGR8,     // image pixel data format
    400,                                    // image width
    300,                                    // image height
    1200,                                   // image row pitch in bytes
    FALSE,                                  // set to TRUE if the data is in srgb 
    FALSE,                                  // set to TRUE to rotate the image horizontally when displayed
    raw_data                                // char* pointer to the image raw data
    );

By instrumenting a program with the code above, Pico Pixel interface mirrors the markers that have been defined.

Attached Image: PixelPrintf.png

To take a snapshop of a texture buffer, set a value greater than 0 in the marker used to export the surface. When markers in Pico Pixel are synchronized with markers in the program, images are sent whenever code execution goes over the PixelPrintf call that use markers with positive counter's values.

Conclusion


The best feature of Pico Pixel SDK is that you have a lot of control over images and the moment they are exported. When images are not exported, your program is unaffected by the presence of the SDK.

I am very excited about this feature. It makes Pico Pixel useful in a graphics developer’s toolset. The idea of sharing images over the network between applications is not a new one. However, for us graphics developers, it opens a lot of possibilities if we take advantage of it easily during development. I believe this is a good feature to have and I hope it will save you time in efforts.



About the Author(s)


I am a 3D graphics developer. I have worked in the game industry at EA and then Ubisoft. I spent several years at Canonical in the graphics engineering team of Ubuntu's desktop interface. I am technical architect at inalogic inc.

License


The MIT License




Comments

Great idea!

 

Questions:

1) When you call pico_pixel_client.PixelPrintf(...), you're passing in the pixel data via a pointer. Doesn't this mean that you have to have the pixel data locally, i.e. you're not viewing what's on the videocard but what's in the CPU's RAM?

 

2) Since pulling texture data off of the GPU is rather slow, is there a way to test the marker's use_count manually, so you don't waste time pulling the data off the videocard if the use_count is 0?

 

3) Does the function pull OpenGL textures off the videocard for you?

 

4) How does that function call inform your image viewing program about the mipmaps?

Thanks!

 

1) Yes you have to get the texture GPU data to CPU memory before you can send it with PixelPrintf

 

2) MarkerUseCount() is the function you need (propably going to rename it to something more suitable). You give it a marker and it returns the marker's use_count value

 

3) You have to get the texture out of the GPU yourself. However, In PicoPixelClient.h, look for PixelPrintfGLColorBuffer. This function will pull the color buffer out of the GPU for you and send it over with PixelPrintf. You need to define PICO_PIXEL_CLIENT_OPENGL first before you can use that function. PixelPrintfGLColorBuffer and PixelPrintfGLDepthBuffer are flagged as experimental but feel free to try them. Similar functions could be done for DirectX.

 

4) There is no support for sending a full mipmap chain with PixelPrintf. You need to send the mip textures individually. Pico Pixel relies on the pixel format, width, height and pitch to reconstruct a texture . It should be possible to send a full mipmap chain but since PixelPrintf is a new feature of Pico Pixel, the focus is on making it work well for 2D textures first.

 

Regards

Neat tool/article. However, it's a bit misleading to call it "GPU Texture Snapshot", without having an example of how to get the texture data out of the GPU in there ;)

Neat tool/article. However, it's a bit misleading to call it "GPU Texture Snapshot", without having an example of how to get the texture data out of the GPU in there ;)

 

Hello

 

There is an example of this in the SDK although it is marked as experimental. Look for PixelPrintfGLColorBuffer. This function simply send the content of your color frame buffer to Pico Pixel.

Making a snapshot of GPU texture/data (or any image data for that matter) is really the objective of this SDK.

 

In the case of OpenGL, you may copy a texture from GPU memory using glGetTexImage. However, the SDK is not making any assumptions as to how your programs behave or if glGetTexImage is the best function for the job.

 

Some platforms give you direct access to GPU memory. And sometimes GPU and CPU memory are one and the same. However, it is up to you to choose the best, fastest and less disruptive method of transfer. In any case the SDK provides the building blocks to let you send the data over to Pico Pixel for visualization.

 

Regards

Maybe it's just me but I cannot go to Pico Pixel page. Is it missing a www? Maybe my internet is a bit lazy today.

Maybe it's just me but I cannot go to Pico Pixel page. Is it missing a www? Maybe my internet is a bit lazy today.

 

Sorry, there was a scheduled manitenance on the servers. It is back online now.

What is the purpose of this when you can use glDebugger which is free.

What is the purpose of this when you can use glDebugger which is free.

 

Hello

 

Did you mean gDEBugger?

gDEBugger works at a much lower level than this SDK. It will get you a lot of information even some you may not need. Whereas with this SDK only gets you images and you choose when and what to extract. It is simpler and less disruptive overall than a library that re-routes OpenGL commands.

 

While gDEBugger works with OpenGL or OpenCL, this SDK can work with any type of programs including OpenGl/CL DirectX.

 

Regards


Note: Please offer only positive, constructive comments - we are looking to promote a positive atmosphere where collaboration is valued above all else.




PARTNERS