Sign in to follow this  

OpenGL Real-time OpenGL screen capture

This topic is 2286 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Well this is my first post here but I wanted to ask hopefully a straightforward question.

I'm trying to take a video of the screen frame by frame and I want to do it as close to real time as possible. I've heard there's ways to do it using Win32 GDI, OpenGL, DirectX, but I know that OpenGL is platform-independent so I want to use that.

I've seen things for games like recording Counterstrike games and stuff but I wasn't sure how these worked. Basically if I can get something in a format where I can work with the individual pixels for each frame, that would work out.

Share this post


Link to post
Share on other sites
I want to capture the output. Mainly at the OS level. Like what someone would see on a monitor.

Share this post


Link to post
Share on other sites

There is FRAPS for example
Or [link deleted by moderator in 2017 after a user reported virus]

If you google "screen capture video" for example, you will get lots of other hits.

Share this post


Link to post
Share on other sites
Each time you render: Grab the frame from OGL with glReadPixels(), dump it to a tmp file (possibly use fast Zlib compression to save disk space), and once done recording, encode tmp file frame-by-frame using libogg/libvorbis.

Share this post


Link to post
Share on other sites
Reading from the backbuffer into system memory is going to incur a pipeline stall though, so you'll never get it really "real-time".

Share this post


Link to post
Share on other sites
Quote:
Original post by thefrost
Each time you render: Grab the frame from OGL with glReadPixels(), dump it to a tmp file (possibly use fast Zlib compression to save disk space), and once done recording, encode tmp file frame-by-frame using libogg/libvorbis.
Quote:
Original post by mhagain
Reading from the backbuffer into system memory is going to incur a pipeline stall though, so you'll never get it really "real-time".
Yes to both. However, the second one is not true if you use a pixel buffer object and properly overlap frames. This is a tiny bit more organisational work, but still pretty straightforward.

When a frame is to be recorded, issue a read into a pixel buffer object and push the PBO's identifier as well as any additional information that you might want, such as the current frame's number into a queue. Forget about the PBO for now, let it do its work.
Each frame, the queue is checked for frames that are at least 2 frames old. You can use the previous frame as well, this works exactly the same. However, I prefer being 100% on the safe side, and it doesn't really matter to wait a fraction of a second longer, so I'm waiting two frames.
If there are any candidate frames on the queue, their PBOs are mapped (GL_STREAM_READ) and handed to GDI+ for compression, and are written to disk (need to do one copy to flip them upside down, but that's a simple fast loop).

In my measurements, the overhead of this is exactly zero (meaning there is no measurable difference whatsoever in frame rates with or without screenshots, and the additional CPU load is under threshold).
Note, however, that supplying wrong flags when creating or mapping your PBO may cut your frame rate in half (due to stalls)!

There is no reason why the same could not be done for recording a video as well, which basically is no more than screenshotting every frame.

Share this post


Link to post
Share on other sites
Wow lots of responses so far! So it sounds like a solution like this is definitely possible. I'm not really concerned about any stalls as long as they're not cumulative. As long as frames enter a program and exit to video at a fixed rate, it's no problem.

Decrius: I agree with the open source libraries solution. I'm not looking for a currently finished application.

Samoth: Sounds like you have a pretty good understanding about this. How much code do you think this would take? Is it something I can run in the background easily, maybe as some kind of DLL?

Share this post


Link to post
Share on other sites
Do you want to make it yourself, or do you want an existing program?

Quote:
Original post by szecs
There is FRAPS for example
Or CamStudio // this is free

If you google "screen capture video" for example, you will get lots of other hits.

Share this post


Link to post
Share on other sites
Quote:
Original post by szecs
Do you want to make it yourself, or do you want an existing program?

Quote:
Original post by szecs
There is FRAPS for example
Or CamStudio // this is free

If you google "screen capture video" for example, you will get lots of other hits.


Prefer to write it myself. It will be part of a larger solution.

Share this post


Link to post
Share on other sites
Quote:
Original post by stevus06
Decrius: I agree with the open source libraries solution. I'm not looking for a currently finished application.


Sorry! Didn't thought you needed it embedded.

Share this post


Link to post
Share on other sites
Quote:
Original post by Decrius
Quote:
Original post by stevus06
Decrius: I agree with the open source libraries solution. I'm not looking for a currently finished application.


Sorry! Didn't thought you needed it embedded.


Well if there is something in those libraries that could be useful, it is an option. Just to clarify though, I don't want to use something already developed and packaged. If I have access to source, or some kind of library, that would be preferable.

Share this post


Link to post
Share on other sites
Quote:
Original post by stevus06
How much code do you think this would take? Is it something I can run in the background easily, maybe as some kind of DLL?
It depends on how easy or sophisticated you want it. My implementation has maybe a dozen OpenGL calls, half a dozen GDI+ calls, and little over a dozen lines of code otherwise.

If "running in the background as some kind of DLL" means you make a DLL that records screens from some unknown foreign program, you will need to make a fake opengl32.dll which exports the functions up to 1.2 (forwarding to the original functions) and hooks a "key" function such as for example SwapBuffers (forwarding to the original function, but also doing the capture).

If "running in the background as some kind of DLL" simply means running without impacting your game, and being able to put it in a DLL -- no problem at all.
Like I said, the overhead (when implemented properly) is neglegible, so running in the background is no issue. As for the DLL, you need to be careful that memory which you allocate stays on the same heap (i.e. do NOT allocate something in the main program and free in the DLL, or vice versa), but other than that I see no reason why there should be any issues.

Quote:
I'm not really concerned about any stalls
Stalls are the only thing you need to be concerned about, really. They're the only thing that may cause your program to become "jerky". The actual overhead is neglegible, it's only the stalls that matter -- but you can avoid them as described.

Share this post


Link to post
Share on other sites
My idea was to make it run in the background, and capture whatever data was being sent to the monitor. Basically whichever data was viewable on the monitor, for example the windows screen with a web browser running or Word or whatever.

I'm not really intending to record any particular foreign program, only what I mention above as whatever is displayed on-screen. Basically I want to copy some of the functionality of the video capture tools like Fraps, but without having to create a UI and all of the other stuff.

Main goal of this app/DLL is to have it running and be able to provide a stream of frames representing what is being displayed on the monitor.

When you mean without impacting my game, are you talking about running a DLL in the background of the OS, and having it not impact the framerate of a separate game application?

Share this post


Link to post
Share on other sites
If you want to capture whatever data goes to the monitor, you don't want an OpenGL capture. Look at for example Greenshot to see how this can be done.

It's an entirely different thing, but yes, this can also be done in a background application.

Share this post


Link to post
Share on other sites
Quote:
Original post by samoth
If you want to capture whatever data goes to the monitor, you don't want an OpenGL capture. Look at for example Greenshot to see how this can be done.

It's an entirely different thing, but yes, this can also be done in a background application.


Hmm wonder if however it gets the screens would work for what I'm planning on doing? Haven't gotten the source yet though, so I can check it out.

Share this post


Link to post
Share on other sites
I got the Greenshot code just now and looked at some of the Capture code.

There is the main part of it that uses Win32 and BitBlt(). I remember seeing somewhere that BitBlt wasn't the best to use if you wanted quick access to framebuffers or something like that. If I find the page, I will try to post it. Most of the code though looks pretty straightforward for taking a screenshot. Lots of it is just the fancy GUI they've developed around it.

So this Greenshot solution could be a better one than one using OpenGL?

Share this post


Link to post
Share on other sites
For the "get whatever goes to the monitor" case, probably yes.
It is quite trivial to capture an OpenGL window (including a big window that covers the entire screen), but not nearly as trivial to capture the entire screen including other windows in the same way.

The BitBlt approach is by no means as efficient as a PBO readback, and it's hard to synchronize e.g. with a game that draws OpenGL (since unless vertical sync is on, you have no idea when new frames are ready).
However, it works for the "whatever goes to the monitor" case (with the exception of hardware accelerated video and webcam windows, which somehow never seem to work properly and mess up with everything). So if you really want to capture everything that's on the screen, this is the way to go.

Share this post


Link to post
Share on other sites
So what I am hearing is that BitBlt is not going to be as efficient of a solution as using this PBO readback method. I was reading a little about it and saw that using the PBO method basically involves fast pixel data transfer to and from the graphics card using DMA, and not hogging a bunch of CPU cycles.

To me, this seems like the option I would want to pursue, since I am planning on running this program in the background while normal computer operation is going on. I don't want to look in the task manager and see my cpu usage pegging.

I don't have this real-time video thing planned so much for games at this moment, but it might come up in the future and it would be good to be prepared with an efficient design.

Share this post


Link to post
Share on other sites
I think you do not understand the difference, because you keep talking of two different things which are not easily exchangeable.

1. Capturing an OpenGL window (or several OpenGL windows, it does not matter). There is a context and a buffer that is drawn to (usually two, front and back), and you can read out that buffer asynchronously in an easy and efficient way using a pixel buffer object.

2. Caputring "what goes to the monitor". This is a totally different thing. For example, under MacOS X Quartz, or Metacity, or the respective Microsoft rip-off Aero, the image that goes to the monitor does not need to be much correlated to what you draw in a window.
Anything drawn inside windows goes to a non-visible buffer and is then transformed and composited in some way (3D transform, wiggly windows, transparency and fade effects, color grading, blur, ... whatever). There is no easy and obvious way (and probably no way at all!) to get to this buffer using OpenGL, as it isn't managed by OpenGL.
BitBlt on the other hand may not be nice and super efficient, but it can read from that buffer.

So, it really depends which of the two you want, but you need to understand that these are different things, and you can't just "mix" them.

Share this post


Link to post
Share on other sites
I think I see what you are saying samoth. I can't really use OpenGL here since the video that is shown on the monitor normally isn't created using OpenGL. For example, when I'm using Firefox or something, that's not OpenGL. It's the operation system basically right?

If I want to capture the screen when normally using a PC, I have to use the operating system. If I am in a game where the graphics are created using OpenGL, I could technically use the PBO readback method?

One question I have now, do you think it would be possible to do this whole screen-capture thing by interacting directly with the video driver? Don't video drivers have APIs written for them?

Share this post


Link to post
Share on other sites
The driver APIs that I'm familiar with let you query and change display settings, but to my knowledge have no documented way of accessing the raw framebuffer. Which of course doesn't mean that there might not be such a thing.
However, I'm not optimistic for you to find such a way, even less so one that works for three different vendors.

Share this post


Link to post
Share on other sites
Quote:
Original post by Decrius
Personally I like http://sourceforge.net/projects/taksi/, it's a free capturer, no limitations etc. (unlike FRAPS). You can also capture a part of the screen, pretty neat.

Taksi is nice and you can use third party codecs, so you can capture to lossless and then use maxed out x264 encoder settings to get high quality recordings in small size (I use Lagarith codec in Taksi and then MeGui for conversion to mp4 with ludicrous settings). However, Taksi has a big disadvantage: no audio capture, which may or may not be an issue for you. In my case, if a client wants a complete preview with audio, I record and mux in the audio track separately, but if you need audio all the time, this is too much trouble and you should look at other solutions.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Similar Content

    • By xhcao
      Does sync be needed to read texture content after access texture image in compute shader?
      My simple code is as below,
      glUseProgram(program.get());
      glBindImageTexture(0, texture[0], 0, GL_FALSE, 3, GL_READ_ONLY, GL_R32UI);
      glBindImageTexture(1, texture[1], 0, GL_FALSE, 4, GL_WRITE_ONLY, GL_R32UI);
      glDispatchCompute(1, 1, 1);
      // Does sync be needed here?
      glUseProgram(0);
      glBindFramebuffer(GL_READ_FRAMEBUFFER, framebuffer);
      glFramebufferTexture2D(GL_READ_FRAMEBUFFER, GL_COLOR_ATTACHMENT0,
                                     GL_TEXTURE_CUBE_MAP_POSITIVE_X + face, texture[1], 0);
      glReadPixels(0, 0, kWidth, kHeight, GL_RED_INTEGER, GL_UNSIGNED_INT, outputValues);
       
      Compute shader is very simple, imageLoad content from texture[0], and imageStore content to texture[1]. Does need to sync after dispatchCompute?
    • By Jonathan2006
      My question: is it possible to transform multiple angular velocities so that they can be reinserted as one? My research is below:
      // This works quat quaternion1 = GEQuaternionFromAngleRadians(angleRadiansVector1); quat quaternion2 = GEMultiplyQuaternions(quaternion1, GEQuaternionFromAngleRadians(angleRadiansVector2)); quat quaternion3 = GEMultiplyQuaternions(quaternion2, GEQuaternionFromAngleRadians(angleRadiansVector3)); glMultMatrixf(GEMat4FromQuaternion(quaternion3).array); // The first two work fine but not the third. Why? quat quaternion1 = GEQuaternionFromAngleRadians(angleRadiansVector1); vec3 vector1 = GETransformQuaternionAndVector(quaternion1, angularVelocity1); quat quaternion2 = GEQuaternionFromAngleRadians(angleRadiansVector2); vec3 vector2 = GETransformQuaternionAndVector(quaternion2, angularVelocity2); // This doesn't work //quat quaternion3 = GEQuaternionFromAngleRadians(angleRadiansVector3); //vec3 vector3 = GETransformQuaternionAndVector(quaternion3, angularVelocity3); vec3 angleVelocity = GEAddVectors(vector1, vector2); // Does not work: vec3 angleVelocity = GEAddVectors(vector1, GEAddVectors(vector2, vector3)); static vec3 angleRadiansVector; vec3 angularAcceleration = GESetVector(0.0, 0.0, 0.0); // Sending it through one angular velocity later in my motion engine angleVelocity = GEAddVectors(angleVelocity, GEMultiplyVectorAndScalar(angularAcceleration, timeStep)); angleRadiansVector = GEAddVectors(angleRadiansVector, GEMultiplyVectorAndScalar(angleVelocity, timeStep)); glMultMatrixf(GEMat4FromEulerAngle(angleRadiansVector).array); Also how do I combine multiple angularAcceleration variables? Is there an easier way to transform the angular values?
    • By dpadam450
      I have this code below in both my vertex and fragment shader, however when I request glGetUniformLocation("Lights[0].diffuse") or "Lights[0].attenuation", it returns -1. It will only give me a valid uniform location if I actually use the diffuse/attenuation variables in the VERTEX shader. Because I use position in the vertex shader, it always returns a valid uniform location. I've read that I can share uniforms across both vertex and fragment, but I'm confused what this is even compiling to if this is the case.
       
      #define NUM_LIGHTS 2
      struct Light
      {
          vec3 position;
          vec3 diffuse;
          float attenuation;
      };
      uniform Light Lights[NUM_LIGHTS];
       
       
    • By pr033r
      Hello,
      I have a Bachelor project on topic "Implenet 3D Boid's algorithm in OpenGL". All OpenGL issues works fine for me, all rendering etc. But when I started implement the boid's algorithm it was getting worse and worse. I read article (http://natureofcode.com/book/chapter-6-autonomous-agents/) inspirate from another code (here: https://github.com/jyanar/Boids/tree/master/src) but it still doesn't work like in tutorials and videos. For example the main problem: when I apply Cohesion (one of three main laws of boids) it makes some "cycling knot". Second, when some flock touch to another it scary change the coordination or respawn in origin (x: 0, y:0. z:0). Just some streng things. 
      I followed many tutorials, change a try everything but it isn't so smooth, without lags like in another videos. I really need your help. 
      My code (optimalizing branch): https://github.com/pr033r/BachelorProject/tree/Optimalizing
      Exe file (if you want to look) and models folder (for those who will download the sources):
      http://leteckaposta.cz/367190436
      Thanks for any help...

    • By Andrija
      I am currently trying to implement shadow mapping into my project , but although i can render my depth map to the screen and it looks okay , when i sample it with shadowCoords there is no shadow.
      Here is my light space matrix calculation
      mat4x4 lightViewMatrix; vec3 sun_pos = {SUN_OFFSET * the_sun->direction[0], SUN_OFFSET * the_sun->direction[1], SUN_OFFSET * the_sun->direction[2]}; mat4x4_look_at(lightViewMatrix,sun_pos,player->pos,up); mat4x4_mul(lightSpaceMatrix,lightProjMatrix,lightViewMatrix); I will tweak the values for the size and frustum of the shadow map, but for now i just want to draw shadows around the player position
      the_sun->direction is a normalized vector so i multiply it by a constant to get the position.
      player->pos is the camera position in world space
      the light projection matrix is calculated like this:
      mat4x4_ortho(lightProjMatrix,-SHADOW_FAR,SHADOW_FAR,-SHADOW_FAR,SHADOW_FAR,NEAR,SHADOW_FAR); Shadow vertex shader:
      uniform mat4 light_space_matrix; void main() { gl_Position = light_space_matrix * transfMatrix * vec4(position, 1.0f); } Shadow fragment shader:
      out float fragDepth; void main() { fragDepth = gl_FragCoord.z; } I am using deferred rendering so i have all my world positions in the g_positions buffer
      My shadow calculation in the deferred fragment shader:
      float get_shadow_fac(vec4 light_space_pos) { vec3 shadow_coords = light_space_pos.xyz / light_space_pos.w; shadow_coords = shadow_coords * 0.5 + 0.5; float closest_depth = texture(shadow_map, shadow_coords.xy).r; float current_depth = shadow_coords.z; float shadow_fac = 1.0; if(closest_depth < current_depth) shadow_fac = 0.5; return shadow_fac; } I call the function like this:
      get_shadow_fac(light_space_matrix * vec4(position,1.0)); Where position is the value i got from sampling the g_position buffer
      Here is my depth texture (i know it will produce low quality shadows but i just want to get it working for now):
      sorry because of the compression , the black smudges are trees ... https://i.stack.imgur.com/T43aK.jpg
      EDIT: Depth texture attachment:
      glTexImage2D(GL_TEXTURE_2D, 0,GL_DEPTH_COMPONENT24,fbo->width,fbo->height,0,GL_DEPTH_COMPONENT,GL_FLOAT,NULL); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); glFramebufferTexture2D(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_TEXTURE_2D, fbo->depthTexture, 0);
  • Popular Now