Sign in to follow this  
Knuckler

OpenGL OpenGL AVI class

Recommended Posts

Knuckler    144
I made an OpenGL AVI player and am sharing it with you guys and gals. This is one of the things that have always seemed elusive to OpenGL programmers; at the least being able to have audio and video in sync. It's based on the nehe playing and loading code. A major downside is that it has a lot of dependencies and requirements.

Its all within the bj namespace.

It uses for libraries:
VFW, vfw32.lib / vfw32.dll Already suppied by windows

FMOD, fmodex_vc.lib If you don't need audio, you can comment out the process_audio and init_fmod functions in the Movie constructor

GLEW, glew32.lib This is for OpenGL. You can replace this with GLEE.h or glext.h. Just make sure there is some OpenGL header that supplies you with the declarations, and that there is already a rendering context created. You also need to have a system capable of using PBOs; basically opengl 2.0

Timer.hpp/cpp - A personal timer class based on the QueryPerformance functions.

NOTE: The audio for the AVI must be in PCM16 format. I'm sorry. AVIStreamRead will not read non-pcm/wma audio. Nor is there any way to supply fmod with .wma formatted audio, that I can see.

Movie class methods

play() - plays video

stop() - stops video

bind_texture() - binds the texture of the current frame being displayed
is_playing() - returns true if movie is playing, false otherwise

update() - updates the texture, and audio system. This needs to be called at the beginning of every cycle.

seek_forward() - fast forwards in increments of seekMilliseconds. seekMilliseconds is the second parameter of the Movie constructor, after the file name. The default is 1000 ms or 1 second.

seek_reverse() - seeks backwards in increments of seekMilliseconds.

seek_to_position() - seeks to a position specified in milliseconds from the start of the video.

mute() - sets volume to 0%

unmute() - sets volume to 100%

Here are the classes
Movie.hpp

#ifndef MOVIE_HPP
#define MOVIE_HPP

#ifdef _MSC_VER // use this to include specific libraries with msvc
#pragma comment(lib, "vfw32.lib")
#pragma comment(lib, "fmodex_vc.lib")
#endif // _MSC_VER

#include <Windows.h>
#include <Vfw.h>
#include <fmod.hpp>
#include <string>
#include <vector>
#include "MyTimer.hpp"

namespace bj
{
enum PLAY_STATE
{
PLAYING,
STOPPED
};

class Movie
{
public:
Movie(const std::string fileName_, DWORD seekInMilliseconds = 1000);
~Movie();
void play(); // start updating the texture and audio
void stop(); // Stop and cycle to beginning
void bind_texture();
bool is_playing()const; // Check play state
void update(); // must be called for every cycle
void seek_forward();
void seek_reverse();
void seek_to_position(DWORD positionInMs);
void mute();
void unmute();

private:
static int movieCount; // For VFW AVI library initialization
PLAY_STATE playState;

// For AVI itself
std::string fileName;
PAVIFILE aviFile;
AVIFILEINFO aviFileInfo;
bool hasAudio; // Has audio and or video
bool hasVideo;

// audio
WAVEFORMATEX waveFormat;
LONG audioBufferSize;
FMOD::System *audioSystem; // audio system
FMOD::Sound *audio;
FMOD::Channel *channel;
PAVISTREAM audioStream;
AVISTREAMINFO audioStreamInfo;
std::vector<char> audioBuffer;
long samplesPerSecond;

// video
unsigned int pbo;
int dataSize;
unsigned int texture; // Assumes opengl context has already been created
double nextFrame;
DWORD currentFrame;
DWORD lastFrame;
PAVISTREAM videoStream;
AVISTREAMINFO videoStreamInfo;
PGETFRAME frameDecompressor;
DWORD width; // width and height of video
DWORD height;
DWORD bytesPerPixel;
double mpf; // length for each frame to be displayed in milliseconds

// For play back
bj::Timer timer;
DWORD seekMilliseconds; // Advance or reverse playback position in milliseconds

void process_avi();
void process_audio();
void init_fmod();
void process_video();
void init_opengl();
void grab_frame();
};
}
#endif





Movie.cpp

#include "Movie.hpp"
#include <stdexcept>
#include <string>
#include <fmod.hpp>
#include <sources/glew.h>

// Initialization of static member
int bj::Movie::movieCount = 0;

using namespace bj;

Movie::Movie(const std::string fileName_, DWORD seekInMilliseconds):
playState(STOPPED),
fileName(fileName_),
aviFile(nullptr),
aviFileInfo(),
hasAudio(false),
hasVideo(false),
waveFormat(),
audioBufferSize(0),
audioSystem(nullptr),
audio(nullptr),
audioStream(nullptr),
audioStreamInfo(),
audioBuffer(),
samplesPerSecond(0),
pbo(0),
dataSize(0),
texture(0),
nextFrame(0),
currentFrame(0),
lastFrame(0),
videoStream(nullptr),
videoStreamInfo(),
frameDecompressor(nullptr),
width(0),
height(0),
bytesPerPixel(0),
mpf(0.0),
seekMilliseconds(seekInMilliseconds)
{
// Increment the movieCount. If movieCount == 1, this is the first time it's been initialized
// for this program. Initialize the AVI library
movieCount++;

if(1 == movieCount)
AVIFileInit();

process_avi();
process_audio();
init_fmod();
process_video();
init_opengl();
}

Movie::~Movie()
{
if(aviFile)
AVIFileClose(aviFile);

if(audioStream)
AVIStreamRelease(audioStream);

if(videoStream)
AVIStreamRelease(videoStream);

if(frameDecompressor)
AVIStreamGetFrameClose(frameDecompressor);

if(audio)
audio->release();

if(audioSystem)
audioSystem->close();

if(pbo)
glDeleteBuffers(1, &pbo);

if(texture)
glDeleteTextures(1, &texture);

movieCount--;
// If movieCount == 0, we're finished with the library. Deintialize the library
if(0 == movieCount)
AVIFileExit();
}

void Movie::process_avi()
{
HRESULT error = AVIFileOpenA(&aviFile, fileName.c_str(), OF_READ, nullptr);

if(error)
{
if(AVIERR_FILEOPEN == error)
throw(std::logic_error("Could not open file " + fileName));
else
throw(std::logic_error("Problem with opening file"));
}

AVIFileInfo(aviFile, &aviFileInfo, sizeof(aviFileInfo));
}
void Movie::process_audio()
{
// Check if there is a stream
LONGLONG error = AVIFileGetStream(aviFile, &audioStream, streamtypeAUDIO, 0);

if(AVIERR_NODATA == error)
{
// No audio stream in this file
return;
}
else
{
if(0 != error)
{
// Problem loading audio stream
throw(std::logic_error("Problem loading audio stream"));
}
}

// everything went fine getting the stream
AVIStreamInfo(audioStream, &audioStreamInfo, sizeof(AVISTREAMINFO));

samplesPerSecond = audioStreamInfo.dwRate / audioStreamInfo.dwScale;
float lengthInSeconds = (float)audioStreamInfo.dwLength * audioStreamInfo.dwScale /
audioStreamInfo.dwRate;
float bufferSize = lengthInSeconds * samplesPerSecond;

// Read the stream format

error = AVIStreamReadFormat(audioStream, AVIStreamStart(audioStream), nullptr, &audioBufferSize);

if(error)
{
throw(std::runtime_error("Could not read audio stream format"));
}

std::vector<unsigned char> audioChunk;
audioChunk.resize(audioBufferSize);
AVIStreamReadFormat(audioStream, AVIStreamStart(audioStream), &audioChunk.front(), &audioBufferSize);

memcpy(&waveFormat, &audioChunk.front(), audioChunk.size());

// Read the audio data
audioBufferSize = 0;
error = AVIStreamRead(audioStream, AVIStreamStart(audioStream), (LONG)bufferSize, nullptr, 0, &audioBufferSize, nullptr);

if(AVIERR_FILEREAD == error)
{
throw(std::runtime_error("File error"));
}

if(AVIERR_BUFFERTOOSMALL == error)
{
throw(std::runtime_error("Buffer too small"));
}

if(AVIERR_MEMORY == error)
{
throw(std::runtime_error("Not enough memory to complete operation"));
}

if(error)
{
throw(std::runtime_error("Problem reading stream data"));
}

audioBuffer.resize(audioBufferSize, 0);
LONG bytesRead = 0;
LONG samplesRead = 0;
error = AVIStreamRead(audioStream, AVIStreamStart(audioStream), (LONG)bufferSize, &audioBuffer.front(), audioBufferSize, &bytesRead , &samplesRead);

if(error)
{
throw(std::runtime_error("Problem reading stream data"));
}

}

void Movie::init_fmod()
{
// Check to make sure that there is audio
if(audioBuffer.empty())
return; // No audio

FMOD_RESULT result = FMOD_OK;

result = FMOD::System_Create(&audioSystem);

if(FMOD_OK != result)
{
throw(std::runtime_error("Could not create FMOD system"));
}

result = audioSystem->init(1, FMOD_INIT_NORMAL, nullptr);

if(result != FMOD_OK)
{
throw(std::runtime_error("Could not initialize FMOD system"));
}

// For now, PCM 16 is only supported
FMOD_CREATESOUNDEXINFO exinfo;

memset(&exinfo, 0, sizeof(FMOD_CREATESOUNDEXINFO));
exinfo.cbsize = sizeof(FMOD_CREATESOUNDEXINFO);
exinfo.length = audioBufferSize;
exinfo.format = FMOD_SOUND_FORMAT_PCM16; // Must specify for FMOD_OPENRAW
exinfo.defaultfrequency = waveFormat.nSamplesPerSec; // Must specify for FMOD_OPENRAW
exinfo.numchannels = waveFormat.nChannels; // Must specify for FMOD_OPENRAW

result = audioSystem->createSound(&audioBuffer.front(), FMOD_OPENRAW | FMOD_OPENMEMORY,
&exinfo, &audio);

// Get rid of excess memory
audioBuffer.swap(std::vector<char>());

if(FMOD_OK != result)
{
throw(std::runtime_error("Could not load sound"));
}

// success
hasAudio = true;
}

void Movie::process_video()
{
HRESULT error = AVIFileGetStream(aviFile, &videoStream, streamtypeVIDEO, 0);

if(error)
{
throw(std::runtime_error("Error getting video stream"));
}

// Get the stream info
error = AVIStreamInfo(videoStream, &videoStreamInfo, sizeof(videoStreamInfo));

if(error)
{
throw(std::runtime_error("Error getting video stream information"));
}

// Final frame
lastFrame = AVIStreamLength(videoStream);

// Time to display each frame
mpf = (double)AVIStreamSampleToTime(videoStream, lastFrame) / (double)lastFrame;

// Get decompressor
frameDecompressor = AVIStreamGetFrameOpen(videoStream, nullptr);

if(nullptr == frameDecompressor)
{
throw (std::runtime_error("Could not get decompressor"));
}
}

void Movie::init_opengl()
{
// Get video properties

LPBITMAPINFOHEADER lpbi = (LPBITMAPINFOHEADER)AVIStreamGetFrame(frameDecompressor, 0);

// width, height, bit count
width = lpbi->biWidth;
height = lpbi->biHeight;
bytesPerPixel = lpbi->biBitCount / 8;
dataSize = width * height * bytesPerPixel;

glGenBuffers(1, &pbo);
glBindBuffer(GL_PIXEL_UNPACK_BUFFER, pbo);
glBufferData(GL_PIXEL_UNPACK_BUFFER, dataSize, nullptr, GL_STREAM_DRAW);

glGenTextures(1, &texture);
glBindTexture(GL_TEXTURE_2D, texture);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);

GLint format = bytesPerPixel == 3 ? GL_RGB : GL_RGBA;
glTexImage2D(GL_TEXTURE_2D, 0, format, width, height, 0, GL_BGR, GL_UNSIGNED_BYTE, nullptr);

// unbind
glBindBuffer(GL_PIXEL_UNPACK_BUFFER, 0);
hasVideo = true;
}

void Movie::grab_frame()
{
static DWORD previousFrame = 0;

if((previousFrame != currentFrame) || (0 == currentFrame))
{
// Update the texture only if the previous frame does not equal the current frame or
// it's the first frame (frame #0)
LPBITMAPINFOHEADER lpbi = (LPBITMAPINFOHEADER)AVIStreamGetFrame(frameDecompressor, currentFrame);
// Get pointer to the data
char *data = (char*)lpbi + lpbi->biSize + (lpbi->biClrUsed * sizeof(RGBQUAD));

// Update the pbo and texture
glBindBuffer(GL_PIXEL_UNPACK_BUFFER, pbo);
glBufferSubData(GL_PIXEL_UNPACK_BUFFER, 0, dataSize, data);

previousFrame = currentFrame;
}
}

void Movie::play()
{
if(STOPPED == playState)
{
// Start audio first
audioSystem->playSound(FMOD_CHANNEL_FREE, audio, false, &channel);
timer.start();
playState = PLAYING;
}
}

void Movie::stop()
{
if(PLAYING == playState)
{
if(hasAudio)
channel->stop();

playState = STOPPED;

currentFrame = 0;
nextFrame = 0;
}
}

void Movie::bind_texture()
{
// TODO previousFrame
static DWORD previousFrame = 0;

if((previousFrame != currentFrame) || (0 == currentFrame))
{
glBindBuffer(GL_PIXEL_UNPACK_BUFFER, pbo);
glBindTexture(GL_TEXTURE_2D, texture);
GLint format = bytesPerPixel == 3 ? GL_RGB : GL_RGBA;
glTexImage2D(GL_TEXTURE_2D, 0, format, width, height, 0, GL_BGR, GL_UNSIGNED_BYTE, nullptr);
glBindBuffer(GL_PIXEL_UNPACK_BUFFER, 0);
previousFrame = currentFrame;
}
else
{
glBindTexture(GL_TEXTURE_2D, texture);
}
}

bool Movie::is_playing()const
{
return (PLAYING == playState);
}

void Movie::update()
{
if(hasAudio)
audioSystem->update();

if(hasVideo && (PLAYING == playState))
{
nextFrame += timer.milliseconds();
currentFrame = (DWORD)(nextFrame / mpf);

if(currentFrame >= lastFrame)
{
currentFrame = 0;
nextFrame = 0;
playState = STOPPED;
}

grab_frame();
}
}

void Movie::seek_forward()
{
if(PLAYING == playState)
{
long sample = AVIStreamTimeToSample(videoStream, (LONG)nextFrame + seekMilliseconds);

nextFrame = AVIStreamSampleToTime(videoStream, sample);


// reset
timer.milliseconds();

if(hasAudio)
channel->setPosition((unsigned int)nextFrame, FMOD_TIMEUNIT_MS);

currentFrame = DWORD(nextFrame / mpf);


if(currentFrame <= 0 || (currentFrame >= lastFrame))
{
channel->stop();
currentFrame = 0;
playState = STOPPED;
nextFrame = 0;
}
}
}

void Movie::seek_reverse()
{
if(PLAYING == playState)
{
long sample = AVIStreamTimeToSample(videoStream, (LONG)nextFrame - seekMilliseconds);

if(-1 == sample)
{
// probably seeked before initial position
nextFrame = 0;
if(hasAudio)
channel->setPosition(0, FMOD_TIMEUNIT_MS);
timer.milliseconds();
return;
}

nextFrame = AVIStreamSampleToTime(videoStream, sample);
if(hasAudio)
channel->setPosition((unsigned int)nextFrame, FMOD_TIMEUNIT_MS);
// reset
timer.milliseconds();

currentFrame = DWORD(nextFrame / mpf);
}
}

void Movie::seek_to_position(DWORD positionInMs)
{
if(PLAYING == playState)
{
long sample = AVIStreamTimeToSample(videoStream, positionInMs);

if(-1 == sample)
{
// Error, do nothing or throw something
return;
}

nextFrame = AVIStreamSampleToTime(videoStream, sample);

if(hasAudio)
channel->setPosition((unsigned int)nextFrame, FMOD_TIMEUNIT_MS);
//reset
timer.milliseconds();

currentFrame = DWORD(nextFrame / mpf);
}
}

void Movie::mute()
{
if(PLAYING == playState && hasAudio)
{
channel->setMute(true);
}
}

void Movie::unmute()
{
if(PLAYING == playState && hasAudio)
{
channel->setMute(false);
}
}





MyTimer.hpp

#ifndef MYTIMER_HPP
#define MYTIMER_HPP

#include <Windows.h>

namespace bj
{

class Timer
{
public:
Timer();


void start(); // start or reset timer
double seconds(); // returns seconds since last time seconds or milliseconds has be called
double milliseconds(); // returns seconds since last time seconds or milliseconds has be called
double delta_from_start()const; // Returns

enum MODE {STOPPED, RUNNING, PAUSED};

private:
void set_state(MODE m);
MODE get_state()const;
LARGE_INTEGER startTime;
LARGE_INTEGER currentTime;
LARGE_INTEGER frequency;
MODE mode;

};
}





Timer.cpp

#include "MyTimer.hpp"

using namespace bj;

Timer::Timer():
mode(STOPPED)
{
QueryPerformanceFrequency(&frequency);
}

void Timer::start()
{
QueryPerformanceCounter(&currentTime);
startTime = currentTime;
set_state(RUNNING);
}



double Timer::seconds()
{
LARGE_INTEGER newTime;
QueryPerformanceCounter(&newTime);

double seconds = ((double)(newTime.QuadPart - currentTime.QuadPart)) / frequency.QuadPart;
currentTime = newTime;
return (seconds);
}

double Timer::milliseconds()
{
LARGE_INTEGER newTime;
double milliseconds;

QueryPerformanceCounter(&newTime);
milliseconds = (((double)(newTime.QuadPart - currentTime.QuadPart) / frequency.QuadPart)) * 1000.0;
currentTime = newTime;
return (milliseconds);
}

double Timer::delta_from_start()const
{
LARGE_INTEGER newTime;
QueryPerformanceCounter(&newTime);

return (((double)(newTime.QuadPart - startTime.QuadPart)) / frequency.QuadPart);
}

void Timer::set_state(MODE m)
{
mode = m;
}

Timer::MODE Timer::get_state()const
{
return (mode);
}





I hope this serves you well.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this  

  • Partner Spotlight

  • Similar Content

    • By pseudomarvin
      I assumed that if a shader is computationally expensive then the execution is just slower. But running the following GLSL FS instead just crashes
      void main() { float x = 0; float y = 0; int sum = 0; for (float x = 0; x < 10; x += 0.00005) { for (float y = 0; y < 10; y += 0.00005) { sum++; } } fragColor = vec4(1, 1, 1 , 1.0); } with unhandled exception in nvoglv32.dll. Are there any hard limits on the number of steps/time that a shader can take before it is shut down? I was thinking about implementing some time intensive computation in shaders where it would take on the order of seconds to compute a frame, is that possible? Thanks.
    • By Arulbabu Donbosco
      There are studios selling applications which is just copying any 3Dgraphic content and regenerating into another new window. especially for CAVE Virtual reality experience. so that the user opens REvite or CAD or any other 3D applications and opens a model. then when the user selects the rendered window the VR application copies the 3D model information from the OpenGL window. 
      I got the clue that the VR application replaces the windows opengl32.dll file. how this is possible ... how can we copy the 3d content from the current OpenGL window.
      anyone, please help me .. how to go further... to create an application like VR CAVE. 
       
      Thanks
    • By cebugdev
      hi all,

      i am trying to build an OpenGL 2D GUI system, (yeah yeah, i know i should not be re inventing the wheel, but this is for educational and some other purpose only),
      i have built GUI system before using 2D systems such as that of HTML/JS canvas, but in 2D system, i can directly match a mouse coordinates to the actual graphic coordinates with additional computation for screen size/ratio/scale ofcourse.
      now i want to port it to OpenGL, i know that to render a 2D object in OpenGL we specify coordiantes in Clip space or use the orthographic projection, now heres what i need help about.
      1. what is the right way of rendering the GUI? is it thru drawing in clip space or switching to ortho projection?
      2. from screen coordinates (top left is 0,0 nd bottom right is width height), how can i map the mouse coordinates to OpenGL 2D so that mouse events such as button click works? In consideration ofcourse to the current screen/size dimension.
      3. when let say if the screen size/dimension is different, how to handle this? in my previous javascript 2D engine using canvas, i just have my working coordinates and then just perform the bitblk or copying my working canvas to screen canvas and scale the mouse coordinates from there, in OpenGL how to work on a multiple screen sizes (more like an OpenGL ES question).
      lastly, if you guys know any books, resources, links or tutorials that handle or discuss this, i found one with marekknows opengl game engine website but its not free,
      Just let me know. Did not have any luck finding resource in google for writing our own OpenGL GUI framework.
      IF there are no any available online, just let me know, what things do i need to look into for OpenGL and i will study them one by one to make it work.
      thank you, and looking forward to positive replies.
    • By fllwr0491
      I have a few beginner questions about tesselation that I really have no clue.
      The opengl wiki doesn't seem to talk anything about the details.
       
      What is the relationship between TCS layout out and TES layout in?
      How does the tesselator know how control points are organized?
          e.g. If TES input requests triangles, but TCS can output N vertices.
             What happens in this case?
      In this article,
      http://www.informit.com/articles/article.aspx?p=2120983
      the isoline example TCS out=4, but TES in=isoline.
      And gl_TessCoord is only a single one.
      So which ones are the control points?
      How are tesselator building primitives?
    • By Orella
      I've been developing a 2D Engine using SFML + ImGui.
      Here you can see an image
      The editor is rendered using ImGui and the scene window is a sf::RenderTexture where I draw the GameObjects and then is converted to ImGui::Image to render it in the editor.
      Now I need to create a 3D Engine during this year in my Bachelor Degree but using SDL2 + ImGui and I want to recreate what I did with the 2D Engine. 
      I've managed to render the editor like I did in the 2D Engine using this example that comes with ImGui. 
      3D Editor preview
      But I don't know how to create an equivalent of sf::RenderTexture in SDL2, so I can draw the 3D scene there and convert it to ImGui::Image to show it in the editor.
      If you can provide code will be better. And if you want me to provide any specific code tell me.
      Thanks!
  • Popular Now