basically, video is like any other content. you'll have data on disk, and code to display it. the tricky part is that the code to display/playback is non-trivial. so you'll want to go with some type of turn-key library solution if possible. otherwise, its time to get out the white papers (the specs) and roll your own. another possibility is converting video to some animation format, and using an animation playback library. worst comes to worst, you can also screen capture frames and play them back as an animated texture - obviating the need for any playback library whatsoever.

One of the most common things to do is to use something like FFmpeg to decode the video/audio and then feed it into SDL to display/play. Be aware that FFmpeg has a bit of a steep learning curve.

Video is unfortunately pretty complicated, so getting something to play (without using a commercial library like Bink), so this may not be as trivial as you're hoping it to be.

From what I've gathered from researching video libraries and codecs there are also quite some nasty legal issues with using most popular video formats (patents and such). I know a lot of open source video playback applications have had legal problems because of this, so you'll probably want to research this very carefully or choose a commercial library.

I'm not an expert on video playback, nor am I a lawyer, but it's something you might want to keep in mind.

Yeah, patents can be problematic. I'd recommend using WebM/VPX (VP8/VP9 video codec, Vorbis/Opus audio codec). You can also use the WebM/VPX library instead of FFmpeg, if you want (FFmpeg is LGPL, which some people don't like/can't use, whereas the WebM project is BSD licensed). I've used FFmpeg, but I've never used the WebM/VPX libraries directly, so I can't comment on learning curve. While there's still a patent mine field to navigate, the WebM stuff is about as safe as you can get without forking over a bunch of money for patent licensing.

Thank you everyone for your feedback it helps. Does anyone know what video formats I could use that are not patented? I have no trouble researching it, but I am trying to save time. SInce this is a huge task for 1 coder.

Patents don't cover specific formats, they cover specific techniques, one codec can be protected by multiple patents and one of those patents might also cover parts of a completely different codec or even completely unrelated software.

Google won't sue you if you use their VP8 codec and the Ogg people won't sue you for using Theora so those codecs are reasonably safe but there is no guarantee that those codecs doesn't accidentally infringe on a patent held by some other company(Who might choose to sue you rather than the codec developer). (it happens quite often, especially in the US as their patent system is fubar)

I'm pretty sure OpenCV should be able to do what you want: http://www.cs.iit.edu/~agam/cs512/lect-notes/opencv-intro/index.html

You could also look at VLC which is open source to see how they do it: http://www.videolan.org/

I'm pretty sure OpenCV should be able to do what you want: http://www.cs.iit.edu/~agam/cs512/lect-notes/opencv-intro/index.html

OpenCV doesn't do video decoding. It can make use of FFmpeg, but it alone doesn't do the actual decoding.