I've been wanting to delve back into audio/video playback for a while now, and I think the 3 libs mentioned above seem like a good place to start. How do these three libraries work together, though? I did some research, and it looks like libogg is used only to read ogg files containing vorbis and theora packet streams into basic structs containing the encoded data, and some info about the packet such as ID, data length, etc. It sounds like an ogg file can have up to 2 streams: one for vorbis (audio), and the other for theora (video). It seems that libvorbis and libtheora are used to decode their respective packets' audio/video data into raw, uncompressed data that could be sent to the hardware for rendering using a lower-level API, such as OpenAL (audio) and OpenGL (video).

That said, OGG is really a container format, and vorbis and theora files are actually specified with different file extensions. Vorbis files only contain an audio stream, and theora files contain a stream for video data with an optional stream for audio data. Does this sound correct so far?

It's possible to use vorbis encoding with other container formats. For example, a Matroska file could carry a vorbis video payload. Ogg can also contain other data, for example text streams, as well as other formats such as AVI video.

I thought that's what was going on. I've read about container formats over the years, but didn't really think it was that simple. It sounds like container formats are meant to break the file into chunks (packets) for streaming in either from a local disk to keep a low memory footprint, or from a network where large amounts of data can't flow all at once.

File extensions are irrelevant unless you're trying to write a trojan virus. Like most data files, a sequence of 'magic' bytes at the start of the file identify the actual container format, and the container indicates what data formats it contains. Consider the advantage of magic bytes in data streaming over a network connection: where do you put the file extension?

Well said! I also totally agree. Magic numbers can also help determine the endian-ness of the CPU relative to the CPU that wrote the data to the file as well, which can be handy. I've seen identifiers at the beginning of binary headings quite a bit, and they can be used to determine the endian-ness of the CPU reading the data in as well. For the past 8 years, I've thought of file extensions as another way for people to easily identify what type of file it is, and OS's help by associating file extensions with apps and type descriptions for people to make sense of, which helps the likeliness of trojans being opened.