At 640x480 you can probably do this in realtime without needing a PBO. Even if not, I'd advise that you go about it in the following order:
1) Write the basic version that just updates and displays the texture without anything extra.
2) Add double-buffering to it, using two textures; update texture 0/display texture 1, then swap for the next frame.
3) Add a PBO.
Only go to the next step here if the step you're currently on proves too slow for your needs.
GPUs have an annoying tendency to prefer texture data in 1, 2 or 4 byte formats, whereas content deliverers have an annoying tendency to prefer texture data in 3 byte formats so there is no fast way to get 24-bit RGB data into a texture. You should expand (and swap) it to BGRA at some stage in the process, yes, and this should preferably happen during decompression from source.
For your colour data, create your texture one-time-only as follows:
You'll see that I'm using glTexStorage2D here rather than glTexImage2D - at this stage we just want to allocate storage for the texture and we're not yet concerned about what data is actually going into it.
Each time you get new data in, update the texture as follows:
The key parameters here are the third and second last ones. You can write a small program to verify this yourself, but the basic summary is that it is absolutely essential that you match these with what the OpenGL driver is going to prefer, otherwise you're going to get nasty slowdowns and this will be irrespective of whether you use a PBO or double-buffer. I've benchmarked upload performance increases of over 25x (versus GL_RGB/GL_UNSIGNED_BYTE) on some hardware from these two parameters alone. So get these correct first, then use other methods to make it faster, but only if you need them.
If your data is coming in at 24-bit RGB and if you don't have control over this, then you should expand and swap yourself; don't rely on the driver to do it for you (by e.g. using GL_RGB/GL_UNSIGNED_BYTE). Expand and swap to a pre-allocated (or static) buffer and then glTexSubImage it and it will still be faster.
For RGB(A) data, this particular combination of format and type parameters should be the fastest on most systems but may not be the fastest on all. For traditional desktop GL (which I'm assuming you're targetting based on your mention of 4.2) you should be safe enough, but as always, benchmark and find out for yourself.
For greyscale data you should be good enough just using a greyscale format for your texture; if you don't want to create a second texture then you can also expand it, but greyscale formats are still fine and fast - the only tricky cases are around 24-bit RGB data formats.
Finally, and if the video is also displayed at 640x480, don't underestimate the power of glDrawPixels. The same trickiness around 24-bit RGB also applies here, but it may well work just fine for you.