To dramatically and hideously oversimplify the Wikipedia article on vacuum energy:
First, you have the energy/time uncertainty principle. To totally rape the mathematics, this states that as time intervals grow shorter, the ability to precisely measure the energy level change of a field diminishes. In other words, as you move interactions towards Planck time scales, you can no longer reliably track the energy of a field. Shorter than the Planck time, all bets are off; you can no longer define the energy level at all. This means that on ultra-brief time scales, energy is more or less doing totally unpredictable things.
This is generalized in particle mechanics as the concept of "virtual particles." Virtual particles are, to again totally molest the mathematics, just particles that exist for such brief fractions of the Planck time that their properties are utterly unmeasurable. (Anything that survives long enough or exists with enough energy to interact with anything measurably is no longer virtual.) A virtual particle exists and then stops existing. One theoretical mechanism for this is particle/anti-particle pairings, where a virtual particle spawns near its virtual antiparticle, the two combine, and then resolve back into component energy via annihilation. I don't know if there are any measured results that indicate that this is going on, but I certainly remember it being one of the most popular explanations of virtual particle interaction.
There's a ton more to this and I'm obliterating a lot of relevant facts in trying to condense the information. Quantum mechanics is notoriously sticky stuff - the kind of thing that some of the world's most brilliant minds have had trouble comprehending - so you'll have to forgive me if (1) something got lost in my own learning of this material and/or (2) I've totally left out something cool and/or important.
Anyways, the point is, because of quantization of energy fields (i.e. you can't have any arbitrary real number value for an energy level, only multiples of the relevant fundamental charge constant) and the uncertainty principles, energy is never zero, even in a totally empty universe.
You are right based on the standard model of physics but as we all know it is not a complete theory. When they talk about virtual particles and the uncertainty principle I am pretty sure these are just rules set in to place to make the standard model of physics work.
What I am saying is obviously wrong based on the standard model of physics but I am looking for more of an contradiction to known facts. An example of this would be "If that were the case then running a test for zero point energy in space would likely return lower levels than on earth." I then would agree but I would question if the difference in levels would be measurable.