Yes, this special case will not be culled. A box needs to be outside one of the clipping planes/lines in its entirety to be able to be culled.
The reason this is not such a big deal is the further away the box from the camera is the larger it needs to be to satisfy the above "unclipable" conditions.
Likely, in practice only a few objects will fail this way, so it's often not worth it to write a more expensive test.
If you do want 100% correct results, the test would essentially become a convex overlap test. A separating-axis test can solve this.