I'm talking about this technique if you don't know about it. What I'm wondering though, is what if my scene is outdoors and I have cars and not just 4 building on each side of me, then I can't approximate the walls of my cube map and perform this ray to cube map type intersection. Has anyone tried this with a more unique world scene?

https://seblagarde.files.wordpress.com/2012/08/parallax_corrected_cubemap-siggraph2012.pdf

Any sort of complex environment is always going to be difficult to represent with simple proxy geo. Most games that use this technique just aim to capture to overall shape of the room or environment, and accept the error for everything. I don't think that I've ever seen anyone try to address the particular situation you've described, although it's possible that racing games having done something special for that case.

Are you actually trying to handle the case with the 4 cars, or are you just using that as a hypothetical scenario? If you're really set on trying to handle arbitrarily complex geometry and you have some memory and performance to spare, then you could try capturing depth along with color for each cubemap. You could then treat the depth as a heightfield, and raymarch through it to find the intersection point. Of course this will come with performance overhead, and you'll run into the typical undersampling artifacts.

The case I was trying to handle is say a rainy day in the middle of a 4 lane street. Assuming there is no moving geometry, but you have parked cars and what not in between your buildings, Then the original approach seems like it will look terrible with car reflections offset pretty badly.

For complex environments, I'm use signed distance fields as the proxy, and sphere-tracing through the field to find the intersection point, which is used for correcting the cube-map lookup direction.

Not sure what you mean by "fields".