do you have a depth map of the scene?, if so it's pretty trivial(although it also depends on the complexity of the 3D surface you want to create).
it's possible to do it with color images, but the results i've seen are used to re-construct entire city's with 1000's of photo's here
more information about what you have, and exactly what you want to accomplish would be helpful in pointing you in the right direction.