I'm was struggling with this for a while. Simply said, I want to approximate the surface reflectance factor for each pixel from a single image. Let's have a look at this simple equation:

Where:

pi - Pixel intensity

sr - Surface reflectance at the current pixel

n, l - Self explanatory

Now, mathematically, it's not possible to recover sr just by having pi. So the only option here is to somehow statistically approximate it. How can I accomplish that? I mean, our brain does it everyday, so it's definitely possible to approximate it accurately. Does anybody have any ideas?

The surface normal can be retrieved from either a depth texture (if available) or approximated by gradient methods. The light direction will have to be inferred from surrounding context (specular lobes, shadows, are all good indicators of the direction of the light source - if you are outside, know your position, view direction, and the time of day/year, latitude and longitude, you can determine the Sun's position relative to you and work out the sunlight's direction). Once you have all that, you just calculate the dot product and solve for the reflectance. But getting the normal and light direction mechanically will probably require some serious computer vision algorithms if you want to automate it somehow.

Not to mention that you are completely ignoring the light's intensity (dependent on the light itself, and on its distance from the surface if it's not too far away). And, of course, this is valid only for a single light source. If you have multiple light sources, things get tricky.

Do you have examples of images you need to work on?

Well, the images I'm working on are illuminated by ambient lightning (most likely sky), so no shadows, and the lightning is really smooth. Here is a good example of an image I'd be working on:

When trying to approximate depth, the dark-is-deep approach doesn't work very well because of the variations in the surface reflectance value. That's why I'm trying to find the surface reflectance value. Diving the above image by its albedo part would result in pure lightning information.

So I was playing around with this normal vector reconstruction from a single image, and I must say I'm getting pretty impressive results.
Here's a normal map generated from the image from my previous post:

Another example of a generated normal map:

What do you think?

This looks quite convincing, but naturally there are still some artefacts related to what you called "dark-is-deep" -- white spots are given little hills that obviously aren't really there. A lot of the instances I've seen of this seem to roughly speaking construct a depth map from a sum of blurred layers of the image (often with what seems like some extra hand-wavy convolution applied at each step), and it looks like you're perhaps doing something similar?

I suspect a purely statistical approach based on windows of pixels might actually work pretty well, but I'm not sure if there's enough easily-available reference data to generate a model.

Yeah it's basically creating a pyramid of blurred greyscale input images, calculating normal maps from them (while treating them like casual depth maps), and adding them together while using some particular weight coefficients. They vary per image. But most of the time the lowest frequency band has the highest weight and it goes down, something like 1/f but not as sudden. While it works pretty good on most images, it doesn't work that good on images with sharp edges. Here's a good example:

As you can see, the cracked concrete at the top has obvious sharp edges, while my algorithm generates smooth edges. I'll be working on it.

Maybe try to use dilation and erosion for different layers of detail?

Maybe try to use dilation and erosion for different layers of detail?

What do you mean by erosion?

This looks quite convincing, but naturally there are still some artefacts related to what you called "dark-is-deep" -- white spots are given little hills that obviously aren't really there. A lot of the instances I've seen of this seem to roughly speaking construct a depth map from a sum of blurred layers of the image (often with what seems like some extra hand-wavy convolution applied at each step), and it looks like you're perhaps doing something similar?

I suspect a purely statistical approach based on windows of pixels might actually work pretty well, but I'm not sure if there's enough easily-available reference data to generate a model.

Also, I forgot to mention something. You were talking about how white spots are given little hills. I know it happens, and it happens because of the variation in the albedo of the surface. But that happens only if you work on a simple unprocessed image. If you were to work only on the lightning data, it would work pretty good. (It naturally happens that something that's deeper is darker, ambient occlusion etc, if you do some crazy math here you can actually recover the depth information pretty accurately)