The misunderstanding actually is with sRGB backbuffer. I thought that sRGB backbuffer is like JPEG in sRGB color space, meaning that all values in sRGB backbuffer are already Gamma Corrected (pow(value, 1/2.2)). If so, then final color values should outputted with pow(value, 1/2.2) correction

That's correct... but no correction is required to be performed when the buffer is outputted, because it's already been applied when the values were stored.

The display expect to receive data that has been encoded into sRGB format (approx pow(1/2.2)).
No matter what kind of back-buffer you use, the bytes in that buffer are sent to the display as-is.
JPEGs are stored with sRGB correction already applied, so JPEG data is sent to the display without any further modifications applied to the data.

If you're using an sRGB back-buffer, then when you draw into this back-buffer, the GPU automatically does the linear->sRGB conversion (pow(1/2.2)).
If you're not using an sRGB buffer, then you have to manually perform the pow(1/2.2) yourself in the shader code.

So:

• pixel shader outputs linear data to sRGB buffer -> data is converted using the linear-to-srgb curve -> sRGB(data) sent to the display -> user sees correct/linear result
• pixel shader outputs linear data to linear buffer -> data is not converted -> linear data sent to the display -> user sees incorrect result (~gamma 2.2 curve)
• pixel shader outputs gamma-corrected data to linear buffer -> data is not converted -> gamma-corrected data sent to the display -> user sees correct/linear result
• pixel shader outputs gamma-corrected data to sRGB buffer -> data is converted using the linear-to-srgb curve -> sRGB(sRGB(data)) sent to the display -> user sees incorrect result (~gamma 0.45 curve).

Either you output linear data to an sRGB buffer, which automatically applies sRGB gamma correction, and sends an sRGB signal to the display.
Or, you output manually perform sRGB correction yourself and send data to a regular buffer, which still results in an sRGB signal being sent to the display.

So, my question is why we need sRGB backbuffers plus modifying final output pixel shader if we can simply use non-sRGB texture?

You shouldn't do either of those things.
You should either use sRGB with no modification during output, or you should use non-sRGB with modification on output.
Your first, third and fourth images are all incorrect. The second image is correct.

The reason you think the first image is 'correct' is because "mathematically linear" is not the same as "perceptually linear". In order to perform correct lighting and shading calculations, or to be able to reproduce the same photograph that we captured earlier, we need all the data to be mathematically linear.

If you're painting pretty gradients, you don't care about maths and you just want it to look good, you care about perceptions, not mathematical correctness So your test, of painting a black/white gradient, is not a very good test for this purpose. It turns out that "gamma 2.2" is also a pretty good approximation of the human perception of brightness! So the non-linear "gamma 2.2" gradient is perceived as being fairly even, even though it's mathematically curved.

In modern games, this perceptual part is taken care of by the tone-mapping algorithm. All of the lighting algorithms still require mathematical linearity though.

I don't know if it's because of JPEG artifacts, or problems with your program, but none of your images are quite correct. When measured at equal points along the horizontal though, I got these intensity measurements. As you can see, the 2nd image is the closest match to a linear gradient

 0%, 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90%, 100% -- Ideal linear gradient
 0%,  2%,  6%, 12%, 20%, 30%, 40%, 53%, 67%, 83%,  99% -- image #1
 0%, 16%, 28%, 39%, 48%, 58%, 67%, 76%, 84%, 93%,  99% -- image #2
11%, 44%, 56%, 65%, 72%, 78%, 83%, 88%, 92%, 96%, 100% -- image #3
 0%,  1%,  5%, 11%, 20%, 30%, 42%, 55%, 69%, 84%,  99% -- image #4

Assuming that the textures were painted on an sRGB monitor, and you're viewing the results on an sRGB monitor, then yep.

n.b. the sharper termination line in the first sphere matches real life physics much more than the soft gradient in the second image too

Why the linear gradient that i can see on screen looks like non-linear with gamma correction (second case in my first post)?

The way that us humans perceive light is not linear.

e.g. If you're in a room lit by one light bulb, and you turn on a second bulb, physically/objectively/mathematically speaking the room is now twice as bright, but subjectively a person won't say that it looks twice as bright (their perceptions don't match the objective truth). You might have to turn on 10 light bulbs before the person says that the room is twice as bright as it was initially.

Likewise, if you get a gradient that looks linear/smooth and then measure it using a light-meter, you'll find that it's probably logarithmically curved!

I'm not sure how photoshop creates gradients - but there are several "smoothing" options from what I remember, to produce results that are perceived as looking nice.

The point of "being gamma correct" is mostly so that we can match how physics/maths works. When we add two lights on top of each other, we need them to behave the way they work in the real world.