However, if my image on disk is sRGB and I specified only D3DX11_FILTER_SRGB_IN, and a non-sRGB texture format, then wouldn't give the same results? It would convert sRGB to linear during load? Then the hardware wouldn't need to do any conversion?

In some cases you may want to store linear data and it's worth the hit, but most of the time it isn't.

First of all note that what SRGB mainly does is save storage space. If you store a texture in linear space in the same number of bits, you'll reduce the quality significantly, especially in darker areas. For that reason any texture or render target used to store linear data tends to be 16-bit per channel.

This will almost certainly lose you much more performance than the small quantity of work doing the linearization on the fly. Textures will be 8 or 16 times bigger if you were previously encoding as D3DFMT_BC2 or D3DFMT_BC1.

May I also recommend converting the texture in an offline tool instead of at load time - it's much quicker that way because you load a fraction of the data(assuming 8888 source assets), and there's much less for the CPU to do.

So if my image on disk is sRGB and I want the texture to be sRGB (and the hardware will convert to linear when sampled), I specify both D3DX11_FILTER_SRGB_IN | D3DX11_FILTER_SRGB_OUT, and an sRGB texture format.

Correct.

However, if my image on disk is sRGB and I specified only D3DX11_FILTER_SRGB_IN, and a non-sRGB texture format, then wouldn't give the same results? It would convert sRGB to linear during load? Then the hardware wouldn't need to do any conversion?

The results would look more or less the same since the sRGB to linear conversion would happen at load time. If the texture format was 8-bit then what you would have is posterization occurring in the dark tones, which is why you should never do that. If the texture format was 16-bits, then it would look fine, but that would be a total waste of GPU memory and bandwidth. Best to stick to 8-bit sRGB texture formats for storing ordinary LDR color information.

You can get away with an 8-bit target if you use some sort of encoding to get an HDR range by utilizing the alpha channel.

http://graphicrants.blogspot.co.uk/2009/04/rgbm-color-encoding.html