I figured out the reason it was taking a relatively long time to load the texture. In my D3DX11_IMAGE_LOAD_INFO structure I had set the format to DXGI_FORMAT_R8G8B8A8_UNORM_SRGB when it should have been DXGI_FORMAT_BC1_UNORM_SRGB. Using D3DX11GetImageInfoFromFile gets rid of that problem.
I took a look at DirectXTex, unfortunately I was unable to compile the source. I am using VS2010 Express and it does not support OpenMP. I think I may just write my own texture tool instead.