https://tse3.mm.bing.net/th?id=OIP.fCJwPm8kAFQIdWH9AZ7lWQEsDI&pid=15.1&P=0&w=251&h=168

First, I'm using standard RGB8 textures for my skybox, not HDR.  If I use a skybox with an image like above, downsampling it simply gives me a blue tint down my mip map chain. For PBR rougher surfaces, they all just look blue tinted when they shouldn't be.

I was thinking maybe this was because of using LDR textures where the brighter pixels would spread further, and thus making the downsamples brighter.  I would think that the clouds wouldn't be so bright to overcome downsampling more to white even as HDR, but maybe that is wrong. Even in a very cloudy day with minimal sun and blue sky, I would think even with HDR that it would just downsample to blue, but it depends on how many bright pixels are clustered and how high their range is.

 

This doesn't really answer your question, but just wondering, do you really need mipmaps for the skybox? isn't it always going to be the same distance from the camera?

The skybox is my cubemap for IBL for my PBR shader.

I can support HDR in my engine, but at the asset level, I dont support HDR textures. So yes at some point I need to support loading in my skies specifically as HDR (no other specific asset in games other than potentially an emissive channel can I think would need HDR).

HDR and PBR are completely separate things. PBR is how light interacts and bounces. HDR is in respect to capturing a proper range of light/photons. So no HDR is not a requirement of PBR.

Your intuition is correct: with true HDR sky intensities it's possible that the lower mip levels could have a very different result. It depends on the relative intensities between the clear pixels and the cloud pixels, which can  very different in a tone-mapped LDR image vs. an HDR image that uses values that are directly proportional to physical intensities.