When lighting a mesh with an outdoor enviroment map(cubemap), in the past Ive lit the object with a blend between a directional light with both specular and diffuse components, and the the reflection colour from the cube map.

out = lerp( ( SunDiffuse + SunSpecular ), CubeMapReflectionColour, Weight );

Which generally gives nice results, but only works with shiny objects, as the cubemap reflection is always a perfect reflection no matter how low the weighting is. I'm thinking that a way for dull diffuse objects like wood would be to treat the map like an actual light.

Out = CubeMapDiffuse + CubeMapSpecular; // Scattered light
Out = SunDiffuse + SunSpecular; // Direct light

Where CubeMapSpecular is a perfect refletion colour from the ennviroment map and both CubeMapSpecular and SunSpecular would be very low values for something like wood.

But the question arises how to compute CubeMapDiffuse? For this I really would need a blurred out copy of the enviroment map to simulate the diffuse reflected light within the enviroment.

Rather than keeping two copies of the enviroment map, one for diffuse(blurred) and one for specular(no changes), I was thinking of just using a low resolution mip map level for diffuse and high res for specular. Any other suggestions?

Yes. diffuse enviroment map can be much in lower resolution.
But. mip map "burring" don`t use same sort of filter as diffuse calculation neads.
So It`s better to use separate low resolution cube map for diffuse enviroment light. (some 64x64 x6 is good for capturing enviroment, and mem usage is minimal)

look HDR shop (version 1 has free version)

/Tyrian