Here the metallic part :

SURFACE_DATA SurfaceData;
float4 DiffuseData = DiffuseMap.Sample( LinearSampler, Input.TexCoord );
float4 FinalBaseColor = BaseColor * DiffuseData;
SurfaceData.DiffuseColor = FinalBaseColor.rgb - (FinalBaseColor.rgb * Metallic);
SurfaceData.SpecularColor = lerp( 0.08f * Specular, FinalBaseColor.rgb, Metallic );

Here the compute spec factor function used in lighting :

float3 ComputeSpecFactor( in SURFACE_DATA SurfaceData, in float3 LightDirection, in float NdotL )
{
  // Variables used to compute the lighting factor.
  float3 ViewDirection = normalize( -SurfaceData.Position );
  float3 HalfDirection = normalize( LightDirection + ViewDirection );
  
  // Compute the lighting factors.
  float NdotH = max( 0.0f, dot( SurfaceData.Normal, HalfDirection ) );
  float NdotV = max( 0.0f, dot( SurfaceData.Normal, ViewDirection ) );
  float VdotH = max( 0.0f, dot( ViewDirection, HalfDirection ) );
  
  // Generalized microfacet specular.
  float D = D_GGX( Roughness, NdotH );
  float G = G_Schlick( Roughness, NdotV, NdotL );
  float3 F = F_Schlick( SurfaceData.SpecularColor, VdotH );
  
  // Return the specular factor.
  return D * G * F;
}

About the reflection, did you tried the importance sampling like in the paper of Brian Karis ?

One thing is when using metallic to 1.0f you have black diffuse color and only specular is visible.

Here the filter of envmap from unreal paper :

float3 PrefilterEnvMap( float Roughness, float3 R )
{
  float3 N = R;
  float3 V = R;
  float3 PrefilteredColor = 0;
  const uint NumSamples = 1024;
  for( uint i = 0; i < NumSamples; i++ )
  {
    float2 Xi = Hammersley( i, NumSamples );
    float3 H = ImportanceSampleGGX( Xi, Roughness, N );
    float3 L = 2 * dot( V, H ) * H - V;
    float NoL = saturate( dot( N, L ) );
    if( NoL > 0 )
    {
      PrefilteredColor += EnvMap.SampleLevel( EnvMapSampler , L, 0 ).rgb * NoL;
      TotalWeight += NoL;
    }
  }
  return PrefilteredColor / TotalWeight;
}

float3 ApproximateSpecularIBL( float3 SpecularColor , float Roughness, float3 N, float3 V )
{
  float NoV = saturate( dot( N, V ) );
  float3 R = 2 * dot( V, N ) * N - V;
  float3 PrefilteredColor = PrefilterEnvMap( Roughness, R );
  float2 EnvBRDF = IntegrateBRDF( Roughness, NoV );
  return PrefilteredColor * ( SpecularColor * EnvBRDF.x + EnvBRDF.y );
}

Oh wow, thanks for this! I see I've done a couple of mistakes regarding the diffuse component.

I prefilter my cubemaps using Lagarde's version of AmdCubemapGen, gives me a nice result (not visible in that screenshot), so I don't do any runtime prefiltering.

half current = roughness * 6;
half f0 = floor(current);
half f1 = ceil(current);
half f = saturate(current - f0) / (f1-f0);
//...
half4 reflection1 = texCUBElod (_IndirectLightTex, float4(reflectDir, f0));
half4 reflection2 = texCUBElod (_IndirectLightTex, float4(reflectDir, f1));
half4 reflection = lerp(reflection1, reflection2, f);

half current = roughness * 6;
half4 reflection = texCUBElod (_IndirectLightTex, float4(reflectDir, current));

You are right, it will. I changed it this morning and it looks basically the same, only as you said, faster on mobile :)

I prefilter my cubemaps using Lagarde's version of AmdCubemapGen, gives me a nice result (not visible in that screenshot), so I don't do any runtime prefiltering.

Is it possible to have more info about your work on this part ? Do you convert the roughness to specular power to filter the cubemap ?

Why do you add the reflection on the diffuse and the specular and not only add the reflection at the end ?

(BaseColor * LightColor + Specular) + (ReflectionFiltered * Metallic)

I use the modified AmdCubemapGen from here, which converts gloss to specular power.

The reflection and specular are added like this at the moment:

return half4 ( ( albedo.rgb - albedo.rgb*metalness) * light.rgb + specular + reflection * F0, 1);