I've got it to look fairly decent for spotlights now, but point lights are a bit more tricky.

I found one way on how to do it here: http://www.sunandblackcat.com/tipFullView.php?l=eng&topicid=36

Basically, he just samples the shadow with offsets in various different directions, then averages the results. I am again unclear about a couple of things though. Here's his fragment shader:

#version 330

// shadow map
layout(location = 0) uniform samplerCube	u_shadowCubeMap;

// world space position of the fragment
in vec4	o_worldPosition;

// color to the framebuffer
layout(location = 0) out vec4 resultingColor;

// position of the point light source
uniform vec3	u_lightPos;
// distances to near and far cliping planes
uniform vec2	u_nearFarPlane;

// array of offset direction for sampling
vec3 gridSamplingDisk[20] = vec3[]
(
   vec3(1, 1, 1), vec3(1, -1, 1), vec3(-1, -1, 1), vec3(-1, 1, 1),
   vec3(1, 1, -1), vec3(1, -1, -1), vec3(-1, -1, -1), vec3(-1, 1, -1),
   vec3(1, 1, 0), vec3(1, -1, 0), vec3(-1, -1, 0), vec3(-1, 1, 0),
   vec3(1, 0, 1), vec3(-1, 0, 1), vec3(1, 0, -1), vec3(-1, 0, -1),
   vec3(0, 1, 1), vec3(0, -1, 1), vec3(0, -1, -1), vec3(0, 1, -1)
);

// function compares distance from shadow map with current distance
void sampleShadowMap(in vec3 baseDirection, in vec3 baseOffset,
      in float curDistance, inout float shadowFactor, inout float numSamples)
{
   shadowFactor += texture(u_shadowCubeMap,
      vec4(baseDirection + baseOffset, curDistance));
   numSamples += 1;
}

void main(void)
{
   // difference between position of the light source and position of the fragment
   vec3 fromLightToFragment = u_lightPos - o_worldPosition.xyz;
   // normalized distance to the point light source
   float distanceToLight = length(fromLightToFragment);
   float currentDistanceToLight = (distanceToLight - u_nearFarPlane.x) /
            (u_nearFarPlane.y - u_nearFarPlane.x);
   currentDistanceToLight = clamp(currentDistanceToLight, 0, 1);
   // normalized direction from light source for sampling
   fromLightToFragment = normalize(fromLightToFragment);

   // sample shadow cube map
   float referenceDistanceToLight	= texture(u_shadowCubeMap, -fromLightToFragment).r;

   float shadowFactor = 0;
   float numSamples = 0;

   // radius of PCF depending on distance from the light source
   float diskRadius = (1.0 + (1.0 - currentDistanceToLight) * 3) / sizeOfCubeTex;

   // evaluate each sampling direction
   for(int i=0; i<20; i++)
   {
      sampleShadowMap(-fromLightToFragment, gridSamplingDisk[i] * diskRadius,
         currentDistanceToLight, shadowFactor, numSamples);
   }

   // average shadow factor
   shadowFactor /= numSamples;

   // output color to framebuffer
   resultingColor.rgb = vec3(shadowFactor);
   resultingColor.a = 1;
}

I don't quite understand the calculation for the "currentDistanceToLight". From the shader it looks like "u_nearFarPlane" represents the near / far plane of the camera, but shouldn't it be the near / far plane of the light instead? The shadow map is generated with the light's near and far plane after all.

Secondly, in the calculation for the diskRadius:

float diskRadius = (1.0 + (1.0 - currentDistanceToLight) * 3) / sizeOfCubeTex;

Where does the 'times 3' come from?