I figured it out!

Given the normals of corners V0, V1 and V2, and the alpha values a0 and a1 that are to be used as in the picture (the picture uses the example values (0.5, 0.5)), I want to use spherical linear interpolation to calculate the normal.

I realized that this can be done using 3 slerps. First I slerp based on a0 between v0 and v2 AND between v1 and v2. The normal we're looking for lies inbetween these two resulting normals, so we slerp again between those two based on (a1 / (1.0 - a0)).

slerp0 = v0.slerp(v2, a0); slerp1 = v1.slerp(v2, a0); result = slerp0.slerp(slerp1, a1 / (1.0-a0));

Sadly it didn't help much in my case, but someone might find it useful.