Did some research together with some trial and error and it is where I sample my 16bpp texture and add to my height that causes this. Here is the code:
vertexPosition.x = vertexPosition.x + (patch[0].sphereNormal.x * ((heightMapSample * 29429.0f) - 8200.0f));
vertexPosition.y = vertexPosition.y + (patch[0].sphereNormal.y * ((heightMapSample * 29429.0f) - 8200.0f));
vertexPosition.z = vertexPosition.z + (patch[0].sphereNormal.z * ((heightMapSample * 29429.0f) - 8200.0f));
It seems that when i multiply by 29429.0f(max height) and subtract by -8200.0f(lowest point) the points that sit in the same position split up causing the texture to move apart
Might have something to do with the normal vector, however that one is just the normal vector of the sphere before adding the height
that shouldn't fuck something up or could it?
More trial and error ongoing
//Toastmastern