gl_FragColor = (behindpixel * 0.5 ) + (tex * 0.5 );
What you're trying to do is pretty much standard alpha blending. GL_SRC_ALPHA and GL_ONE_MINUS_SRC_ALPHA, then output 0.5 for your alpha component.
I'm not sure exactly what you're trying to explain with your picture above, but it kind of seems like you want the areas around the tree leaves to be completely transparent? The same alpha blending flags will work for that too - except that you'll still be writing to the depth buffer for those completely transparent pixels, so they'll block stuff from being drawn behind them.
In that case, what you're looking for is alpha testing, not alpha blending.