Since you are using d3dxeffect you should remove some redundant lines that might cause some conflicts, i chop it up for you:

D3DXHANDLE hTechnique;
hTechnique = pEffect->GetTechniqueByName("TerrainTech");
if (FAILED(pEffect->SetTechnique(hTechnique)))
    return;

pEffect->SetTexture("r0_tex", texDetailMap);
pEffect->SetTexture("r1_tex", tex1);
pEffect->SetTexture("r2_tex", tex2);
pEffect->SetTexture("r3_tex", tex3);
pEffect->SetTexture("r4_tex", tex4);
pEffect->SetTexture("r5_tex", texDetailMap);

D3DXMATRIX matWorldViewProj = terrain->worldMatrix() * camera->viewMatrix() * camera->projectionMatrix();
D3DXMATRIX matWorld = terrain->worldMatrix();

pEffect->SetMatrix("WorldViewProj", &matWorldViewProj);
pEffect->SetMatrix("World", &matWorld);

UINT passes = 0;
pEffect->Begin(&passes, 0);
pEffect->BeginPass(0);

// Render terrain
terrain->render();

pEffect->EndPass();
pEffect->End();

@belfegor: I tried that, it doesn't help, same problem + the texture become not mirrored.

I am sorry, i have overlooked mirror address mode. Set it like this in effect file if you want mirroring:

texture r0_tex;
sampler r0_samp  = sampler_state
{
    Texture     = <r0_tex>;
    MIPFILTER   = LINEAR;
    MAGFILTER   = LINEAR;
    MINFILTER   = LINEAR;
    ADDRESSU = MIRROR;
    ADDRESSV = MIRROR;
};
... same for the rest

As to why is brighter, i don't know, this should be same as with asm shader.

Maybe reverse order in lerp functions:

r1 = r1 * r0.x;
    r2 = lerp( r1, r2, r0.y );
    r3 = lerp( r2, r3, r0.z );
    r0 = lerp( r3, r4, r0.w );
    r5 = r5 * r0;

@belfegor: That resolved the bright problem, now I notice that the only texture that appear on the mesh is tex1 + detail map

I don't see other textures on the terrain even the blend map is correct and works with the old assembly shader.

Resolved! I was setting the detail map as the first texture instead of the blending map.

Will I be able to set a different detail map for each texture if I multiplied each texture with a different detail map?

Thanks