1) In this case the tangent space and the world space are the same for your triangles, so you don't need to worry about the tangent space or calculating tangents etc. Just read the normals from the texture and use them for lighting as if they were normals in world space.
2) having the TBN matrix for a vertex and using tangent space normal map won't produce the lighting anomaly you described in the first post.
Cheers!
I've deleted the post before you replied, weird stuff going on, anyway thanks for reply. Deleted it because I've found few more pages about tangent space, decided to read them on before asking again for help.
Back on the topic: If I use them just as a normals, I need to swap y and z coordinates of the normal vectors sampled from the map, (and negate the z coordinate) to have it work properly. Is it the issue with handiness of the system the program uses, or me?
EDIT: Current result (I don't really mind the transition areas, that's probably the normal map itself)