I noticed that even the default CyanogenMod UI has some delay, so it either is a hardware issue or OS issue.
Further I found two more threads which appear to be about the same problem, no solution through:
It is mentioned that there should be less lag on HTC devices, and I read somewhere else that iphones should have less lag, too.
Seems like not even the developers are sure what the reason is.
Anyway, unless we hack into android we can't change it.