It's hard to imagine it making any material difference, and as you point out, only really favours a subset of one vendor's hardware right now. It also uses up some local memory that could otherwise be used for other purposes.

As for latency being an issue, it'll only be the first access to each cache line that's affected. A typical constant value is going to get accessed by the first vertex or the first pixel and all the rest are going to grab it from the same low latency cache it would have grabbed it from whether the data had come from local or system memory.