If you use player ids the cheating player can just add/sub a small value from his own id and then try to spoof it, but if he doesnt get to know the IP of other players, he cant spoof it easily?
Same goes for knowing the player ID of another player I would think. However, what you say is worth some thought. It would require doing a mapping from source ip/port (which is 6 bytes in ipv4) to the player stream handler. Certainly doable. And since the port is likely to change with each client creation of the source socket, it might work. Would have to make sure the client creates a single datagram handling outgoing socket and uses that rather than comming from a dozen ports. Interesting idea.
Its still spoofable of course, everything non-encrypted is. And whether I can stack on encryption depends on a ton of other issues.