The server should always respond on the same socket it recieves from (otherwise firewalls will block the traffic) (a fairly common mistake is to open 2 sockets on the client (one to send and one to recieve, and that just won't work), the server gets the correct port and ip to send to from the recvfrom call.
That was actually why it was working only one way when using my public IP, thanks!
Unfortunately, it still does not work with client and server not in the same network, I'm still not getting any packets.
Another possibility is that you are not resolving the IP correctly.
Are you properly using htons() for the port in there?
Both of those are fine.
Are you setting the sin_af field? Where do you get the address value from?
What sin_af field are you talking about? The address is from the recvfrom (for the server) and set like this in the client:
memset(&addr, 0, sizeof(SOCKADDR_IN)); addr.sin_family = AF_INET; addr.sin_port = htons(1337); addr.sin_addr.s_addr = inet_addr("220.127.116.11");