Sign in to follow this  

How do I handle recvfrom?

This topic is 3474 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

So I can't depend on that the OS will send all my data or receive all my data with send and recv methods. Beej describes a solution in his guide about calling send multiple times until all data have been sent. I can do that with sendto as well. But what about recvfrom? In Beej's example, he assumes that all data comes in order and that they always arrive. How are recvfrom beeing handled in a game that uses UDP? Another question, how should I keep the list of clients in the server? I mean, I need to search two things in this client list. The IP address and the player position. I need the IP address so I can update the player after receiving data on the server. I need the player pos in order to know which players that needs the updated information (so players that aren't near the affected action won't get the message).

Share this post


Link to post
Share on other sites
Typically, one designs or uses a protocol on top of UDP which allows the sender know which packets arrived, and the receiver know in which order they were sent. There are a number of libraries that implement this for you. Check to Forum FAQ, Questions 7 and 8 in particular.

Share this post


Link to post
Share on other sites
If I understand this correctly, I have to use reliable UDP if the package exceeds a certain size? How can I determine how much I can send and recv without the package being divided?

Share this post


Link to post
Share on other sites
Quote:
Original post by RandomPixel
If I understand this correctly, I have to use reliable UDP if the package exceeds a certain size? How can I determine how much I can send and recv without the package being divided?


You need to use some form of reliable transfer, if you want to transfer data reliably (without loss, knowing which packets got lost, transferring data in order).

The maximum data you can send without fragmentation depends, for practical purposes, on the smallest MTU between peers. Usually that's in 530-1500 byte range.

UDP gives you no guarantees of any kind. If you need any guarantee for anything, you need to implement it yourself.

Share this post


Link to post
Share on other sites
I understand that. I'm not actually looking for guarantee that all packets arrive. I just wanted to understand how to implement recvfrom.

For example, if the client sends these packets:
packet 1: MOV x1 y1
packet 2: TURN d
packet 3: MOV x2 y2

Then the server will maybe get 2 packets:
MOV x2 y2
MOV x1 y1

I can use a timestamp to remove old packets. But is it possible for recv to get a package that is both packet 3 and packet 1 in one:
MOV x2 y2 MOV x1 y1

or even getting packet 3 and the last part of packet 1:
MOV x2 y2 y1

How can I handle stuff like that?

Share this post


Link to post
Share on other sites
Quote:
Original post by RandomPixel
I understand that. I'm not actually looking for guarantee that all packets arrive. I just wanted to understand how to implement recvfrom.


recvfrom under UDP has only one use - it reads from network buffer for whatever is in there. Anything else, you need to implement your own logic.

Quote:
I can use a timestamp to remove old packets. But is it possible for recv to get a package that is both packet 3 and packet 1 in one:


It's possible, if you implement that.

Quote:
How can I handle stuff like that?


send and recv handle with a sequence of bytes, and absolutely nothing more.

If you want to pack multiple logical units into one, you implement that yourself, by constructing buffer appropriately.

If you need reliable transport, you add a sequence number to each packet. This way, you know which went missing. You then send a packet requesting resend. When a packet is received, you send ack notifying which packet arrived. If some packets don't arrive, you resend them. If some arrive multiple times, you discard them.

An example of reliable transport implementation is covered by TCP's implementation, as well as many networking courses (implementation of reliable UDP is a common excercise for these). Another topic of interest here is serialization.

But there is a lot of ground to cover just to achieve efficient, yet robust UDP-based communication.

Share this post


Link to post
Share on other sites
Thanks! I think I understand now.

So basically. If I send the package "MOV x1 x2" and if it arrives on the server, calling recvfrom on the server will get exactly the package "MOV x1 x2" and not part of other packets.

As long as I'm not sending more than 530 bytes, I will get the entire package with recvfrom just as the package was when sending it with sendto.

Share this post


Link to post
Share on other sites
Quote:
Original post by RandomPixel
Thanks! I think I understand now.

So basically. If I send the package "MOV x1 x2" and if it arrives on the server, calling recvfrom on the server will get exactly the package "MOV x1 x2" and not part of other packets.


UDP is packet based, and packets are defined by what you sendto(). So yes.

But it's customary for UDP protocol implementations to use some custom checksum, perhaps based on peer's IP/Port, to ensure the contents are valid, and of appropriate protocol. Even if using custom ports, there's a lot of noise, and receiving unrelated random packets is not uncommon.

Other than that, the documentation on sendto and recvfrom is quite comprehensive.

Share this post


Link to post
Share on other sites
Quote:
Original post by RandomPixel
As long as I'm not sending more than 530 bytes, I will get the entire package with recvfrom just as the package was when sending it with sendto.
You'll get that if you send more, too.
UDP has almost no guarantees, but it does have one guarantee. A datagram is either dropped/lost or delivered completely. If you exceed the MTU, the packet will get fragmented on the IP layer. If only one of the fragments is lost, the entire UDP datagram is discarded. If they all arrive, the whole UDP datagram will be in your buffer when recvfrom() returns.
In theory, you could write a server that never uses anything like select/poll, and simply blocks in recvfrom() inside an endless loop. Whenever data arrives, it will wake up, do some work (maybe send a reply), and block again. If periodic updates are needed, that would obviously have to happen from another thread then, but just for the principle of a really simplistic server, it's quite cool, I think.
Compare that to the pain of managing 500 or 1000 sockets, and passing them to poll (or anything similar) just to know when something happens on a TCP server.

Share this post


Link to post
Share on other sites
Quote:
Original post by Antheus
But it's customary for UDP protocol implementations to use some custom checksum, perhaps based on peer's IP/Port, to ensure the contents are valid, and of appropriate protocol. Even if using custom ports, there's a lot of noise, and receiving unrelated random packets is not uncommon.


I was thinking about doing some kind of validation to check that the package I get is valid to my game. In case of cheating and stuff like that. Like a unique ID for the client or maybe check so the ip addresses matches.

Quote:
Original post by samoth
You'll get that if you send more, too.
UDP has almost no guarantees, but it does have one guarantee. A datagram is either dropped/lost or delivered completely. If you exceed the MTU, the packet will get fragmented on the IP layer. If only one of the fragments is lost, the entire UDP datagram is discarded. If they all arrive, the whole UDP datagram will be in your buffer when recvfrom() returns.


Oh, that's even easier. So no matter how big the package is I use in the sendto method, if it arrives I get the exact package in recvfrom.

Share this post


Link to post
Share on other sites
Quote:
Original post by hplus0603
With UDP, that is true. With TCP, it is not.


Ok. When using TCP, I could just use Beej's method.

But according to Beej, there is no guarantee that sendto will send the whole package. As I understand it, everytime I use sendto will count as one package each? So if sendto couldn't send the complete message, then recvfrom would get an uncomplete message.

Share this post


Link to post
Share on other sites


To simplify the IP lookup you can establish your own system of 'sessions' (user id's) which is included in every packet sent back from a client (it would be an index into an array of all the user sessions you have opened on the server). This is especially useful to cut down the processing when you have many sessions active at once. You would used the id in the packet and THEN compare the IP from recv against THAT seesions IP (no need for a linear lookup or some kind of hashing table mechanism). If it doesnt match then its bogus and you either ignore it or report it.



Even if you arent sending packets reliably, having a connection (session) packet counter (an ascending packet # included in every packet) to detect old delayed packets which might no longer be relevant (stale data). You keep track of the last packet received and if a packet comes in that less-than-or-equal to that value then you should discar the packet as being outofdate. A variation might be to having seperate turn and packet numbers sent (if your game system works with turns) with out oforder packets from the current turn being accepted and older ones (previous turns) being discarded.




Share this post


Link to post
Share on other sites
Quote:
Original post by wodinoneeye
To simplify the IP lookup you can establish your own system of 'sessions' (user id's) which is included in every packet sent back from a client (it would be an index into an array of all the user sessions you have opened on the server).


Thanks! This is a very good idea. This way, the lookup would be O(1).

I was thinking to implement a system where players that are far away from another player won't retrieve messages from that player. For each message the server retrieves from one player, it have to go thru the list and send the message to all nearby players. This would make it O(n). Is there a better way?

Share this post


Link to post
Share on other sites
Quote:
Original post by wodinoneeye


To simplify the IP lookup you can establish your own system of 'sessions' (user id's) which is included in every packet sent back from a client (it would be an index into an array of all the user sessions you have opened on the server). This is especially useful to cut down the processing when you have many sessions active at once. You would used the id in the packet and THEN compare the IP from recv against THAT seesions IP (no need for a linear lookup or some kind of hashing table mechanism). If it doesnt match then its bogus and you either ignore it or report it.


I find this to be one of those really clever hacks which look great, yet never deliver.

If you have enough peers for IP lookup to become a performance problem, then the server load will scale much worse than IP lookup. This is simply such a cheap operation, that unless you're doing just some routing, it doesn't really show up anywhere. Just inspecting the packet (checksum, serialization, dispatch) will cost much more than that.

Quote:
I was thinking to implement a system where players that are far away from another player won't retrieve messages from that player. For each message the server retrieves from one player, it have to go thru the list and send the message to all nearby players. This would make it O(n). Is there a better way?


When an object becomes interesting to a client, subscribe the client. When an object changes, it sends the changes to all subscribers.

Share this post


Link to post
Share on other sites
TCP is stream-based. It will append whatever you put in each send() into a buffer, and send X bits of that buffer at a time, depending on quality of service. For example, you could be sending two bytes, you will either receive the first byte in a packet (one call of recv()), and the next byte in another packet one second later (the next call of recv()), or the two bytes in the same packet.

So the latency is variable and all down to the TCP layer (which optimises the transmission the best it can).

for UDP, what you send is what you get, if you get it (it is not reliable), but the packets might be out of order as well.

[Edited by - oliii on July 14, 2008 5:13:12 AM]

Share this post


Link to post
Share on other sites
Quote:
Original post by Antheus
Quote:
Original post by wodinoneeye


To simplify the IP lookup you can establish your own system of 'sessions' (user id's) which is included in every packet sent back from a client (it would be an index into an array of all the user sessions you have opened on the server). This is especially useful to cut down the processing when you have many sessions active at once. You would used the id in the packet and THEN compare the IP from recv against THAT seesions IP (no need for a linear lookup or some kind of hashing table mechanism). If it doesnt match then its bogus and you either ignore it or report it.


I find this to be one of those really clever hacks which look great, yet never deliver.

If you have enough peers for IP lookup to become a performance problem, then the server load will scale much worse than IP lookup. This is simply such a cheap operation, that unless you're doing just some routing, it doesn't really show up anywhere. Just inspecting the packet (checksum, serialization, dispatch) will cost much more than that..




Well its just one of many that add up to considerable speedups. Together they deliverd more than adaquately on the project I did for the game company. The requirements made alot of optimization necessary.

Share this post


Link to post
Share on other sites
Quote:
Original post by RandomPixel
Quote:
Original post by wodinoneeye
To simplify the IP lookup you can establish your own system of 'sessions' (user id's) which is included in every packet sent back from a client (it would be an index into an array of all the user sessions you have opened on the server).


Thanks! This is a very good idea. This way, the lookup would be O(1).

I was thinking to implement a system where players that are far away from another player won't retrieve messages from that player. For each message the server retrieves from one player, it have to go thru the list and send the message to all nearby players. This would make it O(n). Is there a better way?



If you arent scaling this too large, a simple linear check can be more than adaquate (because of the overhead of maintaining the more complicated 'super-sized' solution)


Space partitioning is what you want to look into to generate a subset to send local events to. ( http://www.devmaster.net/articles/bsp-trees/ )

Another probably simpler space partitioning is a regular grid which divides up the map and you maintain a list for each grid area of which player objects are inside it (and thus you have to pull the object out and put it in the list next over when it moves across a boundry...) and you then send the notifications for an objects event/action to all the object in the same grid area and also to adjacent grid areas (8 neighbors or 1/2/3 neighbors which overlap a box check of the range of observation. You could use a crude check like that to create a subset list of candidates (a cheap 'lazy evaluation') who then are subjected to a more costly (more exact) test -- box or squareroot type distance check.

(remember you have to handle the edge of map special cases...)


Share this post


Link to post
Share on other sites
Sign in to follow this