• Announcements

    • khawk

      Download the Game Design and Indie Game Marketing Freebook   07/19/17

      GameDev.net and CRC Press have teamed up to bring a free ebook of content curated from top titles published by CRC Press. The freebook, Practices of Game Design & Indie Game Marketing, includes chapters from The Art of Game Design: A Book of Lenses, A Practical Guide to Indie Game Marketing, and An Architectural Approach to Level Design. The GameDev.net FreeBook is relevant to game designers, developers, and those interested in learning more about the challenges in game development. We know game development can be a tough discipline and business, so we picked several chapters from CRC Press titles that we thought would be of interest to you, the GameDev.net audience, in your journey to design, develop, and market your next game. The free ebook is available through CRC Press by clicking here. The Curated Books The Art of Game Design: A Book of Lenses, Second Edition, by Jesse Schell Presents 100+ sets of questions, or different lenses, for viewing a game’s design, encompassing diverse fields such as psychology, architecture, music, film, software engineering, theme park design, mathematics, anthropology, and more. Written by one of the world's top game designers, this book describes the deepest and most fundamental principles of game design, demonstrating how tactics used in board, card, and athletic games also work in video games. It provides practical instruction on creating world-class games that will be played again and again. View it here. A Practical Guide to Indie Game Marketing, by Joel Dreskin Marketing is an essential but too frequently overlooked or minimized component of the release plan for indie games. A Practical Guide to Indie Game Marketing provides you with the tools needed to build visibility and sell your indie games. With special focus on those developers with small budgets and limited staff and resources, this book is packed with tangible recommendations and techniques that you can put to use immediately. As a seasoned professional of the indie game arena, author Joel Dreskin gives you insight into practical, real-world experiences of marketing numerous successful games and also provides stories of the failures. View it here. An Architectural Approach to Level Design This is one of the first books to integrate architectural and spatial design theory with the field of level design. The book presents architectural techniques and theories for level designers to use in their own work. It connects architecture and level design in different ways that address the practical elements of how designers construct space and the experiential elements of how and why humans interact with this space. Throughout the text, readers learn skills for spatial layout, evoking emotion through gamespaces, and creating better levels through architectural theory. View it here. Learn more and download the ebook by clicking here. Did you know? GameDev.net and CRC Press also recently teamed up to bring GDNet+ Members up to a 20% discount on all CRC Press books. Learn more about this and other benefits here.
Sign in to follow this  
Followers 0
Shawn619

How do i determine send/recv buffer size

14 posts in this topic

In terms of Winsock,

 

if the client sometimes needs to send 100 bytes and sometimes 1000 bytes for example, how is the server supposed to know before-hand the buffer size for the incoming send() from the client of 100 or 1000 bytes that i would need in recv() on the server?

 

Should i just set them both to a limit that i know for certain it won't reach, say 2048:

Client:
send(socket, cstr, 2048);
 
Server:
recv(socket, recvbuf, 2048);

Or should i do like the Winsock example and save packet space by  when i send() by capping the buffer size at the c-string's size:

Client:
send(socket, cstr, strlen(cstr));
 
Server:
recv(socket, recvbuf, 2048);

Questions:

How do i determine the buffer size to use?

Should the send()'s buffer size always be capped at the length of data im sending(strlen(ctr)), or should it just be 2048 for example across the whole program?

Am i wasteing 2000+ bytes worth of packets if i do something like this

send(socket, "Username=Jimmy".c_str(), 2048);//not proper syntax i know

 ?

 

0

Share this post


Link to post
Share on other sites

Shawn, the last project I worked on used Nypyren's first suggestion and it is by far the eaiest way to manage this problem.  Just send the size first.

0

Share this post


Link to post
Share on other sites

My server did it in this way, packet:

byte - packet id

signed short (2 bytes) - packet size

body (N bytes) - packet body

 

First alloc any size you want, put there required information, when you done, set packet id, calculate size of output buffer and set it to 2,3d bytes.

 

If your packet has constant size, and has id = 1, for example:

id(byte)

size(sshort)

account(char 16)

password(char 16)

save_password(byte(bool))

total 36 bytes, there are always 36 bytes, and you know that client have to read 36 bytes for packet with id 1.

 

If you have dynamic size packet with id 2:

id(byte)

size(sshort)

message(string)

You have always 3 bytes in begin and some bytes in the end. You make dynamic buffer, put there your string, for example("Hello, there!") - 13 bytes. And set it to packet.

id: 2

size: 16

message: "Hello, there!"

You read first 3 bytes to get id, and size, and then read whole string.

 

But i guess dynamic strings you can set only to last value. If you have string in middle of the packet, you should know about that and set constant size for this string.

Edited by TGrey
0

Share this post


Link to post
Share on other sites

The way I have solved this is the following:

 

Set both the send and recv buffers to some pre-defined max-size, for me that is 2048 bytes or 2kb. No packet sent can be larger than this, ever. This allows you to pre-allocate as many 2kb buffers are you need to handle all incoming packets, and re-cycle them by just zeroing the memory again, without the need to free/alloc new memory every time. My game is also written in C#, so this lessens the pressure on the GC a lot, allowing me to recycle packet buffers and not having to re-size them, ever.

 

So, about knowing how much "data" you can read from each received packet, most (all?) socket layers return the amount of bytes read from the recvfrom call. But since we usually deal with bits and not bytes, this only gives us an upper bound on the amount of bits (received bytes * 8) that we have. There are three different ways to deal with getting the exact data out of the packet that I have found, as an example I am going to say that I need to send 5 RPC calls and 2 Player Data objects

  1. The most common and obvious one is to have the first two bytes of the packet be the exact size, in bits, of the packet, so the packet would look like this: [size][rpc][rpc][rpc][rpc][rpc][playerData][playerData]
  2. Know what data you sent in the packet, and pre-fix each block of data with the count of that type of data, for example if i need to send five RPC calls and eight player data objects, my packet would look like this: [5][rpc][rpc][rpc][rpc][rpc][2][playerData][playerData]
  3. Before each segment of data, write one bit set to 1. Before you read each piece of data, you read this bit, if it's 1 you should read the data, if it's 0 you are done, so the packet would look like this: [1][rpc][1][rpc][1][rpc][1][rpc][1][rpc][1][playerData][1][playerData]

Personally i use the third approach because the other two methods require you to "go back" and write the size/count after you have serialized all the data, and it creates an awkward logic flow. The third approach allows each write to be completely self-contained with no need to refer to anything outside of itself.

0

Share this post


Link to post
Share on other sites

The question is concerning to me, because it seems like you think that a single call to send() will end up in a single call to recv() on the other end.

This is not the case.

 

If the sender sends one message of 15 bytes, one message of 238 bytes, and one message of 1 byte, the recipient may receive:

- one message of 15 bytes, and one message of 239 bytes

- or one message of 2 bytes, one message of 4 bytes, one message of 8 bytes, one message of 16 bytes, one message of 32 bytes, one message of 64 bytes, and one message of 128 bytes

- or one message of 254 bytes

- or some other sequence

This assumes there is no failure in the network, in which case the recipient may receive nothing.

 

The receiver in a TCP based system should pretty much always keep a fixed-size receive buffer, and each time calling recv(), call recv() for as much as is left in that buffer. If you use a cyclic buffer, it means you don't have to copy/move data as you dequeue it, but you have to deal with the "break" at the end.

 

Once you have received data into the buffer, you can start looking at it to see if you have received a full message from the other end (whatever that means,) and if so, process and remove it. Repeat.

1

Share this post


Link to post
Share on other sites

The question is concerning to me, because it seems like you think that a single call to send() will end up in a single call to recv() on the other end.

This is not the case.

 

If the sender sends one message of 15 bytes, one message of 238 bytes, and one message of 1 byte, the recipient may receive:

- one message of 15 bytes, and one message of 239 bytes

- or one message of 2 bytes, one message of 4 bytes, one message of 8 bytes, one message of 16 bytes, one message of 32 bytes, one message of 64 bytes, and one message of 128 bytes

- or one message of 254 bytes

- or some other sequence

This assumes there is no failure in the network, in which case the recipient may receive nothing.

 

I don't know if you're referring to Nagle's algorithm, but i've never had a situation where a remote client would recv() partial bytes, like 2 bytes of 100 bytes for example. Either the client receives the server's send() message in full or a winsock error message (ie: error code 10035) is created, per the return value of recv(). Or maybe you're referring to a client that has a network connection that's cutting in and out 

 

 

The receiver in a TCP based system should pretty much always keep a fixed-size receive buffer, and each time calling recv(), call recv() for as much as is left in that buffer. If you use a cyclic buffer, it means you don't have to copy/move data as you dequeue it, but you have to deal with the "break" at the end.

 

Once you have received data into the buffer, you can start looking at it to see if you have received a full message from the other end (whatever that means,) and if so, process and remove it. Repeat.

 

Good, this is exactly what i'm doing and it's working perfect.  

0

Share this post


Link to post
Share on other sites

I don't know if you're referring to Nagle's algorithm, but i've never had a situation where a remote client would recv() partial bytes, like 2 bytes of 100 bytes for example. Either the client receives the server's send() message in full or a winsock error message (ie: error code 10035) is created, per the return value of recv(). Or maybe you're referring to a client that has a network connection that's cutting in and out 

 

 

Maybe it's rare, but it does happen and it is a case that needs to be handled.

1

Share this post


Link to post
Share on other sites

I don't know if you're referring to Nagle's algorithm, but i've never had a situation where a remote client would recv() partial bytes, like 2 bytes of 100 bytes for example. Either the client receives the server's send() message in full or a winsock error message (ie: error code 10035) is created, per the return value of recv(). Or maybe you're referring to a client that has a network connection that's cutting in and out


It's easy to have this happen if you encounter packet fragmentation, e.g. sending a packet larger than routable MTU. It isn't exactly rare if you're sending nontrivial amounts of data.
1

Share this post


Link to post
Share on other sites

 

 

 

 

My guess is you simply haven't been paying attention. This is not particularly rare -- over the greater internet, with your own hosting and a remote user, you will possibly see it on up to a percent of all packets.

I am not referring to the bad case of networks that are physically unreliable. A simple packet collision on any segment between sender and receiver is enough to make this happen.

0

Share this post


Link to post
Share on other sites

It's easy to have this happen if you encounter packet fragmentation, e.g. sending a packet larger than routable MTU. It isn't exactly rare if you're sending nontrivial amounts of data.

 

Oh ok i see, i'll account for this, thanks.

 

 

My guess is you simply haven't been paying attention. 

Relax big boy, there's no reason for personal attacks, you should know that.

Edited by Shawn619
0

Share this post


Link to post
Share on other sites

I don't think "I guess you haven't been paying attention" is an attack in this context; my apologies if you read it as such. I merely meant that you don't see these things unless you explicitly look for them, but once you do, you will see them with some regularity. If you're not explicitly looking for merged packets, you will "only" see "something weird" which you'll have no way to investigate (if you even pay attention at the time it happens.)

 

How about "I guess you haven't been explicitly looking"?

0

Share this post


Link to post
Share on other sites

I don't think "I guess you haven't been paying attention" is an attack in this context; my apologies if you read it as such. I merely meant that you don't see these things unless you explicitly look for them, but once you do, you will see them with some regularity. If you're not explicitly looking for merged packets, you will "only" see "something weird" which you'll have no way to investigate (if you even pay attention at the time it happens.)

 

How about "I guess you haven't been explicitly looking"?

That sounds much better. You may want to be cautious with your phrasing, because "My guess is you simply haven't been paying attention." infers the other person was too lazy to notice something.

 

I accept your apology. I don't enjoy calling random people out in a gaming forum.

0

Share this post


Link to post
Share on other sites

Dude, you need to grow a thicker skin.

 

Especially on the internet it's a bad idea to assume people are malicious with their wording. All we have is text, and human communication is far more nuanced than just the verbal layer.

 

 

Nobody attacked you, nobody insulted you, and you have nobody to "call out."

0

Share this post


Link to post
Share on other sites

Dude, you need to grow a thicker skin.

 

Especially on the internet it's a bad idea to assume people are malicious with their wording. All we have is text, and human communication is far more nuanced than just the verbal layer.

 

 

Nobody attacked you, nobody insulted you, and you have nobody to "call out."

Let's not gang up on me now, I already accepted his apology,

0

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0