Jump to content
  • Advertisement
Sign in to follow this  
RoundPotato

[SOLVED] Receive data into a buffer.

This topic is 2126 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hello all.

 

When you receive data from a socket, you pass a buffer into the "recv()" with the maximum number of bytes you wish to receive per that call.

 

Now imagine the following sizes:

 

Packet = 28 Bytes

Buffer = 56 Bytes (28*2)

 

You call "recv()" with buffer and it's size 56.

You get 43 Bytes from "recv()".

You process the first 28 Bytes (packet).

Now there is 15 (43-28) Bytes left on the buffer of another packet, which leaves you with 2 options:

 

1) Tell "recv()" to fill Buffer+43 (where part of second packet data ended) with 56-43 (remaining buffer) Bytes, after processing the second complete packet you can start all over from beginning by passing the start of buffer with maximum length, Sounds useless... If we make the buffer longer, say 56 Bytes more, it'll take longer to reach the end of the buffer, thus simply receiving and processing until the time comes, but when it comes you can only fill up the limited remainder of the buffer and start over.

2) You can process 28 Bytes (packet) and move everything after to the start of buffer.

 

In here: http://beej.us/guide/bgnet/output/html/singlepage/bgnet.html#sonofdataencap . It is mentioned that you can get around the moving by using a "circular buffer". As I understood it it would be the "1)" case where you will stumble onto that problem of "limited remainder of buffer".

 

I think I misunderstand something here. What is the way to go about it? Please assist unsure.png

Edited by RoundPotato

Share this post


Link to post
Share on other sites
Advertisement

A circular buffer isn't a bad solution, but it can be a little overkill in some cases. For something like this, you'd read until you had a full package, process it, then continue reading into the buffer at the position where you left off until you either have a complete package to work or else you reach the end of the buffer. When you hit the end, you just start reading into the beginning again (hence the 'circular'). You have to keep close track of what data is 'live' and what data is processed, so that you don't accidentally overwrite live data.

 

You'll end up moving data in either case, but the circular case will defer some of that so that you only have to do moves when you cross the end of the buffer (to reconstitute that package for processing). Circular buffers are usually used in a producer/consumer situation. The socket would produce and some other process would consume the data as needed.

 

In many cases you know how much you need to read, though. What's your use case here? Or are you just asking for general knowledge?

Share this post


Link to post
Share on other sites

I don't know how to reply to your question other than the case I described already, sorry :/

 

Re-reading your reply several times I didn't find an answer. The only sentence that seemed the most "answer-like" was 

 

You'll end up moving data in either case, but the circular case will defer some of that so that you only have to do moves when you cross the end of the buffer (to reconstitute that package for processing)

 

Which I'm not sure about what you meant, it's either as I described it in the case "1)" or I have not the slightest idea how that would work since "recv()" can't simply jump to beginning of the buffer after filling it's remainder.

 

unsure.png

Edited by RoundPotato

Share this post


Link to post
Share on other sites

3) You know your packets are 28 bytes, so you make your buffer 28 bytes. Now you don't have to move anything.

That's what I'm getting at there.

 

For the circular buffer, you would pass the offset from the end of the buffer to recv(). When the remaining space is zero, you move the pointer back to the start of the buffer. The idea is that you keep reading in data while there's space, and something else is reading the data and opening up space while you're doing that.

 

It's not trivial to provide a code sample. Circular buffers are a pain in the ass to code. The basic idea is not complex, though. You just pretend (implement) that when you reach the end of the buffer, you go back to the beginning. You have a read position and a write position. Never let the writer pass or equal the reader.

 

What I mean by asking your use case is, "What kind of program are you making?" Or are you just asking for general advice about sockets? I'm asking because the type of program is closely related to how you use sockets in it.

Share this post


Link to post
Share on other sites

One worry I have about circular-buffers or load-as-needed buffers is that, say I'm reading at byte 20, and the buffer is split at byte 22, but I know "in total", the size of the buffer is 30 bytes. If I tried to read or write a 4-byte integer or a struct of some kind, only the first two bytes would be correct, the rest would (on read) be gibberish or (on write) write to an area of memory that I shouldn't be writing to.

 

Getting around that is possible but adds unnecessary performance and convenience issues for what should be a straightforward thing - I'd rather just have a re-usable pre-allocated buffer of reasonable size X and move the data as needed.

 

[Edit:] Wasn't fully thinking the issue through - I was thinking more general-purpose circular buffers. With just holding two packets of a known max size, it could probably be done fairly simply.[/edit]

 

What problem are you trying to solve? What problem was the article trying to solve? When was the article written? It's copyrighted 2013, but I think it was written pre-2000 and maybe even pre-1990. The tutorial is still very much relevant, but the problem he's solving here might not be, especially if he's trying to conserve limited RAM or something, since our RAM has doubly-triply-quadrupled seven times over since then.

Edited by Servant of the Lord

Share this post


Link to post
Share on other sites

First of all thank you for the replies.

 

Well, let me draw the attention of the problem being solved, the example I described shall be the problem with the cases options I already described.

 

Now:

 

If I understood correctly Servant, you pretty much suggest to simply use the "2)" option? (even if not moving after each processed packet but as necessary that would be the "1)" case but with alteration of moving the unprocessed data to start of buffer if remainder of the buffer is insufficient with avoiding 2 calls to "recv()")?

 

As for the "3)" option suggested, the problem is that you limit the "recv()" with one packet size and when more data than 1 packet arrives the rest will not be read requiring additional "recv()", thus slowing down the connection interaction.

Share this post


Link to post
Share on other sites

Yea, I'd just move the memory to the start of the buffer. Moving a continuous block of memory from one location to another happens to be a specialty of computers and, unless the block of memory is very large, should go pretty lightning fast using std::copy or memcpy. I'd go with whatever is the simplest to understand and the easiest to implement - and then only if it turns out to be a bottleneck look into more optimized solutions.

 

You'll get about ~30 packets in a second, right? One move per packet shouldn't be the bottleneck of your application. It might cause latency issues even if it doesn't cause framerate issues, but I seriously doubt that.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!