Sign in to follow this  

The problem of handling incoming network data

This topic is 3806 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

In designing my networking class for my engine, I was recently told about ring buffers and how they could be useful for the temporary storage of data as it comes in from the network and gets stored before being used by the game. This sounds great, as it eliminates data copying. However, the one problem with ring buffers is the fact that a block of data may not be contiguous; the first part of it may be at the end of the buffer while the rest of it wrapped around to the start of the buffer. When designing a networking class to allow the engine to receive data without working about the gory details, how do you provide access to incoming data without exposing the inner quirks of the design, like forcing the game to read the buffer in two separate calls to get the two buffer pieces, or having to do tons and tons of buffer copying to make the incoming data contiguous but render the ring buffer useless? Perhaps I'm thinking too low-level and rather than just having the networking class provide packets of data, maybe it needs to interpret the incoming data and construct more meaningful structures of data for the rest of the engine to use? That could possibly remove the ring buffer fragmentation issue, but then it feels like the networking class is doing too much work, more than just networking, and I don't want it to become a kitchen sink class. How have other people handled this in their engines? Any tips?

Share this post


Link to post
Share on other sites
If you receive data through your DSL line, then you're receiving about 150 kB/sec, tops.

If you have a modern computer, your memory subsystem will be able to copy about 8 GB/s of memory before saturating.

This means that, even if you copy your data, you will lose about 1/53,000 of your available memory bandwidth -- not enough to worry about.

However, if you want to implement a ring buffer, here are the options:

0) In the case where you will overwrite the ring end, receive into another buffer and copy into the ring.
1) Use scatter/gather I/O.
2) When the ring runs out of space for the possible biggest packet, start over at the beginning, skipping the part at the end.
3) Use memory-mapping to arrange two copies of the SAME physical memory after each other, so overwriting the end of the buffer keeps writing in the beginning.

Share this post


Link to post
Share on other sites
Thank you for the excellent analysis, hplus0603. I never considered the bandwidth issue, for me it was always a CPU issue. But more than that I'm just trying to make sure the design I come up with stands the test of time, so that I can leave the internal code unoptimized and be free to optimize it later without design flaws requiring a rewrite of the whole networking system.

I'd never considered your memory-mapping idea, that would solve the ring buffer issue perfectly. I'm familiar with virtual memory mapping from OS theory, but didn't know that type of memory-to-memory mapping was possible from an application standpoint. Is there a portable way of pulling that off? Do you have any references where I can read up on it? I'm somewhat aware of mmap, but only in regards to mapping disk files to memory.

Share this post


Link to post
Share on other sites

Quote:
Original post by Nairou
I'd never considered your memory-mapping idea, that would solve the ring buffer issue perfectly. I'm familiar with virtual memory mapping from OS theory, but didn't know that type of memory-to-memory mapping was possible from an application standpoint. Is there a portable way of pulling that off? Do you have any references where I can read up on it? I'm somewhat aware of mmap, but only in regards to mapping disk files to memory.


I know of no reference. The idea comes from BeOS, where it was considered for multimedia file read buffering. However, doing the copy when we overlapped the end of the ring was sufficient, so we didn't actually use it in the end.

There is no portable way of doing this. Under Windows, I would do (pseudo-code):


ptr = VirtualAlloc(SIZE*2+8192); // find a virtual address range that's big enough
baseA = (ptr + 4095) & ~4095; // page align
map = CreateFileMapping(INVALID_HANDLE_VALUE, "Some Name");
VirtualFree(ptr); // make that virtual address range un-used
tmp = MapViewOfFileEx(map, ..., baseA);
if (tmp != baseA) fail();
tmp = MapViewOfFileEx(map, ..., baseA + SIZE);
if (tmp != baseA + SIZE) fail();


Under UNIX, you could for example create a sufficient range using a shared memory segment, then use mmap with placement.

I haven't actually tried it on Windows or UNIX, but it seems as if it could work. Let us know how it goes :-)

Share this post


Link to post
Share on other sites
There is a semi-portable way using Boost::ASIO.


class RingBuffer {
enum ( SIZE = 16384 };
char buffer[ SIZE ];
char *curr_start; // beginning of occupied space
char *curr_end; // end of occupied space
char *end_ptr; // pointer to end of buffer

template < class Socket >
void read( Socket &s )
{
boost::array<mutable_buffer, 2> buffers = {
boost::asio::buffer(curr_end, end_ptr - curr_end),
boost::asio::buffer(buffer, curr_start-buffer) };
}
bytes_transferred = sock.read(buffer);
curr_end += bytes_transferred;
if (curr_end > end_ptr) {
curr_end -= SIZE;
}
};




This code provides entire available space in ring-buffer, socket fills it, and ring buffer then updates the beginning and end pointers.

This isn't tested code, and it uses raw pointers which can be avoided entirely, but with scatter/gather you can solve the problem of filling the buffer trivially.

Reading is then trivial, what you do need to solve is parsing this buffer, but that's not platform dependant.

Share this post


Link to post
Share on other sites
That's a different solution, though (in fact, solution 1 from my list). It does not actually provide a contiguous virtual address view of the buffer at the point of wrap-around; instead, it uses scatter/gather to wrap around in the buffer. The user of the buffer still needs to take care when the data he wants wraps around.

Share this post


Link to post
Share on other sites

This topic is 3806 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this