Sign in to follow this  
WXY

Designing a new protocol

Recommended Posts

WXY    122
Hi there GD forums. I've recently noticed that both TCP and RUDP may not suit the needs of future real time applications and the needs of a project which I am currently attempting to pursue. The problems I see: - TCP assumes all errors as signs of congestion and in many cases it over reacts (doubling timeout, halving window size, resetting to 1 segment slow start is pretty drastic) - TCP's sliding window size is preventing it from keeping pipes full, despite options to shift the window - TCP's delayed acknowledgment policy prolongs RTT - TCP enforces sequence of arrival, making unrelated messages unnecessarily wait for retransmissions thus needlessly promotes lag - TCP has no sense of QoS, which also promotes lag as even old messages must be retransmitted - RUDP don't implement congestion control, which reduces network resource utilization - RUDP does not offer in built flow control, which makes the user of libraries that implements the protocol fairly difficult to use - RUDP don't implement sequence enforcement, which is sometimes desired - No libraries which I have found for either protocol seems to offer good OO interfaces, thus making the implementers' lives difficult My idea: I've decided to define a transport protocol and make an implementation for it in C++. The protocol at the moment is referred by me as MTP (message transaction protocol). It has 4 variants which addresses the various problems stated in the above section. The framework which will be used to implement the protocols will actually separate communication policy from the code that interfaces with the protocol that it is built on, so thus effectively the 4 variants will be implemented as psudoprotocols, which also enable the user much greater control over the behavior of the framework. Note that the user may also implement his/her own psudoprotocol on top of the framework with a 64 bit header overhead. Of course, the design document for the protocol so far is much longer than my description above, and I have omited many aspects of the protocol and its variants, so if you find the description vague, then thats probably why. If anyone is interested I am willing to provide an interface level UML of the library which will implement MTP. What I am after: Before I go jumping into the project I want to have a sanity check of what I am doing, and see if there are already libraries that permits my desired solution. Thanks, WXY

Share this post


Link to post
Share on other sites
Antheus    2409
Quote:
Original post by WXY
- No libraries which I have found for either protocol seems to offer good OO interfaces, thus making the implementers' lives difficult


What about the is "not good" and makes "lives difficult"?

Note that IP-based protocols are typically accessed via Berkeley sockets API. There are alternate APIs provided (IOCP, AIO, kqueue) and different libraries (libasync, asio) and various utility frameworks. But all of these just provide different API to the very same protocol.

There's also various libraries such as spread which are designed around different concepts, especially WRT to message passing.

Quote:
Before I go jumping into the project I want to have a sanity check of what I am doing, and see if there are already libraries that permits my desired solution.


Have you read the forum FAQ, especially the Gaffer's article on reliable UDP and list of existing socket APIs?

[Edited by - Antheus on November 22, 2008 7:17:07 AM]

Share this post


Link to post
Share on other sites
WXY    122
Quote:
Original post by elFarto
You should look at SCTP.


STCP uses slow start & congestion avoidance similar to the algorithm used in TCP, however it is just adapted for Multihoming (which translates only into meaningless overhead on the typical 1 connection consumer). This destroys potentially provided QoS properties of lower layers, which is unacceptable for the type of application MTP is aiming to facilitate. Also from its specifications document I cannot find anything about its ability to deliver low overhead, unreliable messages and per stream out of sequence messages.

Though it seems good for load balancing networks, thanks for introducing me to this protocol, perhaps it will come in handy one day.


Quote:
Original post by Antheus
What about the is "not good" and makes "lives difficult"?

Note that IP-based protocols are typically accessed via Berkeley sockets API. There are alternate APIs provided (IOCP, AIO, kqueue) and different libraries (libasync, asio) and various utility frameworks. But all of these just provide different API to the very same protocol.


Despite their versatility, Berkeley sockets are created and maintained to be used in a non OO language, C. An experienced programmer usually writes adapter objects to compensate this problem, however without predefined code, even experienced coders are prone to making poor choices. For example making a highly coupled communication object with no real defined set of abstract responsibilities (thus poor cohesion). Also as applications becomes more chatty while CPU speeds stands still (but with more cores) its looking ever better to tap into the wonderful but dangerous world of multi threading. In the end, do you REALLY want to merge berkeley sockets with OO design principals as well as threads for every project that you do?

The framework that I am planing to make, implements MTP as nothing but a communication policy through the "strategy" GoF behavioral pattern, the rest of the framework provides its user with a nice abstraction of the communication mechanism (you guessed it, Berkeley sockets, and threads). The framework will provide an optimized set of objects that asynchronously multiplex inbound and outbound traffic to individual stream objects in accordance to user defined policies, whos life spans are that of connection objects. This which prevents the need for the introduction of a highly coupled, poorly cohesive communication object.


Quote:
Original post by Antheus
Have you read the forum FAQ, especially the Gaffer's article on reliable UDP and list of existing socket APIs


As I have mentioned, RUDP (reliable UDP) provides too little abstractions over the existing UDP protocol. Most programmers who seeks alternatives to TCP are usually migrating because they dislike how TCP is unsuited for real time applications, and they are forced back to TCP due to the lack of essential services in UDP/RUDP. Basically in a nut shell, with TCP and UDP, you are either forced to have too much done for you or have nearly nothing at all done for you.

Share this post


Link to post
Share on other sites
Antheus    2409
Quote:
Original post by WXY

Despite their versatility, Berkeley sockets are created and maintained to be used in a non OO language, C.
And many real-time systems can use nothing but C, if you're lucky. Often you're stuck with vendor-specific compiler.

Quote:
An experienced programmer usually writes adapter objects to compensate this problem, however without predefined code, even experienced coders are prone to making poor choices.
Wouldn't that make them inexperienced, by the very definition of word? Why are *your* choices better?

Quote:
For example making a highly coupled communication object with no real defined set of abstract responsibilities (thus poor cohesion). Also as applications becomes more chatty while CPU speeds stands still (but with more cores) its looking ever better to tap into the wonderful but dangerous world of multi threading. In the end, do you REALLY want to merge berkeley sockets with OO design principals as well as threads for every project that you do?


If CPU is your problem, then threads are likely to be one per core, or one per process. Using more or using them arbitrarily to offload processing is possible to a certain degree in IO bound systems.

Quote:
The framework that I am planing to make, implements MTP as nothing but a communication policy through the "strategy" GoF behavioral pattern, the rest of the framework provides its user with a nice abstraction of the communication mechanism (you guessed it, Berkeley sockets, and threads). The framework will provide an optimized set of objects that asynchronously multiplex inbound and outbound traffic to individual stream objects in accordance to user defined policies, whos life spans are that of connection objects. This which prevents the need for the introduction of a highly coupled, poorly cohesive communication object.


Whether it's gang of four or party of five, I still don't understand which problems you're trying to solve.

There are at least three frameworks which solve the above. ACE, libasync and boost::asio.

ACE is proven in everything from games to military software. libasync is a nice C-based portable reactor-like API. Asio is boost-like proactor API. All these can be considered equivalent in functionality.

In what way are those insufficient for problems of networking? There's a lot of mixups going on as well. A protocol is a contract, threading models are platform-specific, network API as well, and application design is domain specific. I fail to see how it would be possible to wrap this into a single framework labeled as "protocol".

Quote:
In the end, do you REALLY want to merge berkeley sockets with OO design principals as well as threads for every project that you do?


For portable C++ I prefer boost asio. For linux-centric applications I'd use libasync. Under Java or C# I'd use platform API, Twisted for Python, the lower the better, avoiding various wrappers.

For application logic, I prefer to avoid low-level sockets, and will prefer whatever framework-du-jour is if applications are not intended for heavy load. Then there's always the enterprise-grade solutions for "serious" projects, but the choices of framework there are made at corporate level, so there's no choice for developer.

In the end, networking is just a tiny aspect of finding suitable architecture for given problem, and simply analyzing the problem and gathering the requirements will take so much more time, that the choice of library is just logical consequence.

That's what I'm trying to find out - what problems does your framework solve, so that I may consider it when choosing network API for next project.

Quote:
Most programmers who seeks alternatives to TCP are usually migrating


Who are these programmers?

For example, how will your framework solve the problem of a communication protocol which was designed to operate on satellites with 4kb of memory, yet has since been scaled to gigabit networks with 1000Hz data update rates? That doesn't break when you tunnel it through ssh over transatlantic link? That has been tested and proven in field over 20 years.

Or how will your framework handle an embedded linux platform for IPTV that needs to stream data as directly from network into decoder's buffers, and perform flow control in video stream or in image space? For example, things like patching the linux kernel to achieve zero-copy transfers and avoid extra ip lookups?

How will it support message prioritization based on AOI and predefined priorities (stuff like Quake or Half-life networking)? How will it deal with congestion, and why are existing designs completely unsuitable?

I'm just trying to move away from enterpresy weasel words and figure out what I will actually get, and why I should move away from libraries which served me well in the past without any major problems.

Quote:
real time applications


Why use C++ when it will make porting to vxWorks and QNX so much harder. Why trouble yourself with OO when CodeWarrior is known to be one of least compliant C++ compilers. Why use sockets and threads when you can use one of soft, or perhaps even hard real-time Linux kernels. And when it comes to hard, how will you integrate those facilities?

Or better yet, what systems do you classify as real-time?

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this