Sign in to follow this  

Light-weight Network Messaging Library

This topic is 4320 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

In my "copious" spare time, I've put together a small, light-weight network messaging library. Right now, it supports sending, queuing and receiving messages (sized blocks of data) between machines, using TCP or UDP. It adds chunked message semantics (rather than stream) over TCP, and deals with receiving partial messages (or coalesced messages). It does not (yet) add reliability to UDP. The library is MIT license (yay!) and currently only works on Windows (boo!). You can read about it (and download it) here. The name of it is "Etwork". The intent is to make it simple to use first, robust second, efficient third, and there is no fourth intent. My hope is that I can grow the library to: - work on Linux (and possibly other nix-en) - work out any un-known bugs that lurk in there - possibly add reliability semantics on top of UDP - possibly add entity-to-entity messaging as another library on top - possibly add matchmaking and NAT introduction as another library on top (You may already be familiar with libintroduce, which does a few of these things, but is more heavy-weight) What do I want from you? - can you understand the documentation, samples and headers? - can you actually run the samples? - can you find any bugs? - do you have any specific suggestions for changes to the code? - do you have any specific suggestions for changes to the API? I don't particularly want comments like "your C++ is not UML compliant" or "those braces sure do suck"; please keep it civil :-) If you want to contribute patches, real open source style, that'd be cool, though.

Share this post


Link to post
Share on other sites
Quote:
Original post by hplus0603
- can you understand the documentation, samples and headers?


I've never cared for the Doxygen styed documentation, but yours is pretty simple and straightforward, so it's not that big of a deal. Maybe it's just because I'm so used to these docs that I've never learned how to properly 'apprciate' other forms [wink]. Aside from my personal ramblings, the docs look very descriptive and show a good amount of effort put into them, so good job on that. It's been a while since I've made a fully documented project with Dox, but I rem it does take a good portion of time, but is easy to maintain and keep up to date.

Quote:
- can you actually run the samples?


Yes, and it has me interested enough to look into your library to perhaps use in future projects as well. Of course that was with a quick glancing though of the code, but being a network noob, my weakest area of expertise, I do look for things that make life easier, and this looks a bit easier than some of the other libs I've looked at.

Quote:
- can you find any bugs?


Not yet, but if I do, I'll let you know...for a price [smile]

Quote:
- do you have any specific suggestions for changes to the code?


I guess it's always an issue of how much to comment, I myself am considered an over commenter, but for me, I can easily visualize code when it has a good enough amount of comments. That way if I forget about something, I'll always be reminded. Anyways though, I'd like to see more comments in your code as to what each section does, not each line because the code is fairly explanatory, but I'd be nice to read things such as: "If the user specified a port to run on" for the code segment

char const * param = strstr( lpCmdLine, "port=" );
if( param ) {
dialogPort = atoi( param+5 );
}

Rather than just look and guess. Like I said, you don't need a lot of comments, but right now, the code is definitly lacking in them. Oh and I should clarify - the 'code' as in the .cpp files for the demo programs client/server.

Quote:
- do you have any specific suggestions for changes to the API?


Not yet, but if I start to use it, I'd definitly leave opinions on it. It does look rather intuitive at glance though.

Quote:
I don't particularly want comments like "your C++ is not UML compliant" or "those braces sure do suck"; please keep it civil :-) If you want to contribute patches, real open source style, that'd be cool, though.


Those braces sure do suck! [lol] j/k [grin] Overall, this library looks very great and promising, thanks for releasing this to us. I know I probabally won't get around to trying it out until this week with school starting and all, but it is something that definitly does have my attention! If I could come up with any improvments, I'd definitly shoot them your way, but being a network noob, I doubt I'd mess with the core library and rather work with additional wrappers or prefab components. So, hope you keep on working on this, and keep us updated with progress and changes. Oh and thanks for the MIT license [smile]

Share this post


Link to post
Share on other sites
A few questions/ideas: (Don't mind if I play the newb for a minute or two...)

ISocketManager::Connect() documentation states that you can supply a domain name for the addy. In that case, is the gethostbyname() call synchronous or in it's own thread? This is a call that can often stall. I'd make sure you let us know in the docs either way :)

So, after passing listening sockets to ISocketManager::Poll(), we get back the accepted sockets by calling ISocketManager::Accept()? Seems logical enough but never really clearly stated as such. - ok after re-reading the example code I see that isn't how it works :P

How about the documented Buffer class? Is that in use inside the other classes or something we need to link into to get the formatted message capability?

Well that is my 2 cents, just looking for some clarifications. Looks good.

Share this post


Link to post
Share on other sites
Looks good!

I know it's supposed to be a messaging library, but maybe it's a nice idea to generalize the chatserver/client to an abstract client/server class which server/client apps can use as parent and override parts for their specific needs?

Reliable UDP sounds like a nice feature. Im curious, will you be adopting a existing protocol for that (like in Enet) or will it be your own design?

Share this post


Link to post
Share on other sites
Thanks for the feedback!

In no particular order:

- The buffer class is not necessary to use; it's internal. You _can_ use it if you have a need for it, though.

- I'll work on the Doxygen templates to clean it up a bit sometime later. I agree that the default look is... hard on the eyes.

- Reliable UDP would likely be my own design, and likely targeted towards mostly-fixed-datarate games, rather than something like bulk-throughput downloads.

- For a generic client/server, that's something I feel should go in a module on top of the messaging sockets. I e, there might be an IServer interface and an IClient interface (and, possibly, an IPeer interface).

- Yes, after calling poll(), you should call accept() to get back any newly accepted sockets. Poll() returns only sockets you already know about. I'll clear this up in docs.

- Yes, connect() is blocking unless you use dotted-decimal notation. The Etwork Home Page explains this a little better. I'll look into getting that write-up onto the main page of the Doxygen.

- Commenting the samples??! Next you'll want me to justify my use of goto, too?? :-)
Yeah, I've focused on the headers for the comments this time around.

I've got enough to go on for getting me to 0.2. Look for it sometime this spring ;-)

Share this post


Link to post
Share on other sites
I would link the phrase "MIT license" to a page describing the license.

It would make the client code simpler if you could attach read/write/connect/etc event callbacks to the socket object rather than having to iterate through the list of sockets yourself. I didn't download the files and the page doesn't describe the socket interface in detail so perhaps this is already possible.

If you do add reliable UDP it would be nice to have host app callouts to tune it and/or give user notification that things are going wonky.

Share this post


Link to post
Share on other sites
Quote:
- can you understand the documentation, samples and headers?

No problem.
Quote:
- do you have any specific suggestions for changes to the code?

Your error reporting scheme returns a string and dumps to the console. There is already a lot of app-level stuff going to the console, yours is going to get lost. There is no severity level implied, nor anyway for an app to determine what if the error puts the library and the system in an unstable state of if this just that the connection was dropped. Would be better to return an error code that can be analyzed by the app; I can forego strings altogether. That code can be a combinaison of bits that helps corner the area of the code being triggered, the severity level, and then the specific error code. Example:


//---- ERROR AREA
#define ETWORK_ERR_SETTINGS 0x7000 // Something in your settings is wrong, dude.
#define ETWORK_ERR_INIT 0x6000 // Something failed during initialization
#define ETWORK_ERR_ACCEPT 0x8000 // ... while waiting for a remote connection
#define ETWORK_ERR_CONNECT 0x9000 // ... while trying to connect
#define ETWORK_ERR_MEMALLOC 0xA000 // ... not enough RAM
#define ETWORK_ERR_WSOCK 0xB000 // You need to upgrade to WinSock 2

//---- ERROR LEVEL
#define ETWORK_LVL_FATAL 0x0F00 // System in unstable state
#define ETWORK_LVL_ERROR 0x0A00 // Errr....
#define ETWORK_LVL_WARNING 0x0800 // Dumb ass. I corrected your invalid values
#define ETWORK_LVL_DEBUG 0x0400 // Some debug info
#define ETWORK_LVL_STATS 0x0300 // Some stats

// in code somewhere... Too many sockets already opened.
return( ETWORK_ERR_ACCEPT | ETWORK_LVL_ERROR | 27 );

You can look at my UDP library here ( http://members.gamedev.net/cbenoi1/r_1_5_0.zip ) on some ideas around this. I also implemented filtering so that I can select to see only the debugging info or only the fatal errors.


Hope this helps.

-cb

Share this post


Link to post
Share on other sites
Thanks for the additional feedback!

Quote:
It would make the client code simpler if you could attach read/write/connect/etc event callbacks to the socket object rather than having to iterate through the list of sockets yourself.


I did that in my last project (libintroduce), but it has draw-backs. The first is that I either need to add a dependency for signals/slots/events, or I have to declare a separate interface that you implement to get the status. The second is that call chains can get pretty deep -- I actually explicitly went back to the read/write interface, because that's what sockets and files normally have, so it's usually easier for newer programmers to understand.

That being said, notifications of events (and errors) would be very easy to add by using a simple callback interface, so it could slip into an "advanced" section somewhere.

Quote:
There is no severity level implied, nor anyway for an app to determine what if the error puts the library and the system in an unstable state of if this just that the connection was dropped.


There are two kinds of errors, in my mind: errors that mean you must stop dealing with the connection, and errors you can ignore (such as EWOULDBLOCK when you use queuing). The former will close the ISocket (so it'll look like the other side dropped the connection); the latter is just not reported.

It would be very bad for an API to leave either the networking system or the application in a state such that you cannot proceed. The design is such that, I hope, this will never happen. Errors are per socket. An error on a connection means that connection is dropped, and you're told about it. Does it really matter WHY that connection was dropped? What would you do different if you knew?

The only system-wide error would be if you can't even create the networking system, which is reported by returning NULL from the creation function. I agree that this means you can't distinguish "can't bind to port" from "can't initialize winsock," but the latter doesn't usually happen :-)

As for tweaking the behavior of the library, you provide a number of settings (optionally) when creating the networking subsystem (of which you can create more than one); that would be a place to tweak realiable-UDP behavior. If I ever take it that far, that is.

OK, this is more food for thought. Thanks! It is especially true that errors are per-socket, but are returned per-network; that's obviously not great.

Share this post


Link to post
Share on other sites
Allright, this week-end gave us:

- optionally more specific error reporting
- optionally get a callback for socket activity
- documented samples better
- attempted to clean up doxygen documentation

Check it out at www.mindcontrol.org/etwork.

Share this post


Link to post
Share on other sites
I like the error reporting now. It allows me to have both a Debug that can OutputToDebugString what I need as I debug the app, but the Retail version can stick it to the NULL notification interface and run at full speed.

One thing that didn't strike me earlier is the ISocketManager interface. In some way, this is what Steve MacGuire would call a "candy-machine" interface. Depending on the mode you put it in, only a subset of the API is actually useful. I don't think it's a big deal; the whole socket API is a big candy-machine interface anyway... But repackaging the interface as IClientSocketManager and IServerSocketManager with the proper API subset could alleviate the potential confusion for novices.

-cb

Share this post


Link to post
Share on other sites
That's an interesting observation; I had considered splitting accepting out to another interface, but let it stay in a single interface to make it more approachable.

It is quite conceivable that you can call connect() on an "accepting" ISocketManager, and receive a socket that lets you talk to another machine. The only function that you "won't" call if you're not accepting is "accept()" -- although it's perfectly legal to call it; you'll just get no sockets back.

I suppose what you're suggesting is that you create the basic interface first, and then you create an "accepting" interface off of the basic interface, if you want to accept incoming connections.

There's also something to be said for keeping the interface close to the internal structure. When the interface is created for UDP communication, there will always be an open socket, which will be used for accepting if accepting is true. The management of this socket, vs the unknown timing of when the user may create the accepting interface, leads me to believe that the current design is simpler.

It's a valid observation, though; thanks!

Share this post


Link to post
Share on other sites
Umm, I got a problem with your library.
When I try to compile your example it doesn't find do_something_with_active_socket.
I suppose that there is no function that does something with the active socket, but I wonder, what to do? :S
I'm a total newbie to networking, and I don't know very much about sockets and connections and stuff. :P

Share this post


Link to post
Share on other sites
The example code in the documentation shows where you insert your own code.

If you want to compile the actual examples, look in the src/chatserver and src/chatclient directories. (Those are also buildable using the included MSVC project file).

Share this post


Link to post
Share on other sites

This topic is 4320 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this