Light-weight Network Messaging Library
In my "copious" spare time, I've put together a small, light-weight network messaging library.
Right now, it supports sending, queuing and receiving messages (sized blocks of data) between machines, using TCP or UDP. It adds chunked message semantics (rather than stream) over TCP, and deals with receiving partial messages (or coalesced messages). It does not (yet) add reliability to UDP.
The library is MIT license (yay!) and currently only works on Windows (boo!). You can read about it (and download it) here. The name of it is "Etwork". The intent is to make it simple to use first, robust second, efficient third, and there is no fourth intent.
My hope is that I can grow the library to:
- work on Linux (and possibly other nix-en)
- work out any un-known bugs that lurk in there
- possibly add reliability semantics on top of UDP
- possibly add entity-to-entity messaging as another library on top
- possibly add matchmaking and NAT introduction as another library on top
(You may already be familiar with libintroduce, which does a few of these things, but is more heavy-weight)
What do I want from you?
- can you understand the documentation, samples and headers?
- can you actually run the samples?
- can you find any bugs?
- do you have any specific suggestions for changes to the code?
- do you have any specific suggestions for changes to the API?
I don't particularly want comments like "your C++ is not UML compliant" or "those braces sure do suck"; please keep it civil :-)
If you want to contribute patches, real open source style, that'd be cool, though.
Quote:Original post by hplus0603
- can you understand the documentation, samples and headers?
I've never cared for the Doxygen styed documentation, but yours is pretty simple and straightforward, so it's not that big of a deal. Maybe it's just because I'm so used to these docs that I've never learned how to properly 'apprciate' other forms [wink]. Aside from my personal ramblings, the docs look very descriptive and show a good amount of effort put into them, so good job on that. It's been a while since I've made a fully documented project with Dox, but I rem it does take a good portion of time, but is easy to maintain and keep up to date.
Quote:- can you actually run the samples?
Yes, and it has me interested enough to look into your library to perhaps use in future projects as well. Of course that was with a quick glancing though of the code, but being a network noob, my weakest area of expertise, I do look for things that make life easier, and this looks a bit easier than some of the other libs I've looked at.
Quote:- can you find any bugs?
Not yet, but if I do, I'll let you know...for a price [smile]
Quote:- do you have any specific suggestions for changes to the code?
I guess it's always an issue of how much to comment, I myself am considered an over commenter, but for me, I can easily visualize code when it has a good enough amount of comments. That way if I forget about something, I'll always be reminded. Anyways though, I'd like to see more comments in your code as to what each section does, not each line because the code is fairly explanatory, but I'd be nice to read things such as: "If the user specified a port to run on" for the code segment
char const * param = strstr( lpCmdLine, "port=" ); if( param ) { dialogPort = atoi( param+5 ); }Rather than just look and guess. Like I said, you don't need a lot of comments, but right now, the code is definitly lacking in them. Oh and I should clarify - the 'code' as in the .cpp files for the demo programs client/server.
Quote:- do you have any specific suggestions for changes to the API?
Not yet, but if I start to use it, I'd definitly leave opinions on it. It does look rather intuitive at glance though.
Quote:I don't particularly want comments like "your C++ is not UML compliant" or "those braces sure do suck"; please keep it civil :-) If you want to contribute patches, real open source style, that'd be cool, though.
Those braces sure do suck! [lol] j/k [grin] Overall, this library looks very great and promising, thanks for releasing this to us. I know I probabally won't get around to trying it out until this week with school starting and all, but it is something that definitly does have my attention! If I could come up with any improvments, I'd definitly shoot them your way, but being a network noob, I doubt I'd mess with the core library and rather work with additional wrappers or prefab components. So, hope you keep on working on this, and keep us updated with progress and changes. Oh and thanks for the MIT license [smile]
A few questions/ideas: (Don't mind if I play the newb for a minute or two...)
ISocketManager::Connect() documentation states that you can supply a domain name for the addy. In that case, is the gethostbyname() call synchronous or in it's own thread? This is a call that can often stall. I'd make sure you let us know in the docs either way :)
So, after passing listening sockets to ISocketManager::Poll(), we get back the accepted sockets by calling ISocketManager::Accept()? Seems logical enough but never really clearly stated as such. - ok after re-reading the example code I see that isn't how it works :P
How about the documented Buffer class? Is that in use inside the other classes or something we need to link into to get the formatted message capability?
Well that is my 2 cents, just looking for some clarifications. Looks good.
ISocketManager::Connect() documentation states that you can supply a domain name for the addy. In that case, is the gethostbyname() call synchronous or in it's own thread? This is a call that can often stall. I'd make sure you let us know in the docs either way :)
So, after passing listening sockets to ISocketManager::Poll(), we get back the accepted sockets by calling ISocketManager::Accept()? Seems logical enough but never really clearly stated as such. - ok after re-reading the example code I see that isn't how it works :P
How about the documented Buffer class? Is that in use inside the other classes or something we need to link into to get the formatted message capability?
Well that is my 2 cents, just looking for some clarifications. Looks good.
Looks good!
I know it's supposed to be a messaging library, but maybe it's a nice idea to generalize the chatserver/client to an abstract client/server class which server/client apps can use as parent and override parts for their specific needs?
Reliable UDP sounds like a nice feature. Im curious, will you be adopting a existing protocol for that (like in Enet) or will it be your own design?
I know it's supposed to be a messaging library, but maybe it's a nice idea to generalize the chatserver/client to an abstract client/server class which server/client apps can use as parent and override parts for their specific needs?
Reliable UDP sounds like a nice feature. Im curious, will you be adopting a existing protocol for that (like in Enet) or will it be your own design?
Thanks for the feedback!
In no particular order:
- The buffer class is not necessary to use; it's internal. You _can_ use it if you have a need for it, though.
- I'll work on the Doxygen templates to clean it up a bit sometime later. I agree that the default look is... hard on the eyes.
- Reliable UDP would likely be my own design, and likely targeted towards mostly-fixed-datarate games, rather than something like bulk-throughput downloads.
- For a generic client/server, that's something I feel should go in a module on top of the messaging sockets. I e, there might be an IServer interface and an IClient interface (and, possibly, an IPeer interface).
- Yes, after calling poll(), you should call accept() to get back any newly accepted sockets. Poll() returns only sockets you already know about. I'll clear this up in docs.
- Yes, connect() is blocking unless you use dotted-decimal notation. The Etwork Home Page explains this a little better. I'll look into getting that write-up onto the main page of the Doxygen.
- Commenting the samples??! Next you'll want me to justify my use of goto, too?? :-)
Yeah, I've focused on the headers for the comments this time around.
I've got enough to go on for getting me to 0.2. Look for it sometime this spring ;-)
In no particular order:
- The buffer class is not necessary to use; it's internal. You _can_ use it if you have a need for it, though.
- I'll work on the Doxygen templates to clean it up a bit sometime later. I agree that the default look is... hard on the eyes.
- Reliable UDP would likely be my own design, and likely targeted towards mostly-fixed-datarate games, rather than something like bulk-throughput downloads.
- For a generic client/server, that's something I feel should go in a module on top of the messaging sockets. I e, there might be an IServer interface and an IClient interface (and, possibly, an IPeer interface).
- Yes, after calling poll(), you should call accept() to get back any newly accepted sockets. Poll() returns only sockets you already know about. I'll clear this up in docs.
- Yes, connect() is blocking unless you use dotted-decimal notation. The Etwork Home Page explains this a little better. I'll look into getting that write-up onto the main page of the Doxygen.
- Commenting the samples??! Next you'll want me to justify my use of goto, too?? :-)
Yeah, I've focused on the headers for the comments this time around.
I've got enough to go on for getting me to 0.2. Look for it sometime this spring ;-)
I would link the phrase "MIT license" to a page describing the license.
It would make the client code simpler if you could attach read/write/connect/etc event callbacks to the socket object rather than having to iterate through the list of sockets yourself. I didn't download the files and the page doesn't describe the socket interface in detail so perhaps this is already possible.
If you do add reliable UDP it would be nice to have host app callouts to tune it and/or give user notification that things are going wonky.
It would make the client code simpler if you could attach read/write/connect/etc event callbacks to the socket object rather than having to iterate through the list of sockets yourself. I didn't download the files and the page doesn't describe the socket interface in detail so perhaps this is already possible.
If you do add reliable UDP it would be nice to have host app callouts to tune it and/or give user notification that things are going wonky.
Quote: - can you understand the documentation, samples and headers?
No problem.
Quote: - do you have any specific suggestions for changes to the code?
Your error reporting scheme returns a string and dumps to the console. There is already a lot of app-level stuff going to the console, yours is going to get lost. There is no severity level implied, nor anyway for an app to determine what if the error puts the library and the system in an unstable state of if this just that the connection was dropped. Would be better to return an error code that can be analyzed by the app; I can forego strings altogether. That code can be a combinaison of bits that helps corner the area of the code being triggered, the severity level, and then the specific error code. Example:
//---- ERROR AREA#define ETWORK_ERR_SETTINGS 0x7000 // Something in your settings is wrong, dude.#define ETWORK_ERR_INIT 0x6000 // Something failed during initialization#define ETWORK_ERR_ACCEPT 0x8000 // ... while waiting for a remote connection#define ETWORK_ERR_CONNECT 0x9000 // ... while trying to connect#define ETWORK_ERR_MEMALLOC 0xA000 // ... not enough RAM#define ETWORK_ERR_WSOCK 0xB000 // You need to upgrade to WinSock 2//---- ERROR LEVEL#define ETWORK_LVL_FATAL 0x0F00 // System in unstable state#define ETWORK_LVL_ERROR 0x0A00 // Errr....#define ETWORK_LVL_WARNING 0x0800 // Dumb ass. I corrected your invalid values#define ETWORK_LVL_DEBUG 0x0400 // Some debug info#define ETWORK_LVL_STATS 0x0300 // Some stats// in code somewhere... Too many sockets already opened.return( ETWORK_ERR_ACCEPT | ETWORK_LVL_ERROR | 27 );You can look at my UDP library here ( http://members.gamedev.net/cbenoi1/r_1_5_0.zip ) on some ideas around this. I also implemented filtering so that I can select to see only the debugging info or only the fatal errors.
Hope this helps.
-cb
Thanks for the additional feedback!
I did that in my last project (libintroduce), but it has draw-backs. The first is that I either need to add a dependency for signals/slots/events, or I have to declare a separate interface that you implement to get the status. The second is that call chains can get pretty deep -- I actually explicitly went back to the read/write interface, because that's what sockets and files normally have, so it's usually easier for newer programmers to understand.
That being said, notifications of events (and errors) would be very easy to add by using a simple callback interface, so it could slip into an "advanced" section somewhere.
There are two kinds of errors, in my mind: errors that mean you must stop dealing with the connection, and errors you can ignore (such as EWOULDBLOCK when you use queuing). The former will close the ISocket (so it'll look like the other side dropped the connection); the latter is just not reported.
It would be very bad for an API to leave either the networking system or the application in a state such that you cannot proceed. The design is such that, I hope, this will never happen. Errors are per socket. An error on a connection means that connection is dropped, and you're told about it. Does it really matter WHY that connection was dropped? What would you do different if you knew?
The only system-wide error would be if you can't even create the networking system, which is reported by returning NULL from the creation function. I agree that this means you can't distinguish "can't bind to port" from "can't initialize winsock," but the latter doesn't usually happen :-)
As for tweaking the behavior of the library, you provide a number of settings (optionally) when creating the networking subsystem (of which you can create more than one); that would be a place to tweak realiable-UDP behavior. If I ever take it that far, that is.
OK, this is more food for thought. Thanks! It is especially true that errors are per-socket, but are returned per-network; that's obviously not great.
Quote:It would make the client code simpler if you could attach read/write/connect/etc event callbacks to the socket object rather than having to iterate through the list of sockets yourself.
I did that in my last project (libintroduce), but it has draw-backs. The first is that I either need to add a dependency for signals/slots/events, or I have to declare a separate interface that you implement to get the status. The second is that call chains can get pretty deep -- I actually explicitly went back to the read/write interface, because that's what sockets and files normally have, so it's usually easier for newer programmers to understand.
That being said, notifications of events (and errors) would be very easy to add by using a simple callback interface, so it could slip into an "advanced" section somewhere.
Quote:There is no severity level implied, nor anyway for an app to determine what if the error puts the library and the system in an unstable state of if this just that the connection was dropped.
There are two kinds of errors, in my mind: errors that mean you must stop dealing with the connection, and errors you can ignore (such as EWOULDBLOCK when you use queuing). The former will close the ISocket (so it'll look like the other side dropped the connection); the latter is just not reported.
It would be very bad for an API to leave either the networking system or the application in a state such that you cannot proceed. The design is such that, I hope, this will never happen. Errors are per socket. An error on a connection means that connection is dropped, and you're told about it. Does it really matter WHY that connection was dropped? What would you do different if you knew?
The only system-wide error would be if you can't even create the networking system, which is reported by returning NULL from the creation function. I agree that this means you can't distinguish "can't bind to port" from "can't initialize winsock," but the latter doesn't usually happen :-)
As for tweaking the behavior of the library, you provide a number of settings (optionally) when creating the networking subsystem (of which you can create more than one); that would be a place to tweak realiable-UDP behavior. If I ever take it that far, that is.
OK, this is more food for thought. Thanks! It is especially true that errors are per-socket, but are returned per-network; that's obviously not great.
Allright, this week-end gave us:
- optionally more specific error reporting
- optionally get a callback for socket activity
- documented samples better
- attempted to clean up doxygen documentation
Check it out at www.mindcontrol.org/etwork.
- optionally more specific error reporting
- optionally get a callback for socket activity
- documented samples better
- attempted to clean up doxygen documentation
Check it out at www.mindcontrol.org/etwork.
Looks excellent. I like the format of the html documentation :)
This is now on the top of my list of network libraries, given the need for one arises.
This is now on the top of my list of network libraries, given the need for one arises.
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement