Hello,

I hope this is the right forum.  A product I am working on will have a computer with camera attached that will be taking real-time video in grayscale.  The frames will need to be sent over a fast network to an "image processing server" that will have multiple GPUs.  Each frame will be assigned to the next available GPU for processing.  Then the output will be sent to another computer in the network.  The idea is for the frames to be processed in real time for computer vision type application.

I don't have that much experience with network programming, but I think this should be pretty simple?  Would I just create a basic client/server program using TCP/IP or UDP to send the image byte data, and a small amount of meta data (image dimensions, frame #, time stamp, etc)?  Speed is the most important thing so that we can process in real-time.  Any suggestion on protocols and design?

Http-like (or actual http) protocol would be a good choice here, as you could send the metadata in the headers and the image data in the request body.

Thanks for the replies.  I did a little more research and was curious where the socket APIs fit in.  Those seem to be built on top of transport protocols, or are sockets still considered too low-level?  I like that sockets are mostly portable API, as we may use Linux.

HTTP sounds simple and that it could work.  I found this MS Rest API https://msdn.microsoft.com/en-us/library/jj950081.aspx and they even have an example of pushing a chunk of data to the server, which is pretty much what I need.  I have a question though for HTTP.  Can the server push data to a specific client, or must the client put in a request to get the image processing output?

So basically, I'm leaning towards sockets (TCP/IP) or HTTP, as they seem like the simplest for what I need to do.  Would one be significantly faster that the other?  Or is HTTP likely using sockets under the hood?  

Video compression would create latency and quality loss. 

Are all of the cameras and processing cards in the same local network (such as in a car or in a closet) and do you control all the traffic on the systems?

If you control all the traffic, then TCP and TCP-based protocols (such as HTTP) could work fine, because packet loss will be very rare, and your "real time" requirements won't be impacted.

If the system requires some kind of shared link -- internet uplink, shared switch with other application data, office LAN, or what have you -- then there is some risk that TCP-based protocols will end up sometimes "hiccup-ing" because of packet loss and re-send intervals. If you'd rather just have the next frame of data, than wait for the previous frame, when packet loss happens, then you need to use some protocol on top of UDP.

When going over UDP, there are also encodings that allow you to send a few more packets per frame, but recover from some number of lost packets. (Look into self-correcting codes, self-healing protocols, and so forth.)

The rest of your problem isn't really possible to given good advice about unless we have a lot more information about your specific system hardware and requirements.

Don't handle the transport by yourself.

Use a message queue. That will take care of managing connections, fail overs, etc...

It will allow you to do such things, as take a server off line and add more servers to a role.

You can even dynamically change the roles of different nodes in your cluster.

But one serious remark, regardless of what you choose to do:

If a frame has multiple stages of processing, it might be wiser to do all of the stages on a single machine.

For exaple: If you are processing 10 frames on 10 machines, if you dedicate each machine to a different stage, you are transmitting each frame 10 times = 100 IO operations.

However, if instead you send each frame to a separate machine, and keep it there, you will only be doing 10 I/O operations.

Your run time should remain the same because you are still processing 10 frames in parallel, but your network will be much less stressed.

Also, you will utilize your machines better in the case that one of the steps takes shorter to do. If each machine is dedicated to a specific step (instead of being dedicated to a frame): Your entire pipeline will only be as efficient as your slowest step.