I am encountering some performance issues with my SERVER<->CLIENTs tcp communication and I think it is related to how I perform my communication. Specifically - the way it works today - I have tried the following two approaches: 1- Foreach Client the main form launches (create and start) a thread per client (so many threads), each thread performs the tcpclient connection, sends the data, and then close. 2- Main thread launches a single thread (create and start) which will loop through the list of clients (tcpclient connection, send data, close) and send the data In both cases I use TCPClient & NetworkStream to perform the actual transfer (so create a TCPClient to the Client with the corresponding IP and PORT and then use the Network Stream to send the data itself). As mentioned this seems to work a little slow so I was trying to look at ways of improving the code (as there is a LOT of information passed between the server and the 1-100 clients depending)... So I did some reading and found the concepts of "BROADCAST" and "MULTICAST" - but I have no clue if these can apply to my situation (tcp) ...? Therefore I am here asking for advice or confirmation - if either option (1) or (2) (mentioned above) are the CORRECT way of hanlding the situaton then I know I am on the right path. But I thought there must be a type of concept where I can just BROADCAST or MULTICAST the data to a single TcpClient (or socket, or something) that would handle sending the information to all clients automatically. Wouldn't that be the CORRECT approach? How is this typically resolved? I am looking for any kind of advice on how to proceed... Any ideas, hints, and help would be greatly appreciated, thanks

hplus0603: thanks for the reply - I guess that explains why my research didn't turn up anything for TCP...

Then I am curious - which approach (1 or 2) would you recommend me using?

Quote:Original post by hplus0603
There is no such thing as TCP multicast or broadcast or even unicast. A TCP connection is explicitly a point-to-point connection between exactly two peers.

Unicast/multicast/broadcast is an IP feature, and is provided using specific address(es); so ... how does that preclude TCP from being setup between a source and dest IP address, the latter of which is a multicast address? Is this a specific part of the TCP over v6 spec that TCP is not allowed to do this?

/me has not touched IPv6 in about 4 years, but from the sounds of it you have

Quote:Original post by redmilamberUnicast/multicast/broadcast is an IP feature, and is provided using specific address(es); so ... how does that preclude TCP from being setup between a source and dest IP address, the latter of which is a multicast address?

Nope, it is undefined/disallowed in IPv4 too.
If you think about it, TCP (being reliable and offering in-order deliverance and such) doesn't make much sense with multicast: A million hosts could join a multicast group, and using TCP, you would have to make sure that each of these gets all data in correct order. How to do it, as you don't even know who might be getting packets?

To make matters worse, IPv4 (unlike IPv6) does not force you to support multicast at all. In fact, it is generally not supported except on your LAN, routers normally don't forward multicast packets. Any multicast traffic that you send to the IPv4 internet must go through a tunnel (comparable to a proxy).
As far as I know, most of the multicast backbone belongs to universities or other large scientific organisations. So if at all, you can use multicast from one uni to another, but I may be wrong here. (Please tell me that I am wrong, that would be too good to be true)

Most ISPs in most parts of the world to my knowledge don't use IPv6 (because it sucks, and because they don't need to -- despite the fears in the late '90s, IP addresses are still abundant except in some parts of Asia) and they don't offer multicast tunnelling either (because it costs them extra, lessens their revenues, and they probably don't see a need for either).

Sidenote: Out of curiosity, I'd like to know if anyone has any rough figure of how many hosts you can really reach with multicast. Maybe there is really a noticeable number (even if it is as small as 5% of the internet)?

Quote:Original post by redmilamber
Unicast/multicast/broadcast is an IP feature, and is provided using specific address(es); so ... how does that preclude TCP from being setup between a source and dest IP address, the latter of which is a multicast address?

Let's examine what would happen if you tried to start a TCP connection whose destination was an established multicast address.

The server would send a SYN packet destined for every client.
Every client would respond to this SYN packet with a SYN-ACK packet.
The server accepts the first SYN-ACK packet it gets back and denies the rest, then it sends an ACK packet to every client.
It'd goes downhill from there.

TCP's design is for a conversation between two parties. There's no reason you couldn't create a similar-looking protocol for multicast, but raw TCP will not be useful to you.

Hi,

Reading solution 1 and 2 the main difference i think is really about memory usage.
When you create a thread in response to a client connection you've got (if you don't do anything) a large memory reservation that is made for the newly created thread that will lead to hang your application at about +- 2500 threads.
The second solution will be less memory/cpu consumming because you've got only one thread.
I don't know about c# but in c++ you would use socket event to make the work.

An alternatiove would be to use IO completion port. Go to codeproject.com to find some samples. This approach is much more adapted to a very large number of client connecting and can be seen as your solution 2, but i've never experienced.

As mentionned in this thread, the question is : do you intend to use your application on a private LAN or over the Internet ?
The problem with the Internet is that you won't be able to broadcast or multicast because you do not have control aver www routers. And those routers wont route your packets.
On the other hand, over a private LAN you will face router configuration problems using multicast (broadcast also !).
I've got an application that is designed to make use of multicast. This application is used over a private LAN.
During the developpement process the multicast solution was running pretty well and consumming near nothing over the network to send 15Mg to multiple targets. The problem was the time to send such a file since multicast packets cannot exceed 516 bytes.
When deploying the application over a customer private LAN we then realize that the multicast router configuration wasn't an easy task for network 'expert' people. So we finished in using multicast for simple client notification that they are using to open TCP socket to make the transfert stuff.

I've also been using broadcast to notify client applications to act on receiving. The same problem has occurs : router configuration.
We have ended in udp/tcp & tcp/udp software plugs to 'jump' over the routers or over the internet.

I have also made some server using only UDP packets, but this time we have fall into 'non reliable' protocole feature. This lead in having to repeat packets, having to send 'confirmation' packets and so.

So all application and technics have their pros and cons.

To answer your problem, the real question is : why do your clients are connecting/disconnecting, what is the protocol flow, what is beeing slow (intenciating or transferring datas ?).

My 2 euro/cents

Hmm. I'm really not stupid, guys :). You don't need to tell me about things any competent network programmer already knows. But, since a couple of people seem to have misunderstood:

1. Clarification: no-one I've ever known talks about IPv4 multicast, since it de-facto does not exist within the protocol. To grossly summarize: most people don't count something that "sort of might exist on a LAN segment depending upon your hardware and software" as a real feature. I thought that with the mention of v6 it would be pretty obvious to people that I wasn't even considering v4 multicast.

2. To anyone/everyone who thinks multicast TCP can't/shouldn't exist for any kind of theoretical reason: just how, precisely, do you think the many existing reliable multicast networks actually work anyway? Especially all those built as connection-oriented dynamically routed protocols (i.e. functionally the same as TCP)?

3. In general, all bets are off when it comes to predicting v6 stuff merely by looking at v4: a core design aim for the last decade or so has been that it should specifically fix the fundamental design mistakes of its predecessor.

But you see, TCP really makes no sense with multicast. How should it work?

First, a host makes a connection with your server and sends you some data, whatever it is (let's say a HTTP request, it doesn't matter). You send back the reply using multicast. How does the host know which group to join to receive the data?
1. The host could have told you "send it to this group". Sending bulk data to arbitrary unverified addresses is a bombing idea... :)
2. You could tell the host what group to join. Better, but how do you tell him? So you have to send him one unicast packet telling him which multicast group to join. What if that packet is lost, and how do you know? Well... we could open a TCP connection for that... ;)

Next, you send out data in a stream and you guarantee that it will arrive and it will arrive in order, too (after all, this is TCP). So you have to wait for an ACK at some point. If you don't know that your previous packets have arrived, you cannot send more.
Fair enough, you can send more than a single packets even if no ACKs have been received yet, but at some point you MUST wait -- you will not send out 200 kB of data without receiving one single ACK.

Now, remember you're doing multicast. You send out one packet and 485387 hosts join this multicast group. One might wonder why, because they don't have a reason to (they did not negotiate a connection with you). But let's assume they magically know that data is coming to that group, anyway.
So, the intermediate routers do their trick, and your packets arrive at (nearly) all of them.

Right... so how do you know who is receiving your packets? During the next few seconds, you get back ACKs from 472014 different IP addresses. You don't know about the other 13373 hosts which would like to receive your packets, too, beacuse unluckily their ACKs were dropped due to congestion.
Now what? Which packets to retransmit? Who to wait for?

I just don't see how it could have any chance of working.

The only thing that I think might have a reasonable chance in conjunction with multicasting would be to first advertise the coming of data on a specific group via reliable channels. Hosts interested in your data will know what group to join. They might send out a datagram that you drop, too, so their NAT works (if any).
Then you puke the bulk data plus some forward error correction and add a few random retransmits onto the UDP stream. If they get it, fine, if they don't get it, not your problem. You might even choose to retransmit *every* packet 2-3 times, that's still extremely cheap compared to sending all data to a few thousand hosts separately.
Most people have a fair chance to get most of it that way. What they don't get can be transmitted individually via a plain normal TCP connection.

I have a similar problem. I have to code a C# multipoint video conference application. Because the app supports video streaming, it will use a large bandwidth. My ideea is to write some kind of client/server app that will do this multicast at OSI application level. From my knowledge there is no protocol that implements this kind of multicast (some specific IPs and subnetmasks), because a router transparent multicast will be a big network security problem (ie. a router transparent arp request could cause a never-ending multicast). The easy way of implementing my application is to route all incoming packages and find the nearest app-node (someone who usess the same app) that I can use as server. But this is not an optimal script because in the perfect I must connect to the node with which I have the largest available bandwidth (for me) and its connection has a larger available bandwidth than mine. Also I have to manage this connection so if I lose it, I have to reconnect to someone. So a custom implementation of the Spanning-Tree Protocol at application level is needed. The problem: how do I calculate the connection cost?