Hello, I would like to know, if you can answer, how reliable are Windows Services (WCF) to recieve, say, 20mb long files every 10 seconds each (using streaming) from at least 50 different sources?

Do you think a simple (fast PC) server could handle it?

I ask this because using .NET would reduce the development time exponentially but I have serious doubts about the performance...

Thanks in advance for any oppinion.

EDIT: That'd be each client sending such file every 10 seconds simultaneously.

[Edited by - owl on June 17, 2010 12:14:56 AM]

Hi,

You will probably run into I/O problems that fall outside the .NET scope. 50 files, each 20MB in ten seconds, that's 1GB in ten seconds or on average 100MB/s, or 800mbit/s + overhead when transferred over a network. I don't know what the sources are and how you process the data, but a conventional PC will probably not handle this reliably. If the sources are connected via network, this will probably be the bottleneck. If there's harddisk activity involved at the sources or the service, than this will become difficult (especially on writes). But if you only generate data in memory and process it in memory and all happens on the same machine, the bandwidth is not so dramatic anymore (800mbit/s in memory is not much). In this case, processing speed/processor speed will play a role.

But before I continue speculating: why don't you build a prototype and see how far it gets you?

Regards,
Andre

Hi, thank you for your time!

I was kind of thinking about regulating the amount of connections depending on the capabilities of the server, maybe returning some (boolean?) value from the exposed method in charge of recieving the stream... I'm currently working on a test but I was hoping that, maybe, before I waste precious time, someone would have come across this and could be able to tell me something along the lines of "Forget it dude! Just put up an FTP server because that's the best you can have".

All my experiences (Not many I must say) with networking programming have been quite traumatic and it's been a while since I started hoping some library could do the job as simple as copying a file from/to the filesystem... :(

PLEASE TELL ME .NET WILL MAKE EVERYTHING EASY FOR ME AS A DEVELOPER!

WCF itself can handle such requirements just fine, but the network interface bandwidth is going to limit you as the number of connections increase. And if you invest in professional networking hardware (1...n 10Gb fibers), the rest of the machine should be pretty powerful too in order to actually process the data.

You can configure WCF so as to accept a maximum number of simultaneous connections. When that maximum is reached, the system can automatically notify incoming clients that it is busy and they should retry the connection in a while.

It is also possible to dynamically determine the maximum feasible number of connections by observing the system workload (cpu load, memory load, page faults etc.) programmatically using the WMI instrumentation; you can then reject the connections manually based on your conditions by handling simple events of the WCF endpoint.

WCF and .net do make network programming quite easy :)

Awesome.

Thanks!

Also note that if you're planning to actually store the incoming data in the server HDD, it is wise to invest in a robust storage system that includes its own battery backup.

The newer Windows servers (2008 and 2008 R2) can be configured to take advantage of advanced write caching on compliant HDD hardware, but if the power goes off during a non-flushed write operation, the partition could be corrupted in a difficult manner - hence the necessity of the battery backup.

The advanced write caching is otherwise very useful for operations like yours, as it can increase the total write performance by an order of magnitude. You can enable it on Vista and 7 too, but it should not be used on a whim on non-backed-up hardware precisely because of the same reason as on a server.