6 replies to this topic

[Endemoniada]

Hi guys, as a programming exerise I'm using Winsock to download images. I'm using regular blocking sockets.

 

// send request
result=send(...);
 
// I got this from the MS example (see below)
result=shutdown(hSocket,SD_SEND);
 
// receive
 
// big buffer
int bufferSize=1048576;
char* buffer=new char[bufferSize];
 
// receive buffer
int bufferlen=512;
char inbuffer[512];

int total=0;
 
do {

 result=recv(hSocket,inbuffer,bufferlen,0);

 if(result > 0){
 
  // copy to big buffer
  for(i=0;i<result;i++){
   buffer[total+i]=inbuffer[i];
  }

  total+=result;

 }

} while(result > 0);

 

Am I doing that correctly ? It seems to work alright. Is it true that recv() will block until the buffer I pass to it is full ? If I remove the shutdown line it hangs (but eventually completes), I don't know why. And should I be checking for any specific errors ? I'm just not fully understanding the recv() function.

 

Thanks.

 

[slicer4ever]

Hi guys, as a programming exerise I'm using Winsock to download images. I'm using regular blocking sockets.

 

 

// send request
result=send(...);
 
// I got this from the MS example (see below)
result=shutdown(hSocket,SD_SEND);
 
// receive
 
// big buffer
int bufferSize=1048576;
char* buffer=new char[bufferSize];
 
// receive buffer
int bufferlen=512;
char inbuffer[512];

int total=0;
 
do {

 result=recv(hSocket,inbuffer,bufferlen,0);

 if(result > 0){
 
  // copy to big buffer
  for(i=0;i<result;i++){
   buffer[total+i]=inbuffer[i];
  }

  total+=result;

 }

} while(result > 0);

 

Am I doing that correctly ? It seems to work alright. Is it true that recv() will block until the buffer I pass to it is full ? If I remove the shutdown line it hangs (but eventually completes), I don't know why. And should I be checking for any specific errors ? I'm just not fully understanding the recv() function.

 

Thanks.

recv blocks until it receives data, that data might fill your buffer, or it might be a partial message(recv returns what it has received, and it is imperative that you ensure the message size is correct by comparing it's result to the expected file size of the message(what determines the expected file size is determined by some protocol you have implemented to negotiate file/message transfers).

[Erik Rufelt]

The reason you need shutdown is because recv() will not return 0 until the server decides to disconnect you. If the program doesn't exit immediately after receiving the image, the server is probably keeping the connection open until it decides to time out and disconnect you later. If you use shutdown with SD_SEND you tell the server that you will not be sending anything more, so it knows that it's OK to disconnect you when the image is sent.

 

You are using recv() correctly. It will not necessarily block until the buffer is full, only until at least one byte is received or an error occurs. You already handle that correctly by saving "result" number of bytes. An error is when result < 0 (disconnect from timeout or similar), graceful disconnect is when result == 0, and result > 0 is received data as you already know.

 

Assuming this is HTTP.. and you have some more recv() code earlier to handle the HTTP header?

You could ask the server to disconnect you faster by sending a "Connection: close" HTTP header in your request, and hopefully it will conform to protocol and do so.

 

You will obviously get a problem if the image is larger than 1 MB or whatever size you use for new[]. If you want to improve your program and this is HTTP, you can parse the Content-Length: header in the response before the image, to allocate exactly the number of bytes needed.

[TMarques]

It doesn't have anything to do with your question, but since you're excercising, you might want to use memcpy() instead of the for loop as it may be faster depending on the amount of data you're copying.

 

//copy to big buffer
memcpy(buffer+total, inbuffer, sizeof(char)*result);

 

even better you could try:

 

total += recv(hSocket,buffer+total,bufferSize-total,0);

which you end the need to copy from inbuffer to buffer at each iteration in the first place.

[hplus0603]

First, recv() will block until it can return at least one byte. Once it's decided to unblock, it returns as many bytes as it can in one go (meaning, if more than one byte is available in the incoming buffer.)

Second, protocols over TCP always include some kind of framing that lets you decide at the application level how much data to wait for, and when to break out of your loop.

Third, real network programs don't typically sit in a blocking loop like that, but instead are implemented as big state machines that use select() or something like it to figure out what needs to be done (if anything) at any one time. Thus, each iteration of the receive would be driven off of select() saying there is data available.

Fourth, you don't need to receive into a temporary buffer; you can receive straight into the buffer you want to get out, instead of doing a memcpy(). This saves some copying, and also lets you pass a larger amount of bytes into the potential recv() buffer, as you can pass in the distance between end-of-buffer and your current revceive buffer offset.

[Endemoniada]

Hi guys, thanks a lot.

 

I'm using HTTP and was not geting the header first, as Erik asked, I just learned how to do that with HEAD. When I retrieve the header do I do that with a seperate operation like this:

 

// get the header
hSocket=socket(...);
connect(...);
send(...);  //  use HEAD request
shutdown(hSocket,SD_SEND);
recv();  //  loop
closesocket(hSocket);
 
//  get the content using same steps above except GET instead of HEAD request
...
...

 

When I get the actual content can I request not to get the header ? Right now I get the whole thing, parse for the Content-Length, then offset the pointer to get at the image.

 

I was also going to ask about that temporary buffer but hplus0603 cleared that up for me.

 

Eventually I'll learn about select() but for now I just want something basic that's more fun than getting text.

 

Thanks again.

 

[hplus0603]

You get the header as part of a GET request. It will tell you how many bytes are in the actual body (Content-Length) in HTTP 1.1; or the connection will be closed when all data is sent (HTTP 1.0.)

In general, when you receive data from a network, you receive into one long, contiguous buffer. Then you can decide what to do with all the data in the buffer. If it's a line-based protocol, you can look for a newline between the start and the end of the buffer. If you find the newline, process that part as a line, and then move whatever is left in the buffer down to the beginning. (Sliding the "start offset" value forward for a while is generally more efficient than actually memcopy-ing data.)

So, when receiving a HTTP response, keep reading into the buffer. Once you see \r\n\r\n, you know that's the end of the header, and the beginning of the content (let's leave Content-Transfer-* out of it for now.) The length of the data will be in the Content-Length: header, and the data will follow starting right after the \r\n\r\n separator token.

If you want to look at a simple library written in C++ to make a GET request and receive the response, you can try the HTTP-GET library and for actually using HTTP in real life, you probably want to use a ready-made library like libcurl.