Jump to content
  • Advertisement
Sign in to follow this  
kag1

HTTP Get Request..How big to make the buffer

This topic is 4773 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi, I'm coding in C++ with winsock, and am trying to do some things with HTTP. One of my first questions is..how does a web browser work? Does it send a single get request and download the whole page? Or does it just retrieve the HEAD, and then look at the length..and then send another request for the whole page? I ask this because I have no idea how big to make my buffer. Currently i'm only interested in the header, so I just set my char buffer to 1000 and it seems to grab enough of it.. when I say my char buffer I mean something like this
char buffer[1000];

//winsock recv function.
buffer = recv(param, param, param);
this method doesn't seem like the way it should be done..any suggestions would be nice.. thanks

Share this post


Link to post
Share on other sites
Advertisement
Quote:
Original post by kag1
One of my first questions is..how does a web browser work?

Mozilla source code could come in handy.

I haven't written any browsers but I imagine you would make a small buffer, read in the data, parse what you have, get some more if necessary until you reach the end of file or you have what you're looking for.

shmoove

Share this post


Link to post
Share on other sites
I've written some, propably quite unorthodox, http code once. Like shmoove said, I had a smaller buffer, which I used in a loop to recieve the stuff in. Then I applied the data in a bigger buffer, which I reallocated bigger each time.

About the content-length - field, it seemed that it wasn't always there. I think it's more like a bonus than a required thing, so it can't be trusted.

Share this post


Link to post
Share on other sites
http://en.wikipedia.org/wiki/Http
http://www.jmarshall.com/easy/http/

Anyways, I would expand my buffer on incoming data but I've been dealing with some pretty small and speacial case sessions. You can install ethereal and watch packet sniff for a bit, then filter to "http". This page is probably more then 4k worth of data of text, and the images etc are seperate downloads. So a non-growing 1k buffer isn't going to cut cheese :)

Edit:
shmoove
The vast majority of Mozilla is a rendering engine(Gecko), then framework. Very very small portion is it is the network layer that handles and processes the transactions.

Share this post


Link to post
Share on other sites
For cases like this, I think it is fairly safe to allocate a 4K buffer on the stack that will get used 99% of the time. The other one percent, you can use a std::string or other growable buffer. After you decide which buffer to use, make a pointer to it and use it later on in the function to do your processing. Safe, easy, fast. Alternatively, use a growable buffer that won't go out of scope at the end of a function (e.g. member variable).

Share this post


Link to post
Share on other sites
Quote:
Original post by MustEatYemen
shmoove
The vast majority of Mozilla is a rendering engine(Gecko), then framework. Very very small portion is it is the network layer that handles and processes the transactions.

That's means there is less relevant code to go through ;p

shmoove

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!