Jump to content
  • Advertisement
_WeirdCat_

Linux sockets + http get request = no data

Recommended Posts

Currently.i need to fetch some data from a web page but i cant get anything from it. Reading through internet all samples provide something like this:

- connect socket to server with port 80

- write http request

- read back data

And they all say it works, but well not for me..

Heres a sample code i use - i use a class from my networking engine so it should work.

 

int main()
{
	TCPWindowLayerClient * c = new TCPWindowLayerClient();
	c->ConnectTo("http://www.google.com", 80);
	SendStrToFd( "GET / HTTP/1.1\r\nHost: http://www.google.com\r\nConnection: close\r\n\r\n", c->sockfd);
	int * ppos = new int;
	bool breakme = false;
	while ( !breakme )
	{
		int nbytes;
		 (*ppos) = 0;
	AnsiString r =	read_from_fd2(c->sockfd, c->pdata, ppos, max_tcp_buff_size, nbytes);
breakme = (nbytes <= 0);
		cout << r.c_str();
	}
	
	cout << "end.";
	return 0;
}

Only end. Is displayed and errno is 107 which means connection is closed.

Share this post


Link to post
Share on other sites
Advertisement

Some suggestions:

(1) Don't use 'Connection:close" in the header (shouldn't hurt, but why complicate things when troubleshooting?)

(2) You're using some kind of mysterious third-party library. Start by consulting the documentation on the library.  Since the problem appears to be either your use of the library or the library itself, and you have posted nothing about the library including its name and where someone else can find information on it, it's not possible to offer any kind of help about your problem here.

Share this post


Link to post
Share on other sites

sendstrtofd is just a write function, (        int wb = write(sockfd, &pdata[ count ], size_t( len_to_write ));)

 

readfromfd2 is just a read function. 

(  nbytes = read (sockfd, buffer, TCP_PROTOCOL_MAX_PACKET_LEN);)

 

Nothing more.

Connection is done like this:

	    int ConnectTo(AnsiString serv, int port)
	    {

Clear();

	    	 portno = port;
	    	    sockfd = socket(AF_INET, SOCK_STREAM, 0);

	    	    if (sockfd < 0) return -1;

	    	    server = gethostbyname(serv.c_str());

	    	    if (server == NULL) {
	    	    	LOG("serve null no host");
	    	       return -1;
	    	    }

	    	    bzero((char *) &serv_addr, sizeof(serv_addr));
	    	    serv_addr.sin_family = AF_INET;

	    	    serv_addr.sin_addr.s_addr = INADDR_ANY;
	    	    bcopy((char *)server->h_addr,
	    	         (char *)&serv_addr.sin_addr.s_addr,
	    	         server->h_length);
	    	    serv_addr.sin_port = htons((uint16_t)portno);

	    	    if (connect(sockfd,(struct sockaddr *) &serv_addr, sizeof(serv_addr)) < 0)
	    	         	{
	    	         		LAST_EVENT = TCP_ERROR;
	    	    	
	    	    	return -1;
	    	         	}
	    	         	Clear();
	    	         	timer.startTimer();
	    	         
return 1;
	    }

Also about this header connection:close it was just a sample i downloaded from internet, with or without it the result is the same (no data and this errno 107)

Share this post


Link to post
Share on other sites

If you're on linux, use Wireshark to watch what is going over the wire.  If the response is coming in to your machine, but you're not receiving it, the problem is when you read the socket.  If you're not getting a response back at all, the problem is elsewhere.

Use telnet to connect to the remote and and emulate what you think your program is doing, while watching the Wireshark output.  See if they're different.

Share this post


Link to post
Share on other sites

Unfortunatetly theres error on connection (cant even connect) now im not quite aware if i should set ip address or can i use www.google.com, maybe theres additional option for the socket that i do not set?

Share this post


Link to post
Share on other sites
Posted (edited)

For comparison, this connects. Maybe it is useful, maybe not 🙂

// @todo check errno
int fd = socket( AF_INET, SOCK_STREAM, 0 );
if( fd > 0 )
	fputs( "Socket created\n", stdout );

struct sockaddr_in addr;
//memset( &addr, 0, sizeof( addr ) );
addr.sin_family = AF_INET;
addr.sin_port = htons( 80 );
if( 0 == inet_aton( "13.32.90.60", &addr.sin_addr ) )	// www.n-tv.de
	fputs( "Error converting address\n", stderr );
else
	fputs( "Address converted\n", stdout );

if( 0 != connect( fd, (struct sockaddr *)&addr, sizeof( addr ) ) )
	fputs( "Error trying to connect\n", stderr );
else
	fputs( "Connection established\n", stdout );

// ... do somehting ....

 

Edited by Green_Baron

Share this post


Link to post
Share on other sites
Posted (edited)

That's bad.

Here's a plan b for converting the inet address to something connectible that connects. I love that tinkering 🙂

int fd = socket( AF_INET, SOCK_STREAM, 0 );
if( fd > 0 )
	fputs( "Socket created\n", stdout );

struct hostent *s = gethostbyname( "www.n-tv.de" );
if( NULL == s )
	fputs( "Address lookup failed\n", stderr );

struct in_addr **r = s->h_addr_list;

struct sockaddr_in addr;
//memset( &addr, 0, sizeof( addr ) );
addr.sin_family = AF_INET;
addr.sin_port = htons( 80 );
addr.sin_addr = *r[0];

if( 0 != connect( fd, (struct sockaddr *)&addr, sizeof( addr ) ) )
	fputs( "Error connecting to socket\n", stderr );
else
	fputs( "Connection established\n", stdout );

 

Edited by Green_Baron
Code cleanup

Share this post


Link to post
Share on other sites
Posted (edited)

in my eample gethostbyname was returning NULL (just to clarify)

in your first example i get error converting address

and in second one i cant compile this line 

struct in_addr **r = s->h_addr_list;

however i see that you use ip address instead of url

 

Additionally if i try to resolve www.google.com to ip address errno claims that HOST IS NOT FOUND....

Edited by _WeirdCat_

Share this post


Link to post
Share on other sites
Posted (edited)
1 hour ago, _WeirdCat_ said:

in my eample gethostbyname was returning NULL (just to clarify)

Yes, because gethostname expects a hostname, not an url. man gethostbyname helps, but also says it is deprecated and getnameinfo getaddrinfo() should be used instead. But some finger training won't harm 🙂

Quote

in your first example i get error converting address

That example works on my machine. Something is probably borked and needs further unborking. The exact code might help.

Quote

and in second one i cant compile this line 


struct in_addr **r = s->h_addr_list;

however i see that you use ip address instead of url

Yeah, it results in a warning. It is a working example. Explicitly casting would be better. But you can use the pointer to hike along the list of ip addresses, convert them and print them out.

while( *r != NULL ) {
	fputs( inet_ntoa( **r ), stdout );
	putchar( '\n' );
	++r;
}

 

Quote

 

Additionally if i try to resolve www.google.com to ip address errno claims that HOST IS NOT FOUND....

No idea. I pasted that code directly from the editor after it ran. If you resolve "https://..." you should get that error, "www.google.com" should work ...

 

 

Edited by Green_Baron

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!