Sign in to follow this  
vbx_wx

Get data from web browser using posix threads C++

Recommended Posts

I am trying to implement this drawing using posix threads and sockets(TCP).
[code]
webserver <--- proxy <--- client
webserver ---> proxy ---> client

[/code]


I tried using 4 threads for every arrow.I want to ask if i implement the code good,doesnt send data as it supposed too.Here is my code:
[code]
void* client_proxy(void* arg)
{
TCPSocket* sock = (TCPSocket*)arg;
pthread_mutex_lock(&mutex);
numbytes = sock->Recv(data, sizeof(data));
pthread_mutex_unlock(&mutex);
cout <<"Received from client: " << numbytes << endl;
}

void* proxy_server(void* arg)
{
ClientSocket* client = (ClientSocket*)arg;
pthread_mutex_lock(&mutex);
numbytes2 = client->Send(data, numbytes);
pthread_mutex_unlock(&mutex);
cout << "Sent to web server: " << numbytes << endl;
}

void* server_proxy(void* arg)
{
ClientSocket* client = (ClientSocket*)arg;
pthread_mutex_lock(&mutex2);
numbytes2 = client->Recv(data2, sizeof(data2));
cout <<"Received from web server: " << numbytes2 << endl;
pthread_mutex_unlock(&mutex2);
}

void* proxy_client(void* arg)
{
TCPSocket* sock = (TCPSocket*)arg;
pthread_mutex_lock(&mutex2);
numbytes = sock->Send(data2, numbytes2);
pthread_mutex_unlock(&mutex2);
cout << "Sent to client: " << numbytes << endl;
}

[/code]


PS: the web server is a browser,and it always send data to load the page only 40-50% of it. Then it remains stuck. I just can't get the right design to make this work :(

Share this post


Link to post
Share on other sites
Using a thread per socket (arrow), as you described it, is really inefficient and totally over complicates what you are trying to do. All you really need is one thread and the right use of socket functions.

Before we get into that though, you will first want to read this forums [url="http://www.gamedev.net/index.php?app=forums&module=forums&section=rules&f=15"]FAQ[/url]. It is very helpful and will address quite a few things you are doing wrong in that snippet of code. Check out [url="http://beej.us/guide/bgnet/"]Beej's Guide to Network Programming[/url] as well.

The problem you are having right now is TCP is a stream protocol. So any operation you perform for N bytes, 0 - N bytes might actually be processed. As a result, you have to keep track of how many bytes were actually processed, and then handle that scenario accordingly. So for example, when you call send, you need to make sure the number of bytes returned matches how many were requested to be sent. If it does not, then you have had a partial send take place, so you must wait until you can send again to finish sending the rest of the data. If you do not correctly implement that, then you will have stream corruption which to the end user looks like data loss.

There's a lot of wrapper code you are using that isn't shown, so you will need to make sure you understand the concepts of TCP first with the help of the FAQ and the guide, and then double check all your logic to make sure it will work as expected. In addition, to fix the thread issue you have currently, take a look at the [url="http://linux.die.net/man/2/select"]select[/url] function. It will basically allow you to handle all your sockets in one thread through the function. Threading is not simple and adds quite a lot of complications when you have shared data, so it would help to use only one thread to start off with along with select.

One last note, judging by the code posted, it looks like you are also sharing the same buffer for recv and send operations. That is a big no-no so you will need to research the different proper ways to handle those data buffers with TCP. In short, that's simply making sure each socket has its own unique receive buffer and send buffers are all unique or you just merge all data into one buffer to send and slide data over to reuse it, implementing a pseudo-circular buffer.

Spend some time checking out the FAQ and the guide and then try to update your code accordingly to solve your problems. Good luck!

Share this post


Link to post
Share on other sites
[quote name='vbx' timestamp='1298499961' post='4778160']
I am trying to implement this drawing using posix threads and sockets(TCP).
[code]
webserver <--- proxy <--- client
webserver ---> proxy ---> client

[/code]
[/quote]

I think you'll need to give us a little more information here.

If this is all in-process, why wouldn't you just pass data between the subsystems as a simple object?

If this is between processes, then you just want one thread per process.

Share this post


Link to post
Share on other sites
@ hplus0603: i am using a client Proxifier that intercepts all programs connecting to the internet. So for example if i open a webpage it will intercept that page and send it(ip) to the proxy.Then the proxy connects to that ip and a connection between client and server has established trough a proxy,and the client send's data to the proxy who forwards to the server,and vice versa.

I am confused that everyone tells me a different way to a aproach: use blocking sockets or non blocking sockets,use select,use threads. I would want to implement it using only threads without select(even if is ineficient),but if there is no way,i will use select too. So should i use blocking or non blocking sockets?

@ Drew_Benton: i will be greatfull if you tell me whats is the right way to use sockets in my implementation :)


PS: btw i am making a proxy server.


Thanks in advanced for help.

Share this post


Link to post
Share on other sites
Why not just use two threads. One from server-to-client, the other from client-to-server. Streams are then independent.

Also, if this all there is to code, then it's missing a loop that reads until there is any data left.

[code]struct SocketPair {
TCPSocket * sockFrom;
TCPSocket * sockTo;
};
void* client_proxy(void* arg)
{
SocketPair * p = (SocketPair*)arg;

while (p->sockFrom->open()) { // whatever you use to determine if there is more data to read
int nRead = p->sockFrom->Recv(data, sizeof(data));

int sent = 0;
int remaining = nRead;

do {
int nSent -= p->sockTo->Send(data+sent, remaining);
sent += nSent;
remaining -= nSent;
while (remaining > 0);
}
}[/code]

Now just start two threads, one with (server,client) and the other (client,server) in SocketPair parameter.

Obviously the above is lacking error handling so make sure to properly handle the return codes or exceptions or similar.

Share this post


Link to post
Share on other sites
[quote name='vbx' timestamp='1298582720' post='4778642']
PS: btw i am making a proxy server.
[/quote]

You want to use select(). Or, if you're going to have more than 50 sockets open, you want to use GetQueuedCompletionStatus() with I/O completion ports.


Share this post


Link to post
Share on other sites
I am still having problem i changed my send() and recv() data,and i made it work with two threads. Here is my complet code,surelly i have a mistake:

[code]

// forward declaration
class TCPSocket;
class ClientSocket;

// type used to encapsulate a TCPSOcket and a ClientSocket
struct SocketPair {
TCPSocket *clientProxySocket;
ClientSocket *proxyServerSocket;
};

// creates two threads for client and proxy
pthread_t pid1, pid2;

// declare a global SocketPair struct
SocketPair socketPair;

static void fillAddr(const string &address, unsigned short port, sockaddr_in &addr) {
memset(&addr, 0, sizeof(addr));
addr.sin_family = AF_INET;

hostent *host;
if ((host = gethostbyname(address.c_str())) == NULL)
{
cout << "Error" << endl;
}
addr.sin_addr.s_addr = *((unsigned long *) host->h_addr_list[0]);
addr.sin_port = htons(port);
}

class Socket {
public:
Socket()
{
descriptor_m = ::socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
if(descriptor_m == SOCKET_ERROR)
{
cout << "Error in Socket()" << strerror(errno) << endl;
}
}
Socket(unsigned int descriptor)
{
this->descriptor_m = descriptor;
}

// return the socket descriptor
unsigned int getSocket()
{
return descriptor_m;
}

void Close()
{
#ifdef WIN32
::closesocket(desc);
#else
::close(desc);
#endif
}

~Socket()
{
#ifdef WIN32
::closesocket(descriptor_m);
#else
::close(descdescriptor_m);
#endif
}
private:
unsigned int descriptor_m;

};

class TCPSocket: public Socket {
public:
TCPSocket(): Socket() {}

TCPSocket(int sockDesc): Socket(sockDesc) {}

int Send(const char* buffer, int size)
{
int total = 0;
int bytesleft = size;
int n;
while(total < n)
{
n = ::send(getSocket(), buffer + total, bytesleft, 0);
if(n == -1) break;
total += n;
bytesleft =- n;
}
size = total;
return size;
}
int Recv(char* buffer, int size)
{
int total = 0;
int n;
do {
n = ::recv(getSocket(), buffer, size, 0);
total += n;
}while(n == 0);
return total;

}
};

class ServerSocket: public Socket {
private:
sockaddr_in server;
public:
ServerSocket(): Socket()
{
memset(&server, 0, sizeof(server));
server.sin_family = AF_INET;
server.sin_addr.s_addr = INADDR_ANY;
server.sin_port = htons(LISTEN_PORT);
}
void Bind()
{
if((::bind(getSocket(), (sockaddr*)&server, sizeof(server))) == SOCKET_ERROR)
{
cout << "Error in Bind() -->";
cout << strerror(errno) << endl;
}
}
void Listen()
{
if((::listen(getSocket(), MAX_CONNECTIONS)) == SOCKET_ERROR)
{
cout << "Error in Listen() -->";
cout << strerror(errno) << endl;
}
}
TCPSocket* Accept()
{
int remoteSocket = ::accept(getSocket(), 0, 0);
if(remoteSocket == SOCKET_ERROR)
{
cout << "Error in Accept() -->";
cout << strerror(errno) << endl;
exit(1);
}
return new TCPSocket(remoteSocket);
}
};

class ClientSocket: public TCPSocket {
public:
void Connect(string address, unsigned int port)
{
sockaddr_in sin;

fillAddr(address, // in
port, // in
sin); // out
cout << "CONNETING TO :" << address << " " << port << endl;
if((::connect(getSocket(), (sockaddr*)&sin, sizeof(sin))) == SOCKET_ERROR)
{
cout << "Error in Connect()" << strerror(errno) << endl;
}
}
};

void Initialize()
{
WSAData wsadata;
if(WSAStartup( MAKEWORD( 1, 1 ), &wsadata ) != 0 )
{
cout << "Error creating socket" << endl;
exit(1);
}
wsadata.wVersion = 5;
}



void* client_server(void* arg)
{
SocketPair* socketPair = (SocketPair*)arg;

pthread_mutex_lock(&mutex);
numbytes = socketPair->clientProxySocket->Recv(data, sizeof(data));
pthread_mutex_unlock(&mutex);
pthread_mutex_lock(&mutex);
numbytes2 = socketPair->proxyServerSocket->Send(data, numbytes);
pthread_mutex_unlock(&mutex);
cout <<"Received from client: " << numbytes << endl;
cout << "Sent to web server: " << numbytes2 << endl;
return arg;

return NULL;
}

void* server_client(void* arg)
{
SocketPair* socketPair = (SocketPair*)arg;
numbytes = socketPair->proxyServerSocket->Recv(data, sizeof(data));
pthread_mutex_unlock(&mutex);
pthread_mutex_lock(&mutex);
numbytes2 = socketPair->clientProxySocket->Send(data, numbytes);
pthread_mutex_unlock(&mutex);
cout <<"Received from web server: " << numbytes << endl;
cout << "Sent to client: " << numbytes2 << endl;
return arg;
}

void dispatchConnection(TCPSocket *clientProxySocket, ClientSocket *proxyServerSocket)
{

// class for authentication protocl SOCKS 5 (source not incldued)
Handle handle(clientProxySocket);
handle.HandleAuthentication();


// get address and port from packet send by client
string addr = handle.getAddress();
short prt = handle.getPort();

proxyServerSocket->Connect(addr, prt);

// populate the socketPair
socketPair.clientProxySocket = clientProxySocket;
socketPair.proxyServerSocket = proxyServerSocket;

if(pthread_create(&pid1, NULL, client_server, &socketPair) != 0)
{
cout << "Unable to create pipe client-server thread (pthread_create()" << endl;
}
if(pthread_create(&pid2, NULL, server_client, &socketPair) != 0)
{
cout << "Unable to create pipe server-client thread (pthread_create()" << endl;
}
//delete clientProxySocket;
}


int main()
{
pthread_mutex_init(&mutex, 0);
pthread_mutex_init(&mutex2, 0);
#ifdef WIN32
Initialize();
#endif
ServerSocket s;
s.Bind();
s.Listen();

// memory
while(true)
{
// loop forever, to accept any clients
TCPSocket *clientProxySocket = s.Accept();
ClientSocket *proxyServerSocket = new ClientSocket();

dispatchConnection(clientProxySocket, proxyServerSocket); // function Accept returns dynamic allocated memory
// the client is responsible in release it
}

return 0;
}


[/code]

Share this post


Link to post
Share on other sites
Both your TCPSocket::Send and TCPSocket::Recv functions are incorrect and contain bugs.

In Send, you are using uninitailized variables, not returning the actual number of bytes sent, and have implemented subtraction wrong (?!).

In Recv, your loop condition is incorrect, you need to check for a SOCKET_ERROR return value like in send (-1), and you don't really need a loop of any sort.

[b]Please[/b], take the time to review those functions and make a good effort to fix the issues on your own. If you simply post code and ask people to help you fix it, you won't learn anything and will continue to make the same mistakes over and over.

After you fixed your bugs (or at least really tried hard to), this is how simple those functions need to be:
[spoiler]
[code]
// Attempts to send 'size' bytes from 'buffer'. The 'buffer' should be at least
// 'size' bytes upon entry and readable. The function returns how many bytes
// were sent. There is no error reporting mechanism built in, so if the number of
// bytes returned does not match the number of bytes requested to be sent, assume
// an error has occurred and obtain the last socket error to know what happened.

// NOTE: This logic is still susceptible to malicious clients that stop receiving so
// larger sends might end up not going through. You can either not try to send the
// entire buffer or add timing code to timeout the send if it is not completed.

int Send( const char * buffer, int size )
{
// TODO: Implement sanity checks for parameters to make sure they are valid

int offset = 0;

// Loop while we have not sent the entire buffer
while( offset < size )
{
int sent = ::send( getSocket(), buffer + offset, size - offset, 0 );
if( sent == SOCKET_ERROR )
{
break;
}

offset += sent;

/*
// TODO: Consider using a platform sleep of the lowest value
// to give the system a chance to catch up since this was a
// partial send. If you do not and are trying to send too much
// to the client, you will loop many, many times and waste
// a lot of useful CPU in the process.
if( offset != size )
{
Sleep(1);
}
*/
}

// Return how many bytes we actually sent
return offset;
}

// Will attempt to receive up to 'size' bytes into 'buffer'. The 'buffer' should be at least
// 'size' bytes upon entry and writable. The function returns the result of recv, so
// SOCKET_ERROR is returned upon an error, 0 is returned upon a disconnect,
// and all other values returned > 0 indicate how many bytes were actually received.
int Recv( char * buffer, int size )
{
// TODO: Implement sanity checks for parameters to make sure they are valid

return ::recv( getSocket(), buffer, size, 0 );
}
[/code]

Since you are not using [i]select[/i], you should not try to receive with fixed sizes since that opens your program up to denial of service attacks where you are expecting N bytes but only 0..N - 1 are ever sent. That is why the Recv function will return 0..N requested bytes rather than N and no looping is required. As a result, you have to properly handle the return value in the code that calls the function. Likewise is true of the Send function, you must properly handle the return value and check any error codes where necessary.[/spoiler]

Fixing those two functions still won't fix all your problems though. It looks like you are still using one shared buffer for all communication, so your locks are wrong. Specifically, you should be doing: "Lock -> Operation 1 -> Operation 2 -> Unlock" rather than "Lock -> Operation 1 -> Unlock -> Lock -> Operation 2 -> Unlock". In your current code, you now have a data corruption possibility from the second thread using the buffer and overwriting data from the first or vice versa. Alternatively, you can just setup a local receive buffer in each thread and there is no need to lock then.

You should also keep track of the pointers you allocate via new so you can delete them at the program end or when both connections are done. Right now, you will simply leak resources until the program crashes from an out of memory exception. Also, you are not handling the error codes for ::accept properly either. You need to make sure the error is not on the remote side. Right now, someone could make your program exit simply by starting an accept and then purposefully trigger a [url="http://msdn.microsoft.com/en-us/library/ms740668%28v=vs.85%29.aspx#WSAECONNRESET"]WSAECONNRESET[/url] and all of your connections would go poof. Not good! Is there any particular reason you request winsock 1.1 then set the version to 5 in the Initialize function? It's pretty unnecessary.

Finally, and this is the biggest issue of it all, you start threads to handle your connections, but you exit them after the first send/recv that takes place. This means you are only able to support the first send/recv pair between the program and the remote host, which is pretty useless for anything but simple sites that do not support keep-alive. Reread Antheus's post that shows the logic behind this. You loop while the socket is open and then read some data and immediately send it out to the pair. Since you have a unique buffer for each thread in that case, there is nothing for you to lock/synchronize with the mutexs. Once you do that, you will be able to support traffic past the initial send/recv on the connections. Using a lock and a counter inside the SocketPair object, you can also implement a simple reference counter to know when you can delete the object as well so you do not leak memory.

Share this post


Link to post
Share on other sites
I follow your adviced and made changes in the implementation,but still it doesnt work good. Sometimes, recv() returns -1 amd 0 but then it continue to work and receivede data from the webserver but id doesnt send it back to the client or viceversa:

[code]

Received from client: 865
Sent to web server: 0
Received from client: 865
Sent to web server: 0
Received from client: 865
Sent to web server: 0
Received from client: 865
Sent to web server: 865
You are connected !!!

Received from client: 3
Sent to web server: 3
Error receiving data
No error
Error receiving data
No error
Error receiving data


[/code]




This is how i implement them:

[code]

void* client_server(void* arg)
{
pthread_detach(pthread_self());

char data[1500];
SocketPair* socketPair = (SocketPair*)arg;

int bytes_recv = 0;
int bytes_sent = 0;
while((bytes_recv = socketPair->clientProxySocket->Recv(data, sizeof(data))) > 0)
{
int sent = 0;
int remaining = bytes_recv;
do {
bytes_sent = socketPair->proxyServerSocket->Send(data, bytes_recv);
sent += bytes_sent;
remaining -= bytes_sent;
cout <<"Received from client: " << bytes_recv << endl;
cout << "Sent to web server: " << bytes_sent << endl;
}while(remaining > 0);
}
return NULL;
}

void* server_client(void* arg)
{
pthread_detach(pthread_self());

char data2[1500];
SocketPair* socketPair = (SocketPair*)arg;

int bytes_recv = 0;
int bytes_sent = 0;
while((bytes_recv = socketPair->proxyServerSocket->Recv(data2, sizeof(data2))) > 0)
{
int sent = 0;
int remaining = bytes_recv;
do {
bytes_sent = socketPair->clientProxySocket->Send(data2, bytes_recv);
sent += bytes_sent;
remaining -= bytes_sent;
cout <<"Received from web server: " << bytes_recv << endl;
cout << "Sent to client: " << bytes_sent << endl;
}while(remaining > 0);
}
return NULL;
}

void dispatchConnection(TCPSocket *clientProxySocket, ClientSocket *proxyServerSocket)
{
Handle handle(clientProxySocket);
handle.HandleAuthentication();

string addr = handle.getAddress();
short prt = handle.getPort();

proxyServerSocket->Connect(addr, prt);

// populate the socketPair
socketPair.clientProxySocket = clientProxySocket;
socketPair.proxyServerSocket = proxyServerSocket;

if(pthread_create(&pid1, NULL, client_server, &socketPair) != 0)
{
cout << "Unable to create pipe client-server thread (pthread_create()" << endl;
}
if(pthread_create(&pid2, NULL, server_client, &socketPair) != 0)
{
cout << "Unable to create pipe server-client thread (pthread_create()" << endl;
}
}


int main()
{
Initialize();
ServerSocket s;
s.Bind();
s.Listen();

// memory
while(true)
{
// loop forever, to accept any clients
TCPSocket *clientProxySocket = s.Accept();
ClientSocket *proxyServerSocket = new ClientSocket();

dispatchConnection(clientProxySocket, proxyServerSocket); // function Accept returns dynamic allocated memory
// the client is responsible in release it

delete clientProxySocket;
delete proxyServerSocket;

}

return 0;
}


[/code]




And the send() and recv() functions i implemented like sugested ( i hope) :

[code]

int Send(const char* buffer, int size)
{
int offset = 0;
while(offset < size)
{
int n = ::send(getSocket(), buffer + offset, size - offset, 0);
if(n == SOCKET_ERROR)
{
break;
}
offset += n;
if(offset != size)
{
Sleep(1);
}
}
return offset;
}

int Recv(char* buffer, int size)
{
int n = ::recv(getSocket(), buffer, size, 0);
if(n == SOCKET_ERROR)
{
cout << "Error receiving data" << endl;
cout << strerror(errno) << endl;
}
if(n == 0)
{
cout << "Remote host closed connection" << endl;
}
return n;
}


[/code]




PS: I'm sorry for just pasting code but i really study and i can't find the answer why it doesnt work good. I don't expect to fix my code, but i will welcome sugestions with what I am doing wrong like you did until now. And trust me,I learn better from mistakes.

Btw, the only thing i didn't understand was the accept() part,how can you figure it out if the error is an the remote side?

Thanks again.

Share this post


Link to post
Share on other sites
You should not be calling delete on clientProxySocket and proxyServerSocket. You will corrupt the memory used by the threads. Instead, you have to work out a different system to free them after they are no longer needed. For now, don't delete them but keep in mind the memory leak.

Now that you have your core recv/send functions fixed up, you need to simplify your applications send/recv logic. It's way too complex and uses a "too clever" coding style that gets you into trouble. In addition, you will want way larger buffers for a web proxy. You can also increase the internal socket send/recv size to the max, but you can read up on how to do that later.

Your client_server function should look like this:
[code]
void * client_server( void * arg )
{
pthread_detach( pthread_self() );

SocketPair * socketPair = (SocketPair*)arg;
char data[65536];

while( true )
{
// First, receive as many bytes as possible
int bytes_recv = socketPair->clientProxySocket->Recv( data, sizeof( data ) );

// 0 for disconnect, -1 for error
if( bytes_recv <= 0)
{
// TODO: Log this event
break;
}

// Now try to send all the bytes
int bytes_sent = socketPair->proxyServerSocket->Send( data, bytes_recv );

// Debugging info
cout <<"Received from client: " << bytes_recv << endl;
cout << "Sent to web server: " << bytes_sent << endl;

// On success, bytes_sent should equal bytes_recv
if( bytes_sent != bytes_recv )
{
// TODO: Log this error since Send should send all the data by design
break;
}
}

return NULL;
}
[/code]

As you can see, since your core Send function attempts to send all the data, you do not need to do it again yourself. Any errors in the core send function are unrecoverable so you cannot fix that in your application logic. Instead, you just have to compare the final byte count that was sent and proceed from there.

Fix the delete issue first, since that is going to be a huge problem in messing up the execution of your program. Once that is done, fix up your client_server as well as the server_client function (it too should follow a similar format as client_server) and you should be good to go barring other issues in different parts of the code.

Share this post


Link to post
Share on other sites
A couple of questions:

1) Regarding the memory allocation problem, i wonder if it is advise to use smart pointers, so I don't need to deallocate myself ?

2) When recv() function return -1 or 0, it is correct to call closesocket() or no ?

Share this post


Link to post
Share on other sites
1. Smart pointers are usually advised in C++.

With that said though, as long as both your client_server and server_client thread functions exit when they are supposed to, you can just keep a mutex object and reference count in SocketPair struct to know when you can delete the pointers. For example, have a look at the [url="http://www.yolinux.com/TUTORIALS/LinuxTutorialPosixThreads.html"]mutex[/url] example. You just acquire the mutex at the start of the thread, increment a counter, then release the mutex. At the end of the thread function, you acquire the mutex, decrement the counter, check to see if it's 0, if so delete the memory, then release the mutex. Don't forget to correctly instantiate the members though.

2. From [url="http://msdn.microsoft.com/en-us/library/ms737582%28v=vs.85%29.aspx"]MSDN[/url]: [quote]An application should always have a matching call to closesocket for each successful call to [url="http://msdn.microsoft.com/en-us/library/ms740506%28v=vs.85%29.aspx"][b]socket[/b][/url] to return any socket resources to the system.[/quote]

So you do need a matching call to closesocket, but where you do that just depends on your application. Since you have two threads each handling the same socket, you'd only want one of them to call it. Also, you want to make sure you handle as graceful shutdown as possible and call [url="http://msdn.microsoft.com/en-us/library/ms740481%28v=vs.85%29.aspx"]shutdown[/url] if you are going to be disconnecting the socket yourself.

Also, I noticed another problem in the earlier code you posted. You cannot have a global SocketPair object like you do right now. Since you are using threads, you need to allocate memory for the object before passing it to the threads. The threads would then deallocate the memory as well. This could contribute to some other issues you might be seeing if you were to connect more than one connection through your proxy. So get rid of the global object and allocate one for each pair in your dispatchConnection function.

You will need to check your program for other global variable issues as well. When you use threads, you have to synchronize access for reading/writing between shared variables. If you do not, you get undefined results, of which may result in the program working sometimes, but not others.

Share this post


Link to post
Share on other sites
If I understood good its something like this ?

[code]

class SocketPair {
private:
TCPSocket* clientProxySocket;
ClientSocket* proxyServerSocket;
pthread_mutex_t mutex;
int counter;
public:
SocketPair()
{
mutex = PTHREAD_MUTEX_INITIALIZER;
counter = 0;
}
~SocketPair() {}
void addClientProxy(TCPSocket* clientProxySocket)
{
this->clientProxySocket = clientProxySocket;
}
void addProxyServer(ClientSocket* proxyServerSocket)
{
this->proxyServerSocket = proxyServerSocket;
}
TCPSocket* getClientProxy()
{
return clientProxySocket;
}
ClientSocket* getProxyServer()
{
return proxyServerSocket;
}
pthread_mutex_t getMutex()
{
return mutex;
}
void incr()
{
++counter;
}
void decr()
{
--counter;
}
int getCounter()
{
return counter;
}
};





void* client_server(void* arg)
{
pthread_detach(pthread_self());

char data[65536];
SocketPair* socketPair = (SocketPair*)arg;

pthread_mutex_t mutex = socketPair->getMutex();
pthread_mutex_lock(&mutex);
socketPair->incr();
pthread_mutex_unlock(&mutex);

while(true)
{
// First, receive as many bytes as possible
int bytes_recv = socketPair->getClientProxy()->Recv(data, sizeof(data));

// 0 for disconnect, -1 for error
if( bytes_recv <= 0)
{
cout << "Error receiving bytes from client" << endl;
break;
}

// Now try to send all the bytes
int bytes_sent = socketPair->getProxyServer()->Send(data, bytes_recv);

// Debugging info
cout <<"Received from client: " << bytes_recv << endl;
cout << "Sent to web server: " << bytes_sent << endl;

// On success, bytes_sent should equal bytes_recv
if( bytes_sent != bytes_recv )
{
cout << "Error, didnt send all bytes from client to server" << endl;
break;
}
}

pthread_mutex_lock(&mutex);
socketPair->decr();
if((socketPair->getCounter()) == 0)
{
delete socketPair->getClientProxy();
delete socketPair->getProxyServer();
delete socketPair;
}
pthread_mutex_unlock(&mutex);

return NULL;
}








void dispatchConnection(TCPSocket* clientProxySocket, ClientSocket* proxyServerSocket)
{
Handle handle(clientProxySocket);
handle.HandleAuthentication();

string addr = handle.getAddress();
short prt = handle.getPort();

proxyServerSocket->Connect(addr, prt);

// Create a SocketPair object and populate it
SocketPair* socketPair = new SocketPair;
socketPair->addClientProxy(clientProxySocket);
socketPair->addProxyServer(proxyServerSocket);

// create threads
if(pthread_create(&pid1, NULL, client_server, &socketPair) != 0)
{
cout << "Unable to create pipe client-server thread (pthread_create()" << endl;
}
if(pthread_create(&pid2, NULL, server_client, &socketPair) != 0)
{
cout << "Unable to create pipe server-client thread (pthread_create()" << endl;
}
}


[/code]

Share this post


Link to post
Share on other sites
Getting better, you have a few more design flaws.

1. The "pthread_mutex_t getMutex()" function currently returns a copy of the mutex object. You instead want to return a reference of it. You could also just make it return a pointer, but if you do that, you will need to update all your calls to pthread_mutex_lock/unlock so they don't pass the address (see #4 below).

2. You must release the mutex before deleting the class object (socketPair)! Otherwise, you are deleting the mutex and then trying to release it, which is undefined behavior .

3. You can move "pthread_t pid1, pid2;" into the SocketPair class so keep track of the threads that are being used. Right now, your program will only track the last two threads created. Consider just making them public member variables rather than trying to write get/set for them, but it's up to you.

4. Now that the socketPair object is a pointer, you do not need to pass its address in pthread_create; you just pass the pointer address. Right now, you are passing a SocketPair ** to the function, so it should crash when you run it.

5. You can also consider calling the close socket logic before you delete the variables if you do not have it anywhere else right now. That would be the ideal place since you do not really have anywhere else to put it.

Share this post


Link to post
Share on other sites
Yes i will make all the methods and elements public in SocketPair, is much easy. I knew that #2 will give me that problem, then i will try to move it after i release the mutex. Regarding #4, it doesnt crash, but it doesnt do nothing, just gives me error in receiving data (strerrno = "No Error"),so i knew something its not good. And about #5 i am calling closesocket() in the destructor of Socket() so when i call delete to release memory, i guess it will call closesocket() too, its ok like this ?

Share this post


Link to post
Share on other sites
Well, I am a litlle shocked....it works ! I open two sites,without problem, but then it crashes :) I'm glad that I am getting results.

Using backtrace in gdb it gives me this:

[code]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 7396.0x1d74]
0x6e0c30cf in pthread_mutex_lock () from c:\MinGW\bin\libpthread-2.dll
(gdb) backtrace
#0 0x6e0c30cf in pthread_mutex_lock () from c:\MinGW\bin\libpthread-2.dll
#1 0x00401828 in _fu9___ZSt4cout (arg=0x3647a0) at ..\src\server.cpp:484
#2 0x6e0c5121 in ptw32_threadStart@4 () from c:\MinGW\bin\libpthread-2.dll
#3 0x76cc1287 in msvcrt!_itow_s () from C:\Windows\system32\msvcrt.dll
#4 0x76cc1328 in msvcrt!_endthreadex () from C:\Windows\system32\msvcrt.dll
#5 0x758e1174 in KERNEL32!AcquireSRWLockExclusive ()
from C:\Windows\system32\kernel32.dll
#6 0x771bb3f5 in ntdll!RtlInsertElementGenericTableAvl ()
from C:\Windows\system32\ntdll.dll
#7 0x771bb3c8 in ntdll!RtlInsertElementGenericTableAvl ()
from C:\Windows\system32\ntdll.dll
#8 0x00000000 in ?? ()

[/code]




EDIT: I'm not sure, but could the problem be when I release memory for socketPair?

[code]

{

// more code here.....

pthread_mutex_lock(&socketPair->mutex);
socketPair->counter--;
if(socketPair->counter == 0)
{
delete socketPair->clientProxySocket;
delete socketPair->proxyServerSocket;
}
pthread_mutex_unlock(&socketPair->mutex);
delete socketPair;

return NULL;


}

[/code]

Share this post


Link to post
Share on other sites
[quote name='vbx' timestamp='1299111517' post='4781217']
[code]
pthread_mutex_lock(&socketPair->mutex);
socketPair->counter--;
if(socketPair->counter == 0)
{
delete socketPair->clientProxySocket;
delete socketPair->proxyServerSocket;
}
pthread_mutex_unlock(&socketPair->mutex);
delete socketPair;

return NULL;


}

[/code]
[/quote]

You are deleting the socketPair even when the counter is not 0.

Share this post


Link to post
Share on other sites
Yes, thats way i made the mutex global so I can include delete socketPair in the if() test. Now its working for a few sites, but again it crashes this time when the authentication takes place. I will post the code here if I dont find the answer.

Thanks.

Share this post


Link to post
Share on other sites
I run some tests and i see that are 3 different type of crashes everytime. The crashes are happening, after loading one site or a few sites.

1)

[code]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 2784.0xd0c]
0x004032c4 in TCPSocket::Send (this=0xfeeefeee,
buffer=0xa56ff04 "HTTP/1.1 200 OK\r\nDate: Fri, 04 Mar 2011 11:15:11 GMT\r\n
Server: Apache\r\nLast-Modified: Fri, 04 Mar 2011 10:46:03 GMT\r\nETag: \"682344
a-11a9-49da5da3650c0\"\r\nAccept-Ranges: bytes\r\nVary: Accept-Encoding,User"...
, size=2231) at ..\src\server.cpp:118
warning: Source file is more recent than executable.
118 int n = ::send(descriptor_m, buffer + offset, si
ze - offset, 0);


[/code]

2)

[code]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 3196.0x784]
0x004031f1 in TCPSocket::Recv (this=0xfeeefeee, buffer=0x311ff04 "",
size=65536) at ..\src\server.cpp:134
134 int n = ::recv(descriptor_m, buffer, size, 0);


[/code]

3)

[code]

Program received signal SIGTRAP, Trace/breakpoint trap.
[Switching to Thread 336.0x4e0]
0x773a4685 in ntdll!RtlpSetUserPreferredUILanguages ()
from C:\Windows\system32\ntdll.dll


[/code]




Anyone knows what could be the problem ?

Share this post


Link to post
Share on other sites
Looks like you are deleting a variable but still using it somewhere else. After you delete your vars, set them to NULL to see if you can trigger a null pointer crash sooner, rather than later. Make sure you set 'counter' correctly too.

You might have to handle a rare edge case when one thread finishes before the second one starts. In that case, you need another flag variable that is set after counter == 2 on thread start. Before you execute the decrement and then check for 0, you have to wait until that flag is set. Otherwise, one thread might run and delete the class before the second one starts. When the second one starts, the memory is already gone so it crashes. This is just a side effect of using threads and shared memory, you have to guard against such race conditions.

Also make sure you clean and rebuild, one of the warning messages looks like your files are out of sync to the exe.

Share this post


Link to post
Share on other sites
Well i guess what you suggest is the same with this ?

[code]

pthread_mutex_lock(&mutex);
if(socketPair->counter == 2)

{
socketPair->counter--;
if(socketPair->counter == 0)
{
delete socketPair->clientProxySocket;
delete socketPair->proxyServerSocket;
delete socketPair;
}
}
pthread_mutex_unlock(&mutex);


[/code]




Doing this it didnt crashed at all. I opened many sites including watching video from youtube, from 25 sites that i have in my Speed Dial in Opera only 6 of them didint open, the program didnt crashed, it loads only a few from site than remains block. The strange thing is that all 6 of them are from my country, I didnt find a foreign site that hasnt work. What could be the problem ?

Another anoing think is that now even if I am doing nothing the programs still tries to send() and recv() and gives me errors like :

[code]

No error
Error receiving bytes from server
Error in Connect()No error
You are connected !!!

CONNETING TO :127.0.0.1 80
Remote host closed connection
Error receiving bytes from client
Error receiving data


[/code]

And after this (if i dont do nothing for a few minutes) it doesnt work anymore, i have to stop it and run again.

Share this post


Link to post
Share on other sites
[quote name='vbx' timestamp='1299334972' post='4782056']
Doing this it didnt crashed at all. I opened many sites including watching video from youtube, from 25 sites that i have in my Speed Dial in Opera only 6 of them didint open, the program didnt crashed, it loads only a few from site than remains block. The strange thing is that all 6 of them are from my country, I didnt find a foreign site that hasnt work. What could be the problem ?[/quote]

Youtube forbids proxying and caching of its content. It pushes the TCP and HTTP corner cases and is very likely to break any non-compliant and even many compliant proxying attempts. At some point in the past it appeared as if they were deliberately closing or misreporting status of the connection.

Either way, at least as far as youtube goes, proxying is against EULA, so they are technically free to do anything they want.

If you want to access youtube, they provide an API which can be used to access the videos under certain obvious restrictions. For circumventing the region ban, rumor has it there are certain ways but they require payment.

For general HTTP proxying, squid (and others) are world-tested proxies.

Share this post


Link to post
Share on other sites
if you want to have a different point of presence on the internet, you should make arrangements with a VPN provider.
If you want to establish a TCP tunnel, I suggest using SSH with port forwarding.
If you want to run regular HTTP proxying, I suggest using Squid or something similar.
If you want a good answer to your question, I suggest you describe:
- what problem you're trying to solve
- how you're trying to solve it
- what particular symptoms you've seen
- what you've already tried to solve the problem
- posting code that's relevant to the symptoms you're posting

Share this post


Link to post
Share on other sites
I'm trying to surf webpages with my proxy program on, but a couple of sites dont load only 20-30% of the content, and few ones loads lets say 97%. Maybe my problems are in my 2 threads that transfers data, altough after Drew_Benton pointed out that a thread may start and finish before the other ones starts , I thought that everything is ok now. Program didint crashed anymore. Maybe there is something still wrong with it, that I can see. So i show you how my thread looks now (the other one works almost the same):

[code]

void* client_server(void* arg)
{
pthread_detach(pthread_self());

char data[65536];
SocketPair* socketPair = (SocketPair*)arg;

pthread_mutex_lock(&mutex);
socketPair->counter++;
pthread_mutex_unlock(&mutex);

while(true)
{
// First, receive as many bytes as possible
int bytes_recv = socketPair->clientProxySocket->Recv(data, sizeof(data));

// 0 for disconnect, -1 for error
if( bytes_recv <= 0)
{
cout << "Error receiving bytes from client" << endl;
break;
}

// Now try to send all the bytes
int bytes_sent = socketPair->proxyServerSocket->Send(data, bytes_recv);

// Debugging info
cout <<"Received from client: " << bytes_recv << endl;
cout << "Sent to web server: " << bytes_sent << endl;

// On success, bytes_sent should equal bytes_recv
if( bytes_sent != bytes_recv )
{
cout << "Error, didnt send all bytes from client to server" << endl;
break;
}
}

pthread_mutex_lock(&mutex);
if(socketPair->counter == 2)
{
socketPair->counter--;
if(socketPair->counter == 0)
{
delete socketPair->clientProxySocket;
delete socketPair->proxyServerSocket;
delete socketPair;
}
}
pthread_mutex_unlock(&mutex);

return NULL;
}


[/code]

Share this post


Link to post
Share on other sites
[quote name='vbx' timestamp='1299372184' post='4782266']
I'm trying to surf webpages with my proxy program on, but a couple of sites dont load only 20-30% of the content, [/quote]


What have you actually tried to debug this problem?
What does the proxy program print, if anything? How does it exit the infinite loop?
Your counter test will never hit the counter==0 case, btw, given that you test that counter==2 and then decrement only once. Assuming you always hold the mutex when changing the counter, there's no way that that code can be hit.

If you haven't yet tried capturing the incoming and outgoing data using "tcpdump -s 10000 -i any" (or Wireshark, if the proxy runs on Windows), then I suggest you to this. It will allow you to see what data comes in and goes out of your proxy.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this