10 replies to this topic

[Vorel512]

Hi,

My problem is, that I have created an MFC DLL which handles the network side of my program.
Everything goes well except one thing...
The parent/main process can switch between server/client modes like: server,client, server and client (like WarCraft3).
When I want to run two or more "server and client" instance of the program, the first goes fine, but the others come back with error messages.
Also I have set up IPC so they can communicate. The first gets the process id, and calls WSADuplicateSocket(), and the caller calls WSASocket().
All of them return 0 as error message, so as far as I know, they works fine, but the recvfrom command returns with error message 10038 WSAENOTSOCK.
Ohh I almost forgot... The binding fails in the processes except in the first. While the first process should call bind, not the others, I think that is not the problem.

My code:

#define PIPE_SIZE sizeof(WSAPROTOCOL_INFO)
WSAPROTOCOL_INFO protocolinf;
.
.
.
If the process can create the pipe
.
.
WSADuplicateSocket(serverSocket,outBytes,&protocolinf);
.
.
If not...
.
.
serverSocket = WSASocket(protocolinf.iAddressFamily, protocolinf.iSocketType, protocolinf.iProtocol, &protocolinf, 0, NULL);
.
.
.

Every help is appreciated!

[hplus0603]

Why are you duplicating the socket?
Either you're using UDP, in which case you can keep using the same socket all the time.
Or you're using TCP, in which case you will use one socket per peer you are connected to.
In neither case do you need to ever duplicate a socket.
There are probably some assumptions in your code that you are not spelling out, and that we need to understand in order to help.

[Vorel512]

OK, so my corresponding code (UDP):

I have a class for the network stuff, it contains:
(.h)

class asd_API asd {
public:
asd(){};
~asd(){};
void Connect(char *toIP);
void ConnectUDP(char *toIP);
private:
static int InitializeUDP();
static void UDPResponseThread();
static int UDPStartSocket();
static int UDPStartServerListening(int serverSocket2);
static void UDPReceiveThread();
static void EndServer(int socket);
static void DulpicateSocket();
};

(.cpp)

#define g_szPipeName "\\\\.\\Pipe\\CSEPPPipe"
#define PIPE_SIZE sizeof(WSAPROTOCOL_INFO)
#define MAX_MESSAGE_SIZE 100
int nBytes;
char buffer[MAX_MESSAGE_SIZE];
char myID[MAX_MESSAGE_SIZE-3] = "id=none";
u_long iMode=1; //1 -non-blocking
int p_int;
int n_Type;
int serverSocket;
int clientSocket;
int thread;
char *n_IP = "127.0.0.1";
unsigned short n_Port = 7700;
bool n_RunThread = true;
bool n_Protocol = 0;
HANDLE AcceptOrReceiveThread;
HANDLE ResponseThread;
HANDLE DuplicateThread;
HANDLE IdentThread;
HANDLE Hmutex;
HANDLE Imutex;
struct sockaddr_in server;
struct sockaddr_in si_other;
struct hostent *hostEntity;
int silen=sizeof(si_other);

int asd::InitializeUDP()
{
serverSocket = UDPStartSocket();
DuplicateThread = CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE)DulpicateSocket, NULL, 0, NULL);

if (DuplicateThread == NULL)
{
EndServer(serverSocket);
return 1;
}
SetThreadPriority(AcceptOrReceiveThread,THREAD_PRIORITY_BELOW_NORMAL);
if (serverSocket == -1)
{
return 1;
}
Hmutex = CreateMutex(NULL, false, NULL);// Create the mutex
if (Hmutex == NULL)
{
EndServer(serverSocket);
return 1;
}
Imutex = CreateMutex(NULL, false, NULL);// Create the mutex
if (Imutex == NULL)
{
EndServer(serverSocket);
return 1;
}
ResponseThread = CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE)UDPResponseThread, NULL, 0, NULL);

if (ResponseThread == NULL)
{
EndServer(serverSocket);
return 1;
}
SetThreadPriority(ResponseThread,THREAD_PRIORITY_BELOW_NORMAL);
AcceptOrReceiveThread = CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE)UDPReceiveThread, NULL, 0,NULL);

if (AcceptOrReceiveThread == NULL)
{
EndServer(serverSocket);
return 1;
}
SetThreadPriority(AcceptOrReceiveThread,THREAD_PRIORITY_BELOW_NORMAL);

Sleep(100);// Let the threads start up;
return 0;
}

int asd::UDPStartSocket()
{
int error;
WSAData wsaData;
if ((error = WSAStartup(MAKEWORD(2, 2), &wsaData)) == SOCKET_ERROR)
{
return -1;
}
int mySocket;
mySocket = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);
if (mySocket == SOCKET_ERROR)
{
return -1;
}
switch (n_Type)
{
case 0:
UDPStartServerListening(mySocket);
break;
default:
ioctlsocket(mySocket,FIONBIO,&iMode); // in UDPStartServerListening also.
break;
}
return mySocket;
}

int asd::UDPStartServerListening(int serverSocket2)
{
server.sin_family = AF_INET;
server.sin_port = htons(n_Port);
server.sin_addr.s_addr = htonl(INADDR_ANY);
if (bind(serverSocket2, (sockaddr*)&server, sizeof(server)) == SOCKET_ERROR)
{
int error = WSAGetLastError();

if (error == 10048) //already in use
{
DulpicateSocket();
}
else
{
closesocket(serverSocket2);
return -1;
}
}
else
ioctlsocket(serverSocket2,FIONBIO,&iMode);
return 0;
}

void asd::DulpicateSocket()
{
switch (n_Type)
{
case 0:
HANDLE hPipe;

hPipe = CreateNamedPipe(
g_szPipeName, // pipe name
PIPE_ACCESS_DUPLEX, // read/write access
PIPE_TYPE_MESSAGE | // message type pipe
PIPE_READMODE_MESSAGE | // message-read mode
PIPE_WAIT, // blocking mode
PIPE_UNLIMITED_INSTANCES, // max. instances
PIPE_SIZE, // output buffer size
PIPE_SIZE, // input buffer size
NMPWAIT_USE_DEFAULT_WAIT, // client time-out
NULL); // default security attribute
if (INVALID_HANDLE_VALUE == hPipe)
{
_tprintf( TEXT("Could not create pipe. GLE=%d\n"), GetLastError() );
}
if (GetLastError() == 183)
{
hPipe = CreateFile(
g_szPipeName, // pipe name
GENERIC_READ | // read and write access
GENERIC_WRITE,
0, // no sharing
NULL, // default security attributes
OPEN_EXISTING, // opens existing pipe
0, // default attributes
NULL); // no template file
char szBuffer[10];
DWORD cbBytes;
sprintf(szBuffer, "%d", GetCurrentProcessId());
WriteFile(
hPipe,
szBuffer,
sizeof(szBuffer),
&cbBytes,
NULL
);
bool van = 1;
for(;;)
{
van = ReadFile(
hPipe, // pipe handle
&protocolinf, // buffer to receive reply
sizeof(WSAPROTOCOL_INFO), // size of buffer
&cbBytes, // number of bytes read
NULL); // not overlapped
if(van == 0)
break;
Sleep(1);
}

serverSocket = WSASocket(protocolinf.iAddressFamily, protocolinf.iSocketType, protocolinf.iProtocol, &protocolinf, 0, NULL);
}
else
{
for(;;) // for multiple instances
{
ConnectNamedPipe(hPipe,NULL); //wait for client to connect to pipe
char szBuffer[10];
DWORD cbBytes;
DWORD rBytes;
ReadFile(
hPipe, // pipe handle
szBuffer, // buffer to receive reply
sizeof(szBuffer), // size of buffer
&cbBytes, // number of bytes read
NULL); // not overlapped
DWORD outBytes = atoi(szBuffer);
WSADuplicateSocket(serverSocket,outBytes,&protocolinf);
WriteFile(
hPipe,
&protocolinf,
sizeof(WSAPROTOCOL_INFO),
&cbBytes,
NULL
);
FlushFileBuffers(hPipe);
DisconnectNamedPipe(hPipe);
Sleep(1);
}
CloseHandle(hPipe);
}
break;//switch
default:
break;
}//switch
}

void asd::UDPReceiveThread()
{
bool exists = false;
for (;;)
{
WaitForSingleObject(Hmutex, INFINITE); // Lock the mutex for UDPTalking
nBytes = recvfrom(serverSocket, buffer, MAX_MESSAGE_SIZE, 0, (SOCKADDR *)&si_other, (int *) &silen); // error 10038 when an instace of this already running.
if (nBytes == SOCKET_ERROR)
{

int error = WSAGetLastError();
if (error == 10057 || error == 10054 || error == 10014)
{
}
else if (error == 10035)
{
}
else
{
std::cout<< "----------------------->" << error << "\n";
Sleep(1000);// only for testing
}

}
else
{
...
}
buffer[0] = '\0';
ReleaseMutex(Hmutex);
Sleep(1);
}
UDPClients.RemoveAll();
}

[rip-off]

That still doesn't explain why you are trying to duplicate the socket. It isn't clear, for example, what your processes are going to do with this shared socket?

Can you give a high level description of your goal?

[Vorel512]

I would like to run multiple servers on the same UDP/TCP socket. So users can "dualbox".

[rip-off]

I don't believe your approach will work. WSADuplicateSocket is intended for socket handoff, not socket sharing:

The descriptors that reference a shared socket can be used independently for I/O. However, the Windows Sockets interface does not implement any type of access control, so it is up to the processes involved to coordinate their operations on a shared socket. Shared sockets are typically used to having one process that is responsible for creating sockets and establishing connections, and other processes that are responsible for information exchange.

Maybe this is what you want to do?

If not, a simple solution to the original problem to that is to to have the secondary servers connect to the primary one, and have the primary one demultiplex the data as it comes in (however you propose to handle that) and copy it to the correct "local" connection to the secondary server.

One approach might be for the secondary server to connect to the problematic socket, and identify itself as a server somehow. The primary server would echo data back out that socket when it is intended for this sub-server. Another might use a local IPC mechanism, like a named pipe, to connect to the primary server, not unlike what you were trying to do with the socket description, but with the actual data instead.

That said, this is still highly unusual. Multiplexing connections like that is generally handled at a different level (e.g. a SSH tunnel), if it is indeed necessary at all.

[Vorel512]

I will think about SSH tunneling.
If for example server 1 shut down, server 2 should gain control above the socket, therefore IPC and any relations falls out.

Thank you for your reply, it helped me a LOT!!!!!!

[Antheus]

Are you trying to implement a load balancer? Failover?

Having one server take over a socket doesn't make much sense. To be able to actually serve meaningful data, second server would need to have bit-exact copy of old server's state. So there would need to be two servers running at same time, performing exactly the same actions, including some that might be difficult to duplicate due to state being held by kernel.

While attempts at such systems have been made, native applications simply aren't capable of getting it right, there's too many factors outside of process' control. Java attempted to provide that through Serializable mechanism and one of most isolated VMs, but they gave up due to too many unsolvable issues.

[hplus0603]

I would like to run multiple servers on the same UDP/TCP socket. So users can "dualbox".

Duplicating the socket won't do that, even if you were to manage to make it work. (It is possible to share a socket between processes in Linux, using for example fork() -- that sharing would have the same problem)

Once the socket is duplicated, and a single packet comes in, only one of the two processes that use the socket will get the packet. Which of the two processes gets it is pretty much random. When you run two servers, you want messages from clients of server A to go to server process A, and messages from clients of server B to go to server process B. The only way to arrange this is to use two different sockets, each bound to a different port -- that's the only way the OS is going to know where to send which incoming packets.

Also, there is no such things as a UDP/TCP socket. UDP sockets and TCP sockets work differently at the IP and OS level, and you need to design appropriately for each.

[Vorel512]

On UDP/TCP I meant UDP or TCP socket.

Thanks for the heads up!
You are the best

[hplus0603]

If for example server 1 shut down, server 2 should gain control above the socket, therefore IPC and any relations falls out.

There are better solutions for taking over service on a port. Most commonly, you'll use something like HAProxy to serve requests on the port, and then configure HAProxy to forward the requests to some other process. If you grow in scale larger than what a single HAProxy machine can deal with, you can buy hardware load balancers that can do the same thing for a few million connections at a time :-)