Jump to content

  • Log In with Google      Sign In   
  • Create Account


Drew_Benton

Member Since 29 Jul 2004
Offline Last Active Apr 28 2014 07:22 PM

Topics I've Started

Looking for some math ideas regarding a crypto protocol

01 October 2010 - 07:53 AM

I've run into a road block when trying to understand the math that goes on in a specific crypto protocol I am studying for educational purposes.

First, this is how it's setup. The client generates values using these two equations:
 


- 4 (in this specific case, I think this is an important part of it)
- large 512-bit fixed number (assuming prime)
- large 512-bit fixed number (assuming prime)
- client generated 512-bit number

Now, the client keeps B secret but will send A to the server. The server would already know , , and .

From here, the server processes A and does some unknown operations with it. This is the process that I am currently trying to understand.

After the server does its work, it will send some data back to the client (a 20 byte stream of data) which will XOR it byte by byte with the first 20 bytes of B and then passes the result into a function that modifies it some more, basically performing some additional operations on it to come up with the final key. This key is then used to build the encryption and decryption keys through a "scramble" algorithm (i.e. Like a sliding block puzzle).

Considering all of this, this is what I've come to understand from my research:

* The discrete logarithm problem basically tells us that the a value the client generates cannot be realistically calculated by the server over the entire range of possibilities it might be. Even if we could brute force our way through 0 to the max 512-bit number, the modulus would eventually wrap around and generate multiple solutions.

* According to RSA:
Quote:
Encryption: Sender A does the following:-
1. Obtains the recipient B's public key (n, e).
2. Represents the plaintext message as a positive integer m [see note 4].
3. Computes the ciphertext c = m^e mod n.
4. Sends the ciphertext c to B.


However in this case, the client generates the 'e' and the 'm' is a fixed value known by both parties. The client's 'e' ('a') could not possibly be a private or public key. (Referring to RSA via this link as it keeps it pretty simple.)

* In a Diffie-Hellman key exchange, which is what the process most similarly resembles at first, the server does not send a B value (reference). In addition, no K value is ever calculated on the client side. I've worked with DH in the past a bit, so I'm sure from my current research this is not being used.

* In the Three-pass protocol, I see a lot of correlations between the process and what goes on, however no such full process takes place. The client sends 64 bytes to the server and receives 20 bytes in return and the entire process is done as messages are now ready to be sent encrypted and be decrypted.

* In general Public-key crypto, which I believe something along those lines is going on here,
Quote:
Not all asymmetric key algorithms operate in precisely this fashion. The most common ones have the property that Alice and Bob each own two keys, one for encryption and one for decryption. In a secure asymmetric key encryption scheme, the private key should not be deducible from the public key. This is known as public-key encryption, since an encryption key can be published without compromising the security of messages encrypted with that key.

That is understandable, but it still doesn't answer the question of how the server could know anything about the client's B given only A while not knowing a or how it could even "sync" values.

All of those points leads me to the following:
1. The server cannot generate 'a' in this situation, correct?
2. Given A, there is no way to deduce B not knowing 'a', correct?
3. Assuming I did not miss anything and 1 & 2 are indeed infeasible, what other math concepts am I missing here?

What's driving me crazy is that, for any A sent to the server with the properly generated B value stored locally, the server responds with a different 20 byte sequence of values that is used each time to generate the final set of values used to seed the encryption/decryption tables.

Everything seems to point towards some sort of public key system, but the setup that is being used does not appear to be public key related since the exponent a is client calculated. It looks like the server is calculating 'a' itself but the research shows this shouldn't be happening. Lastly, a look up table is impossible since it'd require (256 ^ 20) * 64 bytes of storage.

Does anyone have any insights or other reference material that could help me out here? I'm just basically trying to understand what else can go on in the server given the apparent limited information it's sent and sends back.

Thanks!

No conversion from bool to string -- user error or bug?

11 August 2010 - 11:43 AM

I didn't see any mention in the docs about limitations of the bool type, but maybe I missed it. I'm trying to using the Print function included with the tutorial on a bool type but I am getting an error. If I use say uint8 instead, it works fine:

Quote:

script (1, 1) : INFO : Compiling void print_bool(bool)
script (1, 45) : ERROR : No conversion from 'string@&' to math type available.
Build() failed.


Here is a example that shows it:
// Comment out to see expected behavior with uint8
#define BUG_BOOL

#include "angelscript.h"
#include "scriptstring.h"
#include <cassert>

asIScriptEngine * engine = 0;
asIScriptContext * ctx = 0;
void Cleanup()
{
if(ctx)
{
ctx->Release();
ctx = 0;
}
if(engine)
{
engine->Release();
engine = 0;
}
}

void MessageCallback(const asSMessageInfo *msg, void *param)
{
std::string type = "UNKNOWN";
if( msg->type == asMSGTYPE_ERROR )
{
type = "ERROR";
}
else if( msg->type == asMSGTYPE_WARNING )
{
type = "WARN";
}
else if( msg->type == asMSGTYPE_INFORMATION )
{
type = "INFO";
}
printf("%s (%d, %d) : %s : %s\n", msg->section, msg->row, msg->col, type.c_str(), msg->message);
}

void PrintString(std::string & str)
{
printf("%s\n", str.c_str());
}

int main(int argc, char **argv)
{
int r;

atexit(Cleanup);

engine = asCreateScriptEngine(ANGELSCRIPT_VERSION);
if( engine == 0 )
{
printf("Failed to create script engine.\n");
return -1;
}

r = engine->SetEngineProperty(asEP_ALLOW_UNSAFE_REFERENCES, true); assert( r >= 0 );
r = engine->SetMessageCallback(asFUNCTION(MessageCallback), 0, asCALL_CDECL); assert( r >= 0 );

RegisterScriptString(engine);

std::string options = asGetLibraryOptions();
if(options.find("AS_MAX_PORTABILITY") != std::string::npos)
{
printf("This configuration is not being supported. [%s]\n", options.c_str());
return -1;
}

r = engine->RegisterGlobalFunction("void Print(string &in)", asFUNCTION(PrintString), asCALL_CDECL); assert( r >= 0 );

#ifdef BUG_BOOL
std::string script = "void print_bool(bool bval) { Print('bval: ' + bval + '\\n'); }";
#else
std::string script = "void print_bool(uint8 bval) { Print('bval: ' + bval + '\\n'); }";
#endif

asIScriptModule *mod = engine->GetModule(0, asGM_ALWAYS_CREATE);
r = mod->AddScriptSection("script", script.c_str(), script.size());
if( r < 0 )
{
printf("AddScriptSection() failed.\n");
return -1;
}

r = mod->Build();
if( r < 0 )
{
printf("Build() failed.\n");
return -1;
}

ctx = engine->CreateContext();
if( ctx == 0 )
{
printf("Failed to create the context.\n");
return -1;
}
#ifdef BUG_BOOL
int funcId = engine->GetModule(0)->GetFunctionIdByDecl("void print_bool(bool bval)");
#else
int funcId = engine->GetModule(0)->GetFunctionIdByDecl("void print_bool(uint8 bval)");
#endif
if( funcId < 0 )
{
printf("The function was not found.\n");
return -1;
}

r = ctx->Prepare(funcId);
if( r < 0 )
{
printf("Failed to prepare the context.\n");
return -1;
}

#ifdef BUG_BOOL
bool val = true;
#else
unsigned char val = 1;
#endif
ctx->SetArgByte(0, val);

r = ctx->Execute();
if( r != asEXECUTION_FINISHED )
{
if( r == asEXECUTION_ABORTED )
{
printf("The script was aborted before it could finish. Probably it timed out.\n");
}
else if( r == asEXECUTION_EXCEPTION )
{
printf("The script ended with an exception.\n");
int funcID = ctx->GetExceptionFunction();
asIScriptFunction *func = engine->GetFunctionDescriptorById(funcID);
printf("func: %s\n", func->GetDeclaration());
printf("modl: %s\n", func->GetModuleName());
printf("sect: %s\n", func->GetScriptSectionName());
printf("line: %i\n", ctx->GetExceptionLineNumber());
printf("desc: %s\n", ctx->GetExceptionString());
}
else
{
printf("The script ended for some unforeseen reason (%i).\n", r);
}
}
else
{
}
return 0;
}



I am on Windows XP, Visual Studio 2008 (w/ SP1), using version 2.19.1 compiling with source as part of the project. Thanks ahead of time for any advice on this issue!

[Lua] Runtime compiling from C/C++

21 April 2010 - 01:42 AM

Recently I've been interested in the network applications of Lua for prototyping out ideas in a server/client environment. I was thinking something along the lines of custom logic generated on a server being sent to a client for execution. For simple security reasons, it'd be best if this logic were not sent in a text format that could easily be modified by anyone, but rather in a compiled form. In order to do that though, I needed the ability to generate compiled Lua bytecode at runtime (from C/C++) without having to use the Luac program. I didn't want to have to write the logic to a file and then read in the compiled results to use. I definitely didn't want it to be bound by disk i/o either. However, I've not really been able to find any existing code that actually does this or any topics of the potential uses of it. So, I spent a little time hacking away at Luac's code to come up with a solution. This is what I ended up with, it seems to work fine, but I've not gone through any intensive testing yet. It is really just a basic PoC project that could be extended in many ways. Source:
extern "C"
{
	#include "zlib/zlib.h"

	#include "lua/lua.h"
	#include "lua/lualib.h"
	#include "lua/lauxlib.h"

	#include "lua/ldo.h"
	#include "lua/lundump.h"
}

#include <string>
#include <sstream>

#include <stdlib.h>
#include <memory.h>
#include <string.h>

// Formats a stream of data into a hex/ascii dump
std::string DumpToString(void * data_, size_t count);

typedef struct TStreamBuffer
{
	const char * src;
	size_t sze;
	size_t ind;
	char buff[LUAL_BUFFERSIZE];
} TStreamBuffer;

typedef struct TCompiledBuffer
{
	char * ptr;
	size_t sze;
	int result;
} TCompiledBuffer;

int MyCustomWriter(lua_State * L, const void * src, size_t sze, void * userdata)
{
	TCompiledBuffer * cb = (TCompiledBuffer *)userdata;
	UNUSED(L);
	cb->ptr = (char *)realloc(cb->ptr, cb->sze + sze);
	if(cb->ptr == NULL)
	{
		return 1;
	}
	memcpy(cb->ptr + cb->sze, src, sze);
	cb->sze += sze;
	return 0;
}

const char * MyBufferStreamer(lua_State * L, void * userdata, size_t * sze)
{
	TStreamBuffer * sb = (TStreamBuffer *)userdata;
	*sze = sizeof(sb->buff);
	UNUSED(L);
	memset(sb->buff, 0, sizeof(sb->buff));
	if(sb->ind >= sb->sze)
	{
		*sze = 0;
		return NULL;
	}
	if(sb->sze - sb->ind < *sze)
	{
		*sze = sb->sze - sb->ind;
	}
	memcpy(sb->buff, sb->src + sb->ind, *sze);
	sb->ind += *sze;
	return sb->buff;
}

TCompiledBuffer CompileString(lua_State * L, const char * input)
{
	const Proto * proto = 0;
	TCompiledBuffer cb = {0};
	TStreamBuffer sb = {0};
	sb.sze = strlen(input);
	sb.src = input;
	cb.result = lua_load(L, MyBufferStreamer, &sb, "");
	if(cb.result != 0)
	{
		return cb;
	}
	lua_lock(L);
	proto = clvalue(L->top+(-1))->l.p;
	cb.result = luaU_dump(L, proto, MyCustomWriter, &cb, 1);
	lua_unlock(L);
	if(cb.result != 0)
	{
		cb.sze = 0;
		if(cb.ptr)
		{
			free(cb.ptr);
			cb.ptr = 0;
		}
	}
	return cb;
}

int main(int argc, char * argv[])
{
	int res;

	lua_State * L = 0;
	lua_State * M = 0;

	TCompiledBuffer cb1;
	TCompiledBuffer cb3;

	L = lua_open();
	luaL_openlibs(L);

	cb1 = CompileString(L, "print(\"Hello World!\")\n");

	cb3 = CompileString(L, 
"-- create private index""\n"
"local index = {}""\n"
"""\n"
"-- create metatable""\n"
"local mt =""\n"
"{""\n"
"	__index = function (t,k)""\n"
"		print(\"*access to element \\\"\" .. tostring(k) .. \"\\\" in table \\\"\" .. t[index][\"name\"] .. \"\\\"\")""\n"
"		return t[index][k]   -- access the original table""\n"
"	end,""\n"
"""\n"
"	__newindex = function (t,k,v)""\n"
"		print(\"*update of element \\\"\" .. tostring(k) .. \"\\\" to \\\"\" .. tostring(v) .. \"\\\" in table \\\"\" .. t[index][\"name\"] .. \"\\\"\")""\n"
"		t[index][k] = v   -- update original table""\n"
"	end""\n"
"}""\n"
"""\n"
"function track (t)""\n"
"  local proxy = {}""\n"
"  proxy[index] = t""\n"
"  setmetatable(proxy, mt)""\n"
"  return proxy""\n"
"end""\n"
"""\n"
"tbl1 = {}""\n"
"tbl1[\"name\"]=\"tbl1\"""\n"
"tbl1 = track(tbl1)""\n"
"""\n"
"tbl1[\"id\"] = 0""\n"
);

	lua_close(L);
	L = 0;

	M = lua_open();
	luaL_openlibs(M);
	if(cb1.result == 0)
	{
		uLongf destLen;
		Bytef * dest;

		res = luaL_loadbuffer(M, cb1.ptr, cb1.sze, "");
		res = lua_pcall(M, 0, LUA_MULTRET, 0);

		destLen = cb1.sze * 2 < 1024 ? 1024 : cb1.sze * 2;
		dest = (Bytef *)malloc(destLen);
		memset(dest, 0, destLen);
		res = compress2(dest, &destLen, (Bytef *)cb1.ptr, cb1.sze, Z_BEST_COMPRESSION);
		if(res != Z_OK)
		{
			printf("compress2 failed with error: %i\n", res);
		}
		printf("Original (%i):\n", cb1.sze);
		printf("%s\n", DumpToString(cb1.ptr, cb1.sze).c_str());
		printf("Compressed (%i):\n", destLen);
		printf("%s\n", DumpToString(dest, destLen).c_str());
		free(cb1.ptr);
		free(dest);
	}
	else
	{
		printf("There was an error compiling the input string.\n");
	}

	if(cb3.result == 0)
	{
		uLongf destLen;
		Bytef * dest;

		res = luaL_loadbuffer(M, cb3.ptr, cb3.sze, "test");
		res = lua_pcall(M, 0, LUA_MULTRET, 0);

		destLen = cb3.sze * 2 < 1024 ? 1024 : cb3.sze * 2;
		dest = (Bytef *)malloc(destLen);
		memset(dest, 0, destLen);
		res = compress2(dest, &destLen, (Bytef *)cb3.ptr, cb3.sze, Z_BEST_COMPRESSION);
		if(res != Z_OK)
		{
			printf("compress2 failed with error: %i\n", res);
		}
		printf("Original (%i):\n", cb3.sze);
		printf("%s\n", DumpToString(cb3.ptr, cb3.sze).c_str());
		printf("Compressed (%i):\n", destLen);
		printf("%s\n", DumpToString(dest, destLen).c_str());
		free(cb3.ptr);
		free(dest);
	}
	else
	{
		printf("There was an error compiling the input string.\n");
	}
	lua_close(M);
	M = 0;

	return 0;
}

// Formats a stream of data into a hex/ascii dump
std::string DumpToString(void * data_, size_t count)
{
	char msg[256] = {0};
	std::stringstream smsg;
	int sze = count;
	int outputsize = sze;
	if(outputsize % 16 != 0) outputsize += (16 - outputsize % 16);
	if(outputsize == 0) outputsize = 16;
	int ctr = 0;
	char ch[17] = {0};
	int x1 = 0;
	unsigned char * data = reinterpret_cast<unsigned char *>(data_);
	for(int x = 0; x < outputsize; ++x)
	{
		if(x < sze)
		{
			unsigned char b = data[x];
			sprintf_s(msg + ctr, sizeof(msg) - ctr, "%.2X ", b);
			ch[x1] = (isprint(b) && !isspace(b) ? b : '.');
		}
		else
		{
			sprintf_s(msg + ctr, sizeof(msg) - ctr, "   ");
			ch[x1] = '.';
		}
		x1++;
		ctr += 3;
		if((x+1) % 16 == 0)
		{
			x1 = 0;
			sprintf_s(msg + ctr, sizeof(msg) - ctr, "  %s", ch);
			ctr += 18;
			sprintf_s(msg + ctr, sizeof(msg) - ctr, "\n");
			try
			{
				smsg << msg;
			}
			catch (std::exception & e)
			{
				printf("[%s] %s\n", __FUNCTION__, e.what());
				return "";
			}
			ctr = 0;
			memset(msg, 0, sizeof(msg));
		}
	}
	return smsg.str();
}

Example Output:
Hello World!
Original (97):
1B 4C 75 61 51 00 01 04 04 04 08 00 00 00 00 00   .LuaQ...........
00 00 00 00 00 00 00 00 00 00 02 02 04 00 00 00   ................
05 00 00 00 41 40 00 00 1C 40 00 01 1E 00 80 00   ....A@...@......
02 00 00 00 04 06 00 00 00 70 72 69 6E 74 00 04   .........print..
0D 00 00 00 48 65 6C 6C 6F 20 57 6F 72 6C 64 21   ....Hello.World!
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
00                                                ................

Compressed (71):
78 DA 93 F6 29 4D 0C 64 60 64 61 61 E1 60 40 05   x...)M.d`daa.`@.
4C 4C 2C 40 92 15 88 1D 1D 18 18 64 1C 18 18 E5   LL,@.......d....
18 1A 18 98 80 7C 16 36 20 51 50 94 99 57 C2 C0   .....|.6.QP..W..
C2 0B 64 7A A4 E6 E4 E4 2B 84 E7 17 E5 A4 28 32   ..dz....+.....(2
A0 03 00 AC 3F 09 B4                              ....?...........

*update of element "id" to "0" in table "tbl1"
Original (727):
1B 4C 75 61 51 00 01 04 04 04 08 00 00 00 00 00   .LuaQ...........
00 00 00 00 00 00 00 00 00 00 02 04 17 00 00 00   ................
0A 00 00 00 4A 80 00 00 A4 00 00 00 00 00 00 00   ....J...........
49 80 00 80 A4 40 00 00 00 00 00 00 49 80 80 80   I....@......I...
A4 80 00 00 00 00 00 00 00 00 80 00 87 80 00 00   ................
8A 00 00 00 87 C0 00 00 85 C0 00 00 89 C0 40 82   ..............@.
85 80 00 00 C5 C0 00 00 9C 80 00 01 87 C0 00 00   ................
85 C0 00 00 89 80 C1 82 1E 00 80 00 07 00 00 00   ................
04 08 00 00 00 5F 5F 69 6E 64 65 78 00 04 0B 00   .....__index....
00 00 5F 5F 6E 65 77 69 6E 64 65 78 00 04 06 00   ..__newindex....
00 00 74 72 61 63 6B 00 04 05 00 00 00 74 62 6C   ..track......tbl
31 00 04 05 00 00 00 6E 61 6D 65 00 04 03 00 00   1......name.....
00 69 64 00 03 00 00 00 00 00 00 00 00 03 00 00   .id.............
00 00 00 00 00 07 00 00 00 0A 00 00 00 01 02 00   ................
08 11 00 00 00 85 00 00 00 C1 40 00 00 05 81 00   ..........@.....
00 40 01 80 00 1C 81 00 01 41 C1 00 00 84 01 00   .@.......A......
00 86 81 01 00 86 01 41 03 C1 41 01 00 D5 C0 81   .......A..A.....
01 9C 40 00 01 84 00 00 00 86 80 00 00 86 40 00   ..@...........@.
01 9E 00 00 01 1E 00 80 00 06 00 00 00 04 06 00   ................
00 00 70 72 69 6E 74 00 04 15 00 00 00 2A 61 63   ..print......*ac
63 65 73 73 20 74 6F 20 65 6C 65 6D 65 6E 74 20   cess.to.element.
22 00 04 09 00 00 00 74 6F 73 74 72 69 6E 67 00   "......tostring.
04 0D 00 00 00 22 20 69 6E 20 74 61 62 6C 65 20   .....".in.table.
22 00 04 05 00 00 00 6E 61 6D 65 00 04 02 00 00   "......name.....
00 22 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ."..............
00 00 00 00 00 00 00 0C 00 00 00 0F 00 00 00 01   ................
03 00 0B 14 00 00 00 C5 00 00 00 01 41 00 00 45   ............A..E
81 00 00 80 01 80 00 5C 81 00 01 81 C1 00 00 C5   .......\........
81 00 00 00 02 00 01 DC 81 00 01 01 02 01 00 44   ...............D
02 00 00 46 42 02 00 46 42 C1 04 81 82 01 00 15   ...FB..FB.......
81 02 02 DC 40 00 01 C4 00 00 00 C6 C0 00 00 C9   ....@...........
80 80 00 1E 00 80 00 07 00 00 00 04 06 00 00 00   ................
70 72 69 6E 74 00 04 15 00 00 00 2A 75 70 64 61   print......*upda
74 65 20 6F 66 20 65 6C 65 6D 65 6E 74 20 22 00   te.of.element.".
04 09 00 00 00 74 6F 73 74 72 69 6E 67 00 04 07   .....tostring...
00 00 00 22 20 74 6F 20 22 00 04 0D 00 00 00 22   ...".to."......"
20 69 6E 20 74 61 62 6C 65 20 22 00 04 05 00 00   .in.table.".....
00 6E 61 6D 65 00 04 02 00 00 00 22 00 00 00 00   .name......"....
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
00 12 00 00 00 17 00 00 00 02 01 00 05 09 00 00   ................
00 4A 00 00 00 84 00 00 00 49 00 00 01 85 00 00   .J.......I......
00 C0 00 80 00 04 01 80 00 9C 40 80 01 5E 00 00   ..........@..^..
01 1E 00 80 00 01 00 00 00 04 0D 00 00 00 73 65   ..............se
74 6D 65 74 61 74 61 62 6C 65 00 00 00 00 00 00   tmetatable......
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
00 00 00 00 00 00 00                              ................

Compressed (395):
78 DA 9D 52 41 4A 03 41 10 AC DE 4C 4C 34 8A 28   x..RAJ.A...LL4.(
06 0F 8A 2C 39 7A F3 07 1B 51 41 F1 E2 5D 94 35   ...,9z...QA..].5
19 25 98 6C 82 19 D1 E3 6C 30 11 7D 43 F0 94 87   .%.l....l0.}C...
64 C1 08 3E C0 9B 8F B1 66 4D 14 45 3D D8 50 3B   d..>....fM.E=.P;
D5 33 CD 74 55 CF AE EC 5F 86 07 10 A5 54 1E 5F   .3.tU..._....T._
C3 53 CB FC CE 10 7B 16 18 8C 77 77 2D EC 20 98   .S....{...ww-...
70 6B 07 76 52 6F 71 4B 7E 4F 76 3B 04 BA C4 DD   pk.vRoqK~Ov;....
30 E8 74 B9 37 22 EF 5B C8 C7 BE 4D 3A 6B AC CF   0.t.7".[...M:k..
B1 36 6D 7B 7C 5C 8B AA FA 1A AA 90 26 91 BE 1A   .6m{|\......&...
E7 53 CC CD 45 58 39 87 CA 3A 7A 52 DF 78 67 51   .S..EX9..:zR.xgQ
D8 D0 50 19 B2 5A 15 99 89 84 09 C9 8D 85 8B 87   ..P..Z..........
FC 02 D7 2E 91 50 74 36 06 02 B1 58 8D 21 E5 04   .....Pt6...X.!..
B8 11 A0 17 0B 7A 52 CE 24 65 C1 CB 30 96 7E 00   .....zR.$e..0.~.
B9 61 7D 8F C2 7B E4 0F BC C6 89 75 52 52 3D AD   .a}..{.....uRR=.
8B 5A 64 A0 8A A4 EB 61 A5 A2 DB 6D DF 34 7D 5D   .Zd....a...m.4}]
D7 0D 1D 19 BF 04 35 ED 94 36 DB 86 75 67 50 73   ......5..6..ugPs
CC 4A 7E 2D F2 4D 78 52 D7 EE FC 53 BF E7 CE F0   .J~-.MxR...S....
53 CC 12 F3 CE 41 06 85 25 AE 23 C7 CB C0 36 1D   S....A..%.#...6.
58 3A 38 A4 83 98 0E 46 B1 7B 29 C8 2B 73 F1 04   X:8....F.{).+s..
5B BC 72 67 D3 23 12 15 77 04 C5 D8 F3 5E E9 E2   [.rg.#..w....^..
91 65 4F 1C FD B3 B5 F8 18 FD 77 37 97 AD 6A 68   .eO.......w7..jh
B4 DF 3C FD DD 4D 2E 75 43 BF A5 FF 3A 5B 24 DC   ..<..M.uC...:[$.
8F 45 B1 59 77 F7 1E E1 E6 BD 4B 83 EE 9D 86 54   .E.Yw.....K....T
A7 E8 B0 1F 58 39 1A CF 5E 9C 5A D7 AE AD 4D 43   ....X9..^.Z...MC
9B 30 6D 88 BF E2 0D AC 13 75 FB                  .0m......u......

Press any key to continue . . .

Download: luaRuntimeCompile (complete project) Basically, I want to explore sending logic to a client to execute rather than sticking to the traditional approach of sending data. I want to cut out the client's low level data processing and aim for something higher level now. I don't actually need the ability to compile Lua scripts at run time for that, since sending strings and calling DoString is good enough, but having it makes it all the more practical for a real demo. Just though I'd share in case anyone else had a need like this. Please feel free to leave any questions or comments!

Do you think this is a feasible server architecture design?

02 August 2009 - 09:23 AM

This post is a bit of an extension of my previous two posts Trying to get my IOCP code on par with expected performance and [Article] Using UDP with IOCP. I've begun working on what I hope would be a scalable high performance server architecture useful in an online game with a large number of concurrent users. This is how I envision it would work: The public interface to the network as a whole is handled on the first layer. There would be 1..n of these IOCP servers that handle the network traffic for the entire system. Packet security and physical validation would take place in this layer. When packets are deemed to be complete and physically valid, they would be passed to the second layer. I achieve scalability here by simply being able to add more servers and update the client with a list to connect to. The second layer is a collection of 1..m switch servers that route complete packets from the first layer to the desired destination in the third layer as well as packets from the third layer to other servers in the third layer. The database is used to store connection specific data sent from the first and third layers so that information can be kept to route the response packets back to where they need to go. That logic would be handled through a specific packet protocol not yet described. I achieve scalability here by simply being able to add more servers and update the 1st/3rd layer servers with a list of available switches to use. The third layer is composed of all of the "logical" servers for the system. If servers need to talk to each other, they will go through the second layer so the switch servers can router the packets to the other servers that need the information. Likewise when a server in layer 3 needs to return data to a client, it just sends the data to the second layer with a specific protocol so the switch server then sends it back to the correct layer 1 server that contains the client to send to. I don't have a blueprint on the exact packet protocol for the internal communication, but it would be really simple, it'd contain the local and remote addresses for the client and the original server, possibly some flags and then the data to be sent. There is no specific UDP or TCP protocol specified right now, but ideally that would be up to the user to choose which one would work best. I believe for now I'd just use TCP, but UDP could be used with a simple protocol like enet. Each "Server" listed on the image does not necessary mean a physical machine on the network, but the design should allow for that. It could just as well be another process! For starting out, I will have one physical server and have one IOCP process in layer one, one switch process in layer 2, the database in layer 2, and one server in layer 3 (world). My reasoning for this design has to do with some recent success with splitting up a networking task into two parts, a low level part that does all the performance critical work in C++ and a high level part that uses readily formatted data in C#. With this setup, I envision all of the performance critical networking traffic being processed by C++ IOCP processes and my actual "logical servers" can be done in a variety of languages as I need. The second layer routing servers is to easily allow all the servers to interact as needed. As of right now I am working on the protocol between the 2nd and 1st/3rd layers. The layer 1 server design is done, so that code can be used in the 2nd and 3rd layers for my servers there, even though I plan on making another server in C# for the 3rd layer. I already have an ambition to actually complete this design and see how it works in practice myself, but I'd like to know what everyone else thinks about it. Am I overlooking anything and are there any serious flaws? So please, let me have all your thoughts about it! [smile] [Edited by - Drew_Benton on October 5, 2009 11:10:39 PM]

[Article] Using UDP with IOCP

28 April 2009 - 01:13 AM

I finally had some time to sit down and write a little article over using UDP and IOCP together. This effort stemmed from my earlier attempts to get IOCP and TCP performance up to par earlier this month. I'll eventually get back to IOCP and TCP, but for now UDP meets the needs of my desired goals. Included in the package is the article itself in DOC and PDF formats, the actual code for the library, as well as a simple echo client/server demo showing how to use it all. You need Boost to compile the code and Visual Studio 2005 or above since I use variable arguments via a macro. I hope the article and source code serve as a good resource to programmers hoping to learn how to properly use UDP with IOCP as there are not any resources on that topic right now. I am fairly confident in the correctness of my work since I've done months of research, development, and trial and error to make sure I wasn't messing anything up. I'm sure there may be minor things though to nitpick. Writing a Windows High Performance Scalable UDP Server Using Input Output Completion Ports The concept itself is of most use as an academic resource right now. Now that RakNet is free for indie development, that library is the best free choice available for game developers looking for UDP. In addition there is enet for a lower level UDP library for games that is more lightweight than RakNet. This library is at a lower level than enet, all it does is handle sending and receiving. That means most people might not have an immediate use for it, but those that do, I hope it helps. I'm open to any feedback! [smile]

PARTNERS