Another item to keep in mind is that UDP is no longer a guaranteed win in any case. Many routers are configured to prioritize tcp over any form of UDP and as such have a very high drop rate for games. Given this, the increased logic of routers to handle tcp packets better, along with the generally higher loss rate of UDP compared to even a few years ago, TCP *can* often provide better gameplay than even the best UDP. Keep in mind, I'm not saying UDP sucks and you should not use it, just that as far as the win scenario goes, it is no longer easily cut and dry like it was.
AllEightUpMember Since 24 Apr 2011
Online Last Active Today, 06:45 AM
- Group Moderators
- Active Posts 523
- Profile Views 8,693
- Submitted Links 0
- Member Title Crossbones+
- Age Age Unknown
- Birthday Birthday Unknown
Posted by AllEightUp on 12 February 2014 - 08:40 PM
Posted by AllEightUp on 04 January 2014 - 11:02 AM
It makes me think it was because game engines, libraries, reusable/recyclable functions are not possible in assembly unlike C/C++ etc, but please correct me on whatever I'm wrong on, and if there's such thing as shared engines, libraries etc in assembly some examples or mentions would be nice.
There is actually a fundamental reason which caused this more than anything else. Let me first mention that I'm doing this on vague memory from the way back machine, so the details may be off, please don't nuke me for such mistakes too much.. Anyway, take a platform such as the Super Nintendo and go over the spec's for the CPU and memory. The processor was basically a 3.5Mhz split 8bit/16bit machine. In general this was simply the 6502 instruction set enhanced with 16 bit registers and if I remember correctly a 16bit multiplier. The problem here is the 3.5Mhz clock rate which directly maps to how many instructions could be processed per second and the lack of registers (only had accumulator and x/y index registers for general usage) exaggerated the problems. Unlike current CPU's where clock rates don't map directly to instructions per second due to pipelining, caching etc, there was no pipelining or cache on that CPU and as such the clock rate directly mapped to the number of bytes of memory which could be read and that mapped directly to the number of instructions which could be processed. (NOTE: zero page could be considered cache as it was faster to read/write from, but you had to manage it manually.)
You could basically fetch or store 3.5 million bytes of memory per second. In the case of a simple NOP (0xEA opcode, I'll never forget that one) instruction this meant the CPU loaded the piece of memory pointed to by the PC, incremented the PC and setup the instruction processor on the first cycle. The second cycle of the instruction performed the actual NOP work, which of course did nothing. So, you have just split the 3.5 million clock cycles in half to 1.75 million per second with the fastest "do nothing" instruction on the CPU. Most instructions could take significantly more clocks and as such you generally ran less than a million instructions per second. So, using round numbers, assume your typical code could run 300,000 instructions per second, average of about 3.5 cycles per instruction which is somewhat realistic as I remember it, and you want to target 30FPS, you have only 10,000 instructions to use each frame. Oh, but there's still more loss, you have to update any hardware/graphics typically only during the vertical blank or you cause tearing and other undesirables and of course there are interrupts etc taking up some of those instructions. So, you might only be able to use 7000 instructions per frame and still hit 30FPS.
So, why were the engines not made reusable and generic? Quite simply because any *generic* code was refined down to do only exactly what was needed in order to fit into the hard limits required to maintain a given framerate. In the process of refining the code you are obviously removing any concept of generality involved. There really were no other reasons, it wasn't a language issue and no compiler could touch the required level of optimization, it was just the only practical way to do things at the time with the available tools.
Posted by AllEightUp on 15 December 2013 - 01:08 PM
I know how to use sockets, at least at a reasonable level
We obviously differ on what a "reasonable level" means as your questions are all "reasonably expected" that you know the answer before you tackle something this large. I'm not trying to be a complete ass here, just stating that your questions are very basic networking 101 items you should have learned. As to select, it was not really a seriously suggested solution, you should really be looking at IO completion ports on Windows for any real scaling. Of course that is a massive pain in the ass to use correctly and won't be covered here. At this point I suggest Google is your friend and you should look for "network api select" to find tutorials on that and/or "IO completion ports" to start the merry journey down the MS black hole of a crappy API's...
Posted by AllEightUp on 11 December 2013 - 11:41 AM
fread/fwrite are blocking wrappers around the OS's internal non-blocking file-system API.
Instead of using a non-blocking-API, wrapped in a blocking API, wrapped in a thread to make it non-blocking, you should just use the native OS APIs
You still might want an "IO" thread for running decompression, or alternatively just treat decompression as a regular job for your worker threads.
Even without decompression you still want to leave the IO calls in a worker thread for most API's. The reasoning is not CPU performance related as 99% of the time these threads should be sitting in a sleep state waiting for the next event. The problem you have with calling these API's from the main thread is the inconsistent latency you introduce which can cause extreme io read/write performance losses. There are two cases which specifically come to mind to explain this. One is Windows specific, unless you go full court IOCP you are likely going to be using the callback variations of the API which means the thread needs to go into a wakeable sleep regularly or it will never fire the events. Doing that on the main game thread would be less than desirable and also given the randomness of most game loops would put in random latencies. The latencies are the bad part for all the API's though as they tend to add up and feedback on each other in the case of file IO.
A simple high level example of the the Io latency causing issues, obviously avoiding details so just the gist here. Say you read at 1k chunks and you are reading a file piece by piece. (This is not how you want to do it, again just for example assume it is not horrible dumb. ) The OS likely reads an entire track in a single revolution and say 20k of the file is on the track. If you don't service the event for the last 1k fast enough the OS may get another request and start moving to service that and flush the 19k worth of data you *could* have gotten if you serviced the IO faster. So, your little bit of latency just cost you 19k of potentially immediately available data, delayed when the OS will get back to servicing your requests and of course you may now have to wait for the drive head to come back from very far away so it all adds up quickly.
Overall, the number of threads you have is not necessarily something you have to work at minimizing to an extreme degree. In the cases of file and audio work, a thread per system is completely viable and has no measurable impact on the remaining systems. They sleep most of the time, wake up to do a tiny amount of work inbetween your primary processing and go back to sleep without impacting on performance at all. This portion of your architecture should really be a non-issue, just use the threads in this case as that is how the OS is designed to use them, your work queue and distributing the real work of the game, that's the tricky bit with a truck load of problems you'll have to worry about.
Posted by AllEightUp on 07 December 2013 - 10:06 AM
In terms of scaling when applied to games work (should apply to just about anything), it scales very well and even dynamically. By simply adjusting the memory size of the cache you can increase/decrease load on SQL to the point that it makes the most sense for you. If things get really big, it scales horizontally also, you can move the cache onto a standalone box with basically nothing more than massive amounts of memory. If that ends up bottlenecking you can split it over multiple machines by breaking up which caches contain different portions of the access API. At the point the DB starts slowing down even with the cache on top you can subdivide the SQL servers into primary table access for each cache separately etc. This is basically an algorithmic optimization instead of arbitrarily throwing data away (which still may be valid) or just throwing bigger boxes at a problem hoping vertical scale can keep up.
Posted by AllEightUp on 30 November 2013 - 10:22 AM
Git and SVN at the start are "somewhat" similar. If you already have a repository on the server, selecting a local folder just tells Git where to do the initial clone on your machine. I.e. your local copy of all the files. If you are simply trying to post files up to github initially, I suggest doing it in a little bit of a two step manner. Create an initial empty repository which it sounds like what you are doing but point the local folder to an empty location. When this completes, copy the files into the new folder and then commit them, then push to fill the remote repository. This is simply a method of getting started easily and without hassles. Using the command line you can initialize a local repository and then target a remote to push into, but usually the UI's are not setup for such things so this is the easy solution.
Posted by AllEightUp on 30 November 2013 - 09:47 AM
Is there anything wrong with this approach? Please do remember that i wanted to try and find a compromise between being explicit about dependencies, while at the same time allowing this kind of gameplay code to be less strict, since it is likely to change more rapidly and such (which is why we use scripting languages for that kind of code - it does not need to be so strict)
In most cases this does not really help much, but im sure there are cases where such a construct might be useful.
The construct is pretty evil as others have pointed out. And the final sentence should have told you that you probably need to rethink the issue and come up with a more suitable reason for wanting this abstraction. I'll provide a reason and an example of how I solved such a problem simply as an alternative solution. My reasoning for needing a transformation such as this was that I was configuring the main loop from script and some things could be C++ and others could be script (lua specifically) experiments. C++ tended to be stabilized code which I could use more common argument passing, construction references etc, but until I moved everything out of script I pretty much needed the update to be "void update( void )" since it was stored in a vector to be configured at startup. I figure this is a valid and concrete reason for needing to remove parameters from the update calls, yet I still wanted the C++ to take parameters.
In steps C++11 and the new shiny built in adapter pattern implementation. A combination of std::function, std::bind and lamba's can pretty much implement any form of adapter you can ever imagine. Unfortunately your example is too simplistic in order to show a decent usage, so I'll make a simple example based on a kinda real case:
// Assumption. I have experimental items implemented in lua. // Problem. I can't pass the luaState as a parameter since the experiments // are usually in unique instances of luaState. // How to pass the appropriate luaState to a generic "update" function? // Update loop consists of: typedef std::function< void ( void ) > Updater_t; typedef std::vector< Updater_t > UpdateVector_t; UpdateVector_t gUpdates; // Evil, horrible global update list. Just for example. LuaState gExperiment1( "Experimental1.lua" ); LuaState gExperiment2( "Experimental2.lua" ); // In order to update experiment 1, I need to call: // gExperimental.Call( gExperimental1.Ref( "Update" ) ); // Getting a ref is kinda expensive so best not to call it every time... // Using bind as an adapter: gUpdates.push_back( std::bind( &LuaState::Call, std::ref( gExperimental1 ), gExperimental1.Ref( "Update" ) ) ); gUpdates.push_back( std::bind( &LuaState::Call, std::ref( gExperimental2 ), gExperimental2.Ref( "Update" ) ) ); // Run the updates. for( Update_t& update : gUpdates ) update();If you are not familiar with std::bind, I highly suggest you get familiar. It is one of the best ways to decouple code and implement function level adapter patterns you are likely to find.
Anyway, hopefully this is appropriate to your actual goals or at least provides another alternative which avoids that construct which is just hiding a global in fancy templates.
Posted by AllEightUp on 29 November 2013 - 11:12 AM
Why so many threads? The I/O subsystem can only efficiently read one file at a time anyway. There's not necessarily a lot of benefit to having more than one thread when I/O bound.
Actually, while there can be only one active io operation at a time, it is not always a linear process through a specific file. Using multiple threads is questionable but in the case of using something simplistic like fread, it can make sense and provide a performance boost. The reason is that the underlying IO for just about every OS anymore is based on a scatter/gather solution which doesn't process the IO in the order you supply your requests, heck it doesn't even have to return the data in front to back order, you often get bits from the end/middle before you get the start of the file. The IO is optimized to reduce physical head movement instead of sequential reading. So, in the case of fread, if you have multiple freads outstanding (obviously from different threads) you generally get better throughput since the underlying IO will fulfill the requested reads in the order the head moves over the media, not the order you requested the files. (NOTE: the IO requests will always complete and the OS's have a fairness system which will eventually cause any long standing request to complete even if it has to move the head to the other side to get it.)
Most OS's supply API's with better exposure of this underlying IO which, typically, when used means you can go back to a single threaded solution and get the gains. Win32 supplies the overlapped IO flags, multiple source/target buffers and the oh so overly complicated completion ports (and buggy up till about Vista). Most *nix variations have AIO or for Linux you have epoll which I believe is the latest/greatest version right now and BSD variations have kevent/kqueue.
There are of course variations, limitations and often goofball implementation details involved with all of these. Fortunately, with a bit of work the result is quite often a pretty massive latency reduction and much higher utilization of the physical media throughput rates. This even applies to SSD's due to how their access is performed by the controllers. If dealing with this is worth it, that's up to your game, but really high performance resource systems generally need to use at least the lowest variation supplied which is just a bit more complicated than fread.
Posted by AllEightUp on 27 November 2013 - 01:14 AM
While frob is correct, there are some missing details which are important. STL for instance is completely usable over libraries if "ALL" items link to a dynamic (and the same) form of msvcrt. All executables, dlls etc, must use "dynamic link" to msvcrt and at that point it is completely viable to use all of C++ std without issue over DLL's. The key bit of difficulty with Windows is that things *can* run in different memory spaces even from within the same process. If everything shares the same msvcrt (must be the dll version), all the memory issues go away because the msvcrt basically defines "the memory space". (For the most part, there are *other* issues of course.)
In general though, this is a fine way to do things if you intend to be cross platform. It solves all the memory issues which vary from Windows to just about any other platform. It is a fairly common practice for most 3rd party items to supply appropriate dll's for this centralized msvcrt linkage etc. The performance cost is pretty much unmeasurable so who cares.
For those interested, the details are pretty simple. Msvcrt implements a memory manager with a singleton. Yup, the evil singleton strikes and in this case it can be a nightmare. You can link to msvcrt either dynamically or statically. *if* everything uses dynamic linkage, everything gets the same singleton. If there is any mixed usage, things blow up quickly because there are multiple memory managers which don't know about each other.
Hope this clarifies things a bit from the Windows unique view.
Posted by AllEightUp on 25 November 2013 - 08:03 PM
Posted by AllEightUp on 23 November 2013 - 04:21 PM
XMPP pubsub does not solve the server-side rules enforcement problem. You need a custom module for that.
It also doesn't solve the Facebook integration problem -- in fact, it makes it harder, as you have to tie multiple identities together.
For the rules enforcement, as mentioned, just write an external XEP service using any of a number of frameworks available for many different languages. In the posted silly example, the rules are enforced by a second browser connected to the server which controls the game. Kinda inefficient to do it that way but basically the same pattern can be used without needing to actually "plug into" the server. Python has a very nice external XEP solution, it just connects with a service account so it can create/own nodes.
I didn't notice the facebook desire, but there are plenty of existing XEP's for the various servers which will do that. A little Googling suggested it would take about 10 minutes to add it, configure it and then use FB as your login. Seemed pretty simple, might try it on my server for fun.
Understand, I'm not against the suggested frameworks you mentioned at all or arguing that XMPP is some silver bullet and better in some way, but from a brief perusal at least one of the suggested frameworks shows strong hints that it is or was an XMPP serverand they just added more features and an easier API instead of the XML stanza's. For most purposes though, there are plenty of existing XEP's to support most things and existing frameworks to make writing custom ones extremely simple. It's the flexibility to turn it into whatever you want with a large existing support base which makes me suggest taking an unbiased look at it. Not to mention the proven scalability seen with Google and Facebook using it along with many MMO's (though just for presence and chat usually) and a number of other online games ranging from the given Chess to many of the online poker games.
Posted by AllEightUp on 23 November 2013 - 01:34 PM
I know of no XMPP server that bakes in the things you're talking about; you'd have to add it into some existing server by changing the server.
That's certainly possible, but at that point, you're no longer XMPP, you're some custom client and server that happen to use XMPP for transport.
You might want to look at the XMPP servers again as all of the ones I've looked at and used implement: XEP-0060 publish-subscribe nodes (http://xmpp.org/extensions/xep-0060.html) and XEP- 0124 BOSH html connections (http://xmpp.org/extensions/xep-0124.html). EJabberD for instance: http://www.ejabberd.im/protocols, as you can see not only are they both implemented they are both on by default in a standard installation. The list of servers I know have those built in XEP's (pubsub enabled by default. bosh sometimes but available): jabberd2.0 (not related to the above ejabberd), Openfire, Citadel, Prosidy, Vines and I'm quite sure others, though this is pretty much your list of the most common servers.
Given you need nothing more to do everything I mentioned, there is no particular reason to think XMPP is any more or less complex to use than any of the other solutions you mention. A "game" can be written as nothing more than a client and XEP service written in just about any language you want.
As a silly example, her's tic tac toe in XMPP played via web browser: https://github.com/metajack/profxmpp/tree/master/ch11
Obviously a downside to XMPP is that being a XML transport it is heavy weight, but that of course allows it to work over HTTP easily so perhaps it is not a downside.
Addition: forgot to list XEP-0045 for the multi-user chat room ability he requested. Again, that list of servers all have this.
Posted by AllEightUp on 23 November 2013 - 10:22 AM
XMPP has a lot of good things to sell it for this. The basic concept of using it for games is already proven, ChessPark is a pretty reasonable example of online chess which can be played very fast (near realtime speed chess) or long term days on end chess. It all runs through XMPP using ejabberd and a couple custom erlang modules and XEP's. Additionally, with the bosh standard, you can route XMPP through a standard web server and use HTML 5 + java script + strophe js as your primary platform. At the minimum, your push notifications can be a simple web page the user can connect to and see turn status in realtime without having to refresh. Of course add in jquery/jquery ui and you can probably write the entire thing with the browser and it will just work on mobile as well as desktop.
Other examples of unusual uses of XMPP to consider in terms of just how flexible it is: last I heard Google Docs collaboration ran through XMPP + Ajax. The chat and online status of Facebook is just XMPP. There are some other games like ChessPark I'm sure you can find out there.
Myself, I've used XMPP to handle presence and chat for MMO's, pretty sure Dota uses XMPP for their chat also. I've also written various status systems using XMPP, server down, starting, locked, up, etc or build success, failure, etc by simply pushing status messages into nodes on the XMPP server that a web page could subscribe to and watch. I've even used the pubsub on nodes to control distributed build processes such as send a build request to a node, an XEP or module picks that up and sends out builds to workers on various machines which report the status back eventually and even zip up artifacts and send them somewhere the user can grab the results.
Overall, it is a great tool and can be used in many ways beyond what folks think of when they see "Jabber".
Posted by AllEightUp on 30 October 2013 - 12:35 AM
In the long run, you are making a lot of assumptions which may or may not be true. There are many engines out there with fixed upfront licence costs so you need to look at a wider range of options. Optimizing graphics, specifically the static lighting, is very common to most engines so I'm not sure why you think that is a custom engine win, Many engines have licensed the Beast lighting system which bakes out incredibly detailed lighting into static scenes to cover your first reason for a custom engine. Finally, writing specific code is often just writing hacky stuff to get things done, you only pay (performance wise) for things if you "use" them in most engines and as such, you can utilize a lot of existing code and only have to write a small percentage of custom code to get to the finish line.
If you "want" to write everything custom, follow your plan. If you want to actually ship a good game, re-evaluate your options. I don't mean to be offensive but something like Unity would probably do everything you are imagining and costs a fix price for *all* features, no royalties involved. The benefit is you can use the free version to make the "game" work, then upgrade to put in all the nice graphic bits.
Just my 0.02
Posted by AllEightUp on 26 October 2013 - 01:05 AM
I tend to use both Globals and Local Variables. I use Globals for mundane task which may be used in several Functions through out the coding.
For Example : int x,y; // Loop Counters These I declare up front instead of declaring them in Each function I would use a Counter.
Pretty pretty please tell me it is April 1st..... Or some other joke day, I simply don't believe this was a serious comment. The only way I can read it is as satire.