On my server side I plan on loading as much resources into ram that will fit of course with an order of priority filling all the available ram. I want certain resources to be available as fast as possible trying to get <= 1ms round trip over gigabit lan. Im working on some hard numbers but the server im using has an hdd not a ssd so resources will be delayed by 7-15ms im working on storing everything in ram now and paging is something that comes to mind that will get in the way.

Does paging affect your server resources?

Is there a way to prevent paging?

I had a thought of reading the variables every so often tricking the computer they are in use not sure if that will work or might even be optimized out if its not something complicated so is this route feasible?

Is there a generic way rather than os specific I know different oses are going to handle thing differently but is there something that works most of the time at least?

If I have to go os specific I found MAP_LOCKED for linux probably mac but I havent found a way to do this in windows which is going to be the main target unfortunately :)

Also getting picky I dont want to prevent paging on the whole process just certain variables

Thanks

@Ashaman73 Would that not have the same paging issues? and secondly that would add to the setup currently it can just be dropped in an ran on windows and linux right from a folder no setup involved id like to keep it very simple there probably are databases I can link but still dont know how they handle paging.

@Hodgman The servers should only be used for the one process im not going to have control over all of them in the long run but hopefully if they will run a setup like this they know and ill make sure to say that it should be dedicated for the one process.

@Promit That would be fine and dandy but a general purpose os pages out alot of memory at least from what I have seen so why would I just accept that and give out a less valuable product the faster resources can be accessed the better user experience and the more one server can handle.

Pinning looks like it would deal with the pages directly thats something I would like to avoid is there no MAP_LOCKED equivalent for windows?

The Goal: have hundreds maybe even 1000+ computers running the game streaming in resources trying to stay less than 64GB lots of little resources that will be loaded on the fly from a central server this way the server just has to boot and send everything instead of downloading it to every machine from my experiments so far that seems to be working well and I have done 12 clients at a given time pretty well but im feeling like the io is the bottleneck grabbing all the random little files (although lots get cached theres still lots of misses) thats why im taking on putting everything into ram and I know the importance of profiling im working on that as we speak!

Thoughts:

There are some thoughts on making the buffer on the client side bigger which will probably happen but still doesnt mean I dont want everything sent out as fast as possible.

Another issue with the clients is that the loading is synchronous so that will have to change but if its asynchronous that means a lot more little requests at once so still much better to have everything in ram.

The data structures are pretty poor in fact there arnt any so requests are completely random access this could probably be handled a little better to a degree but being lots of clients theres still going to be a huge degree of random access.

That would be fine and dandy but a general purpose os pages out alot of memory at least from what I have seen so why would I just accept that and give out a less valuable product the faster resources can be accessed the better user experience and the more one server can handle.

This is a wrong assumption.

An operating system will normally not page out anything as long as there remains unused memory. As you allocate memory and read from files (or map them), more and more memory is getting committed, and physical memory being used, and eventually no unused physical memory is left but you are still asking to commit some more.

At that point, the OS has 3 possible choices:

1. Kill your process

2. Swap out something that is backed by swap

3. Drop pages that are backed by a mapping (this includes pages in the file cache, which is the same thing on most systems)

If you interfere with this by locking pages, you limit the OS in what it can do, and the end result almost certainly gets only worse. Usually, a kind of least recently used strategy is applied and the stuff that you are actively using is swapped out rarely or never (depends on how big your data set is, if it doesn't fit in RAM, then it must obviously be swapped). What's swapped out is usually stuff that you've looked at once at startup and probably won't look at again anyway (or some drivers or daemons that are loaded by default but that don't actually do anything in your present configuration).

It's for the same reason that you do not want to configure a system without swap partition. Two decade ago, you would usually configure 2x or 3x as much swap as you had RAM, but nowadays the amount of RAM that computers (especially servers) have is so large that this is unreasonable, and almost not possible any more (who wants to have 256GB of swap, heh), and it seems like you don't need swap at all anyway. So it would seem like a good idea to turn off swap alltogether, right?

Although it is perfectly possible to do without swap at all, you are limiting the OS in what memory management decisions it can make. You are forcing it to throw away valuable page cache when it could as well swap out stuff about which it's pretty sure that it will not be used any time soon (or ever).

Leaving the management to the OS is a good approach, and interfering with this is unwise. The only thing that is "allowable" is hinting the OS about what you are going to access in the near future (madvise or fadvise) so it can prefetch in time. Obviously, wrong hints will, again, cause more damage than being helpful, so your hints should be correct.

@samoth I have never had an os not start paging something immediately and the more ram I use the more gets paged

Doing nothing at all on fedora linux I have 200MB in swap with nothing running at all out of the box installation

Windows 8.1 doing nothing fresh startup has 960MB memory cached not sure if that means to the disk or what that is and 1125MB in use

Basically everything over the half mark on fedora gets swapped out when running some tasks but sometimes its less not sure what the reasoning is for that im thinking it has something to do with my original though of keeping on accessing the variables periodically might keep them in ram

These are servers setup to do one task I dont care if when the ram is all used the process gets killed or swapped out if theres not enough ram or anything that could happen

It would ideally look to see how much ram is available to use and use say 80% of it to have a little working overhead on a 64GB system thats 12.8GB free thats more than enough to do whatever else

I dont know how to tell exactly what windows is doing how much is being swapped to the disk what number should I look for to see whats going on there?

Im going to spend today testing as much as I can on fedora mainly

The first test of allocating ram just to see if it gets swapped to disk

Its holding data in ram using 82.6 percent thats a pretty close allocation of the 80% I was shooting for

6.4GiB of 7.8GiB

nothing else is running swap is at 258.4MiB that seems acceptable

Now the question is if I leave it alone for a while will it decide to swap out?