@slicer4ever If I can do it programmatically then others who set it up wouldn't have to do anything special I know on linux you can change what it prefers to do with the swap you can tell it to try to never swap which is ideal for this situation but I have found the solution with MAP_LOCKED which so far seems like it works wonders at this point im just trying to achieve the same thing in windows but apparently it doesnt give much for options and thats getting frustrating

Doing nothing at all on fedora linux I have 200MB in swap with nothing running at all out of the box installation

Yes OK, and so what? This does not hurt. These are 200MB that you would not otherwise have available for your application. It's supposed to be that way.

Windows 8.1 doing nothing fresh startup has 960MB memory cached not sure if that means to the disk or what that is and 1125MB in use

Windows 8.1 is a tough one to discuss because Windows 8 (and 8.1) does a lot of obscure stuff, such as letting programs run on in a "kind of frozen" state when it told you that the program has terminated. That's presumably for shorter startup times or such, but it sure makes looking at numbers and judging them even more complicated.

When it says "cached", however, this means file buffer cache. There exist "optimizer software" written by unwitting programmers (or unscrupulous ones, that's also possible) who are sold to unwitting users to "optimize" their memory by freeing this memory. The idea sounds good (if you don't know how VM works), but it's all snake oil.

Freeing memory means no more and no less than the next time data which used to be in the cache must be reloaded from disk, which has millisecond latency and consumes extra power (very noticeable on a portable device). On the other hand, the RAM is plugged into the computer (and powered) anyway, whether you use it or not. There is absolutely no advantage in not using it.

The virtual memory manager makes sure that it kicks off stuff from the cache (or the ready list) first, rather than pages that are hot in your working set. Having 900+MB cached is absolutely not a bad thing, on the contrary.

VirtualLock. The problem is what you're doing is boneheaded and exposes a very serious lack of understanding about what the OS is doing

Windows even tries hard to prevent you from doing this kind of thing because it is so unwise to do it. You are only allowed to lock up to your "maximum working set size", which is 16MB without changing this tuneable parameter first (and that needs administrative privilegues). So even if you do lock your memory (naively), it won't have any effect since the call to VirtualLock will simply fail (and since nobody checks error codes, you'll probably not even notice).

Now you are going to say "Wait, what, maximum working set size???". Yes, it is exactly as you think, the actual set of pages that is -- formally -- resident is very small. Pages are moved in and out of your working set all the time, and the vast majority of your pages is not in it at any time. But that does not mean that these pages are being written to disk all the time. It means that they are a possible candidate, in case no less important page can be found to be purged when you ask for more commit.

It is a logical management thing, which allows the operating system to keep the computer operational and reactive, regardless of what applications you launch or what memory they consume. It is not "swapping to disk" all the time.

This is very similar to high/low priority threads. A low priority thread doesn't "not fully utilize" the CPU (people sometimes believe that!). A lower priority thread runs, well, with lower priority, compared to a higher priority one. But it still uses 100% of what is available, and you do not get better CPU usage by assigning a higher priority.

This is a very specific circumstance the server will only be used for this one thing I can see it as a bad idea on something distributed to the public but in this circumstance it makes a huge difference in the amount of requests the server can do per second so anything that touches the hard drives drastically reduces that.

@Promit Thanks for the VirtualLock pointer that looks like it could be the windows solution!

@Samoth My initial testing with linux proves that something is using more swap over time it can start at 200MiB and over the course of an hour keeping 80% of the ram utilized it goes up to 500MiB which if its the server variables then theres going to be a slow down when one of the clients needs to access that specific part im still experimenting to see whats going into swap and if its even part of the server data or not im also experimenting to see if locking the memory actually helps by getting some hard numbers.

Currently there are

almost 600,000 files that could be requested at any given time

almost 3GB location data

and processing is done on the server

the hdd is used basically just for boot and saving currently saving only happens on server shutdown

each client at a huge zoom out to control lots of units can have thousands of requests a second (lots are batched to prevent network issues but still need thousand of resources from the server)

So keeping everything in ram is a high priority and I dont see a way around that without distributing the load but I dont have the money for that and that increases costs for others and I dont have a processing bottleneck and I have enough ram to hold everything so is it too much to ask to make sure it holds everything?

Also im not looking to control it 100% but looking for my best possible option if I can tell the os to prioritize this data thats all thats needed like on processes that have different priorities does making a process lower priority affect ram?

I did not know that. That does sound reasonable if thats whats happening then thats a good thing! Thanks that makes me feel a little better about some of the early numbers.

I have multiple tests im trying now to see if using MAP_LOCKED and VirtualLock actually make a difference to the numbers im also gathering numbers for just using ram for everything anyways I was building it while asking the question because the thought of swapping did arise so I have those tests im measuring response times and amount of completed requests per second so that should help clear up whats happening if anything with paging/swap

almost 600,000 files that could be requested at any given time

If you can predict a few milliseconds ahead of time which file you'll be accessing (e.g. if throughput matters but requests need not be processed in realtime, this can be done by stalling a request for that long) you can fadvise (if you read the files) or madvise (if you map the files), allowing the OS to asynchronously prefetch the data in case it has really been swapped out. Advising has much less of a negative impact compared to locking.

If the dataset (files plus location data) can fit into RAM, this is entirely unnecessary. It will just magically work.

almost 3GB location data

Should fit in RAM on pretty much every server without even thinking about it. If it doesn't, try to normalize your data. 3GB for location data (as in, two coordinates, plus a name tag or something?) sounds like an awful lot.

each client [...] several thousands of requests a second

Ugh! Bandwidth! This is unaffordable.

Say that you have 100 clients doing 5,000 requests per second each. That's 500,000 requests per second. Let's say "request" doesn't mean fancy stuff with long strings and lots of data and options, but a mere "coordinate pair plus a 4-byte command code", so you'll have a request size of, say, 12 bytes.

This is 15.5TB of payload per month, which, assuming IPv6 TCP over Ethernet (there's practically no way around IPv6 in a datacenter nowadays) with maximum MTU and a 5% packet loss, that's 17.4TB of traffic (not counting TCP ACKs).

17.4TB and you haven't even sent a confirmation for a single request, nor a response or something.

That is what the buffer cache handles in a near-optimal way. Every file that is accessed reasonably often will be served from RAM.

Its the files that arnt ever accessed the all the sudden lots of them do need to be accessed is the concern but thats why in the begining I was thinking I could just keep accessing them and doing something with them to keep everything in ram but im not sure what would be something that wouldnt have much overhead and not be optimized away im sure if I just read them and dont do anything with them it will probably be optimized out not sure havent tried yet

If you can predict a few milliseconds ahead of time which file you'll be accessing (e.g. if throughput matters but requests need not be processed in realtime, this can be done by stalling a request for that long) you can fadvise (if you read the files) or madvise (if you map the files), allowing the OS to asynchronously prefetch the data in case it has really been swapped out. Advising has much less of a negative impact compared to locking.

I probably could predict alot of what the client side would require thats a very good idea I didnt think of that but now to keep the processing low for the 8 cores

If the dataset (files plus location data) can fit into RAM, this is entirely unnecessary. It will just magically work.

lol I hope so thats what im putting together some profiling stats just taking me a bit to design some realistic player like tests

Ugh! Bandwidth! This is unaffordable.

Its over lan and im thinking the lan could be sectioned off if needed to run multiple gigabit connections

The actual packets on the line is less than 5000 because the client will batch requests and the server will batch responses lets say that the request asks for 1000 files which is about whats happening right now if each of those touch the hard drive thats lets say 10ms each thats 10 seconds then say we have lots of clients (10+) thats totaly unacceptable (of course im working on multiple areas of this problem some of which mentioned above)

These are two very different questions. Trying to answer the second without concretely answering the first is a complete and utter waste of time.

True but now I have 3 different ways to profile so by asking "Is there a way to prevent paging?" I have a way to truly test the first question of "Does paging the resources"

The problems still arnt solved and the paging issue is probably more case specific than anything but I have more knowledge than I did before and a path to determine where to go from here and this isnt the only problem either so theres alot more to improve upon im happy with all the knowledge I have received thank you all!