Jump to content

  • Log In with Google      Sign In   
  • Create Account

Interested in a FREE copy of HTML5 game maker Construct 2?

We'll be giving away three Personal Edition licences in next Tuesday's GDNet Direct email newsletter!

Sign up from the right-hand sidebar on our homepage and read Tuesday's newsletter for details!


We're also offering banner ads on our site from just $5! 1. Details HERE. 2. GDNet+ Subscriptions HERE. 3. Ad upload HERE.


Prevent Paging


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
19 replies to this topic

#1 jeff8j   Members   -  Reputation: 767

Like
0Likes
Like

Posted 20 July 2014 - 10:44 PM

On my server side I plan on loading as much resources into ram that will fit of course with an order of priority filling all the available ram. I want certain resources to be available as fast as possible trying to get <= 1ms round trip over gigabit lan. Im working on some hard numbers but the server im using has an hdd not a ssd so resources will be delayed by 7-15ms im working on storing everything in ram now and paging is something that comes to mind that will get in the way.

 

Does paging affect your server resources?

Is there a way to prevent paging?

 

I had a thought of reading the variables every so often tricking the computer they are in use not sure if that will work or might even be optimized out if its not something complicated so is this route feasible?

 

Is there a generic way rather than os specific I know different oses are going to handle thing differently but is there something that works most of the time at least?

 

If I have to go os specific I found MAP_LOCKED for linux probably mac but I havent found a way to do this in windows which is going to be the main target unfortunately :)

 

Also getting picky I dont want to prevent paging on the whole process just certain variables

 

Thanks

 

 

 


Firefox youtube video and audio downloader MP3 MP4 OGG WEBM

https://addons.mozilla.org/en-US/firefox/addon/simple-youtube-converter/


Sponsor:

#2 Ashaman73   Crossbones+   -  Reputation: 7793

Like
1Likes
Like

Posted 20 July 2014 - 11:01 PM


On my server side I plan on loading as much resources into ram that will fit of course with an order of priority filling all the available ram. I want certain resources to be available as fast as possible trying to get <= 1ms round trip over gigabit lan.

My first thought: why are you not using a performance oriented database (paired up with hi-performance hardware) which meets your data-access requirements ?



#3 Hodgman   Moderators   -  Reputation: 30926

Like
2Likes
Like

Posted 20 July 2014 - 11:04 PM

Is there a way to prevent paging?
If I have to go os specific I found MAP_LOCKED for linux probably mac but I havent found a way to do this in windows which is going to be the main target unfortunately.

Yeah, you can pin (this is what the LOCKED thing does) the pages that you don't want to be swapped out of RAM... but it's very harmful to overall system performance to pin too much memory, so this feature is generally only used in moderation by device drviers. I've not done it on Windows either, but it will be somewhere in here.
 
The alternative is simply to not allocate too much RAM tongue.png


Edited by Hodgman, 20 July 2014 - 11:08 PM.


#4 Promit   Moderators   -  Reputation: 7335

Like
9Likes
Like

Posted 20 July 2014 - 11:04 PM

Run a 64 bit process, memory map everything in your game using the default settings (ie don't lock pages), and then let the OS do its job. Interference with the underlying memory management systems will only make things worse. Memory mapping frees you from the problem of worrying about what's allocated in the first place.

 

If things start paging, buy a bigger server.

 

And against my better judgement, I am going to point out the Windows function VirtualLock.


Edited by Promit, 20 July 2014 - 11:07 PM.


#5 jeff8j   Members   -  Reputation: 767

Like
0Likes
Like

Posted 20 July 2014 - 11:22 PM

@Ashaman73 Would that not have the same paging issues? and secondly that would add to the setup currently it can just be dropped in an ran on windows and linux right from a folder no setup involved id like to keep it very simple there probably are databases I can link but still dont know how they handle paging.

 

@Hodgman The servers should only be used for the one process im not going to have control over all of them in the long run but hopefully if they will run a setup like this they know and ill make sure to say that it should be dedicated for the one process.

 

@Promit That would be fine and dandy but a general purpose os pages out alot of memory at least from what I have seen so why would I just accept that and give out a less valuable product the faster resources can be accessed the better user experience and the more one server can handle.

 

Pinning looks like it would deal with the pages directly thats something I would like to avoid is there no MAP_LOCKED equivalent for windows?

 

The Goal: have hundreds maybe even 1000+ computers running the game streaming in resources trying to stay less than 64GB lots of little resources that will be loaded on the fly from a central server this way the server just has to boot and send everything instead of downloading it to every machine from my experiments so far that seems to be working well and I have done 12 clients at a given time pretty well but im feeling like the io is the bottleneck grabbing all the random little files (although lots get cached theres still lots of misses) thats why im taking on putting everything into ram and I know the importance of profiling im working on that as we speak!

 

Thoughts:

There are some thoughts on making the buffer on the client side bigger which will probably happen but still doesnt mean I dont want everything sent out as fast as possible.

Another issue with the clients is that the loading is synchronous so that will have to change but if its asynchronous that means a lot more little requests at once so still much better to have everything in ram.

The data structures are pretty poor in fact there arnt any so requests are completely random access this could probably be handled a little better to a degree but being lots of clients theres still going to be a huge degree of random access.


Firefox youtube video and audio downloader MP3 MP4 OGG WEBM

https://addons.mozilla.org/en-US/firefox/addon/simple-youtube-converter/


#6 Ashaman73   Crossbones+   -  Reputation: 7793

Like
0Likes
Like

Posted 21 July 2014 - 03:51 AM


@Ashaman73 Would that not have the same paging issues? and secondly that would add to the setup currently it can just be dropped in an ran on windows and linux right from a folder no setup involved id like to keep it very simple there probably are databases I can link but still dont know how they handle paging.

Well, what is more important, to get rid of paging or to get a hi-performance system running (what is your goal?). I wanted to say, that other tools are already specialist in certain areas and most likely have other mechanism which works great and completly avoid/ignore paging issues or had the same issues, but solved them in a good way.

 

Low level development always bears the danger of lot of fun (think of Dwarf Fortress) in the long run, what seems to run perfectly on one machine, can suddenly collapse on an other, similar machine.



#7 samoth   Crossbones+   -  Reputation: 4912

Like
1Likes
Like

Posted 21 July 2014 - 06:01 AM

That would be fine and dandy but a general purpose os pages out alot of memory at least from what I have seen so why would I just accept that and give out a less valuable product the faster resources can be accessed the better user experience and the more one server can handle.

This is a wrong assumption.

 

An operating system will normally not page out anything as long as there remains unused memory. As you allocate memory and read from files (or map them), more and more memory is getting committed, and physical memory being used, and eventually no unused physical memory is left but you are still asking to commit some more.

 

At that point, the OS has 3 possible choices:

1. Kill your process

2. Swap out something that is backed by swap

3. Drop pages that are backed by a mapping (this includes pages in the file cache, which is the same thing on most systems)

 

If you interfere with this by locking pages, you limit the OS in what it can do, and the end result almost certainly gets only worse. Usually, a kind of least recently used strategy is applied and the stuff that you are actively using is swapped out rarely or never (depends on how big your data set is, if it doesn't fit in RAM, then it must obviously be swapped). What's swapped out is usually stuff that you've looked at once at startup and probably won't look at again anyway (or some drivers or daemons that are loaded by default but that don't actually do anything in your present configuration).

 

It's for the same reason that you do not want to configure a system without swap partition. Two decade ago, you would usually configure 2x or 3x as much swap as you had RAM, but nowadays the amount of RAM that computers (especially servers) have is so large that this is unreasonable, and almost not possible any more (who wants to have 256GB of swap, heh), and it seems like you don't need swap at all anyway. So it would seem like a good idea to turn off swap alltogether, right?

Although it is perfectly possible to do without swap at all, you are limiting the OS in what memory management decisions it can make. You are forcing it to throw away valuable page cache when it could as well swap out stuff about which it's pretty sure that it will not be used any time soon (or ever).

 

Leaving the management to the OS is a good approach, and interfering with this is unwise. The only thing that is "allowable" is hinting the OS about what you are going to access in the near future (madvise or fadvise) so it can prefetch in time. Obviously, wrong hints will, again, cause more damage than being helpful, so your hints should be correct.


Edited by samoth, 21 July 2014 - 06:03 AM.


#8 jeff8j   Members   -  Reputation: 767

Like
0Likes
Like

Posted 21 July 2014 - 06:42 AM

@samoth I have never had an os not start paging something immediately and the more ram I use the more gets paged

Doing nothing at all on fedora linux I have 200MB in swap with nothing running at all out of the box installation

Windows 8.1 doing nothing fresh startup has 960MB memory cached not sure if that means to the disk or what that is and 1125MB in use

 

Basically everything over the half mark on fedora gets swapped out when running some tasks but sometimes its less not sure what the reasoning is for that im thinking it has something to do with my original though of keeping on accessing the variables periodically might keep them in ram

 

These are servers setup to do one task I dont care if when the ram is all used the process gets killed or swapped out if theres not enough ram or anything that could happen

It would ideally look to see how much ram is available to use and use say 80% of it to have a little working overhead on a 64GB system thats 12.8GB free thats more than enough to do whatever else

 

I dont know how to tell exactly what windows is doing how much is being swapped to the disk what number should I look for to see whats going on there?


Firefox youtube video and audio downloader MP3 MP4 OGG WEBM

https://addons.mozilla.org/en-US/firefox/addon/simple-youtube-converter/


#9 jeff8j   Members   -  Reputation: 767

Like
0Likes
Like

Posted 21 July 2014 - 06:59 AM

Im going to spend today testing as much as I can on fedora mainly

 

The first test of allocating ram just to see if it gets swapped to disk

Its holding data in ram using 82.6 percent thats a pretty close allocation of the 80% I was shooting for

6.4GiB of 7.8GiB

nothing else is running swap is at 258.4MiB that seems acceptable

Now the question is if I leave it alone for a while will it decide to swap out?


Firefox youtube video and audio downloader MP3 MP4 OGG WEBM

https://addons.mozilla.org/en-US/firefox/addon/simple-youtube-converter/


#10 slicer4ever   Crossbones+   -  Reputation: 3943

Like
1Likes
Like

Posted 21 July 2014 - 07:03 AM

Why are you seeking a programming solution to this problem, and not just configuring your servers to not using swaping/virtual memory?

Edited by slicer4ever, 21 July 2014 - 07:05 AM.

Check out https://www.facebook.com/LiquidGames for some great games made by me on the Playstation Mobile market.

#11 jeff8j   Members   -  Reputation: 767

Like
0Likes
Like

Posted 21 July 2014 - 07:08 AM

@slicer4ever If I can do it programmatically then others who set it up wouldn't have to do anything special I know on linux you can change what it prefers to do with the swap you can tell it to try to never swap which is ideal for this situation but I have found the solution with MAP_LOCKED which so far seems like it works wonders at this point im just trying to achieve the same thing in windows but apparently it doesnt give much for options and thats getting frustrating


Firefox youtube video and audio downloader MP3 MP4 OGG WEBM

https://addons.mozilla.org/en-US/firefox/addon/simple-youtube-converter/


#12 Promit   Moderators   -  Reputation: 7335

Like
0Likes
Like

Posted 21 July 2014 - 07:40 AM

@slicer4ever If I can do it programmatically then others who set it up wouldn't have to do anything special I know on linux you can change what it prefers to do with the swap you can tell it to try to never swap which is ideal for this situation but I have found the solution with MAP_LOCKED which so far seems like it works wonders at this point im just trying to achieve the same thing in windows but apparently it doesnt give much for options and thats getting frustrating

Windows has all the same options, specifically VirtualLock. The problem is what you're doing is boneheaded and exposes a very serious lack of understanding about what the OS is doing, why it's doing it, or how to track what it's doing. Paging is not casual and virtual memory control functions are not toys. Your methods of checking the way the OS is handling memory are not accurate and are not telling you the whole story. In addition, your talk of "accessing variables periodically" suggests to me that you don't know how any of this stuff works in the first place.

 

If you really must do this, VirtualLock is more or less equivalent to mlock which is equivalent to mmap with MAP_LOCKED. But what you should do step back and take the time to understand how an OS works in the first place. You should also be using performance analysis tools that can tell you when, where, and why page faults are occurring, if it's even a problem.


Edited by Promit, 21 July 2014 - 07:44 AM.


#13 samoth   Crossbones+   -  Reputation: 4912

Like
2Likes
Like

Posted 21 July 2014 - 08:06 AM

Doing nothing at all on fedora linux I have 200MB in swap with nothing running at all out of the box installation

Yes OK, and so what? This does not hurt. These are 200MB that you would not otherwise have available for your application. It's supposed to be that way.
 

Windows 8.1 doing nothing fresh startup has 960MB memory cached not sure if that means to the disk or what that is and 1125MB in use

Windows 8.1 is a tough one to discuss because Windows 8 (and 8.1) does a lot of obscure stuff, such as letting programs run on in a "kind of frozen" state when it told you that the program has terminated. That's presumably for shorter startup times or such, but it sure makes looking at numbers and judging them even more complicated.

 

When it says "cached", however, this means file buffer cache. There exist "optimizer software" written by unwitting programmers (or unscrupulous ones, that's also possible) who are sold to unwitting users to "optimize" their memory by freeing this memory. The idea sounds good (if you don't know how VM works), but it's all snake oil.

Freeing memory means no more and no less than the next time data which used to be in the cache must be reloaded from disk, which has millisecond latency and consumes extra power (very noticeable on a portable device). On the other hand, the RAM is plugged into the computer (and powered) anyway, whether you use it or not. There is absolutely no advantage in not using it.

 

The virtual memory manager makes sure that it kicks off stuff from the cache (or the ready list) first, rather than pages that are hot in your working set. Having 900+MB cached is absolutely not a bad thing, on the contrary.

 

VirtualLock. The problem is what you're doing is boneheaded and exposes a very serious lack of understanding about what the OS is doing

Windows even tries hard to prevent you from doing this kind of thing because it is so unwise to do it. You are only allowed to lock up to your "maximum working set size", which is 16MB without changing this tuneable parameter first (and that needs administrative privilegues). So even if you do lock your memory (naively), it won't have any effect since the call to VirtualLock will simply fail (and since nobody checks error codes, you'll probably not even notice).

 

Now you are going to say "Wait, what, maximum working set size???". Yes, it is exactly as you think, the actual set of pages that is -- formally -- resident is very small. Pages are moved in and out of your working set all the time, and the vast majority of your pages is not in it at any time. But that does not mean that these pages are being written to disk all the time. It means that they are a possible candidate, in case no less important page can be found to be purged when you ask for more commit.

 

It is a logical management thing, which allows the operating system to keep the computer operational and reactive, regardless of what applications you launch or what memory they consume. It is not "swapping to disk" all the time.

 

This is very similar to high/low priority threads. A low priority thread doesn't "not fully utilize" the CPU (people sometimes believe that!). A lower priority thread runs, well, with lower priority, compared to a higher priority one. But it still uses 100% of what is available, and you do not get better CPU usage by assigning a higher priority.


Edited by samoth, 21 July 2014 - 08:10 AM.


#14 jeff8j   Members   -  Reputation: 767

Like
0Likes
Like

Posted 21 July 2014 - 09:20 AM

This is a very specific circumstance the server will only be used for this one thing I can see it as a bad idea on something distributed to the public but in this circumstance it makes a huge difference in the amount of requests the server can do per second so anything that touches the hard drives drastically reduces that.

 

@Promit Thanks for the VirtualLock pointer that looks like it could be the windows solution!

 

@Samoth My initial testing with linux proves that something is using more swap over time it can start at 200MiB and over the course of an hour keeping 80% of the ram utilized it goes up to 500MiB which if its the server variables then theres going to be a slow down when one of the clients needs to access that specific part im still experimenting to see whats going into swap and if its even part of the server data or not im also experimenting to see if locking the memory actually helps by getting some hard numbers.

 

Currently there are

almost 600,000 files that could be requested at any given time

almost 3GB location data

and processing is done on the server

the hdd is used basically just for boot and saving currently saving only happens on server shutdown

each client at a huge zoom out to control lots of units can have thousands of requests a second (lots are batched to prevent network issues but still need thousand of resources from the server)

 

So keeping everything in ram is a high priority and I dont see a way around that without distributing the load but I dont have the money for that and that increases costs for others and I dont have a processing bottleneck and I have enough ram to hold everything so is it too much to ask to make sure it holds everything?

 

Also im not looking to control it 100% but looking for my best possible option if I can tell the os to prioritize this data thats all thats needed like on processes that have different priorities does making a process lower priority affect ram?


Edited by jeff8j, 21 July 2014 - 09:23 AM.

Firefox youtube video and audio downloader MP3 MP4 OGG WEBM

https://addons.mozilla.org/en-US/firefox/addon/simple-youtube-converter/


#15 Promit   Moderators   -  Reputation: 7335

Like
0Likes
Like

Posted 21 July 2014 - 09:26 AM


@Samoth My initial testing with linux proves that something is using more swap over time it can start at 200MiB and over the course of an hour keeping 80% of the ram utilized it goes up to 500MiB which if its the server variables then theres going to be a slow down when one of the clients needs to access that specific part im still experimenting to see whats going into swap and if its even part of the server data or not im also experimenting to see if locking the memory actually helps by getting some hard numbers.

You know that memory can be copied to swap without being evicted from working memory, right? So the swap usage can go up even while the entirety of server memory is resident. This is done to allow pages to be dropped quickly on demand. Most of that swap usage you're seeing is most likely a copy of your resident pages, not stuff that has actually been paged out.


Edited by Promit, 21 July 2014 - 09:27 AM.


#16 jeff8j   Members   -  Reputation: 767

Like
0Likes
Like

Posted 21 July 2014 - 09:33 AM

I did not know that. That does sound reasonable if thats whats happening then thats a good thing! Thanks that makes me feel a little better about some of the early numbers.

 

I have multiple tests im trying now to see if using MAP_LOCKED and VirtualLock actually make a difference to the numbers im also gathering numbers for just using ram for everything anyways I was building it while asking the question because the thought of swapping did arise so I have those tests im measuring response times and amount of completed requests per second so that should help clear up whats happening if anything with paging/swap


Firefox youtube video and audio downloader MP3 MP4 OGG WEBM

https://addons.mozilla.org/en-US/firefox/addon/simple-youtube-converter/


#17 samoth   Crossbones+   -  Reputation: 4912

Like
1Likes
Like

Posted 21 July 2014 - 10:18 AM

almost 600,000 files that could be requested at any given time
That is what the buffer cache handles in a near-optimal way. Every file that is accessed reasonably often will be served from RAM.

 

If you can predict a few milliseconds ahead of time which file you'll be accessing (e.g. if throughput matters but requests need not be processed in realtime, this can be done by stalling a request for that long) you can fadvise (if you read the files) or madvise (if you map the files), allowing the OS to asynchronously prefetch the data in case it has really been swapped out. Advising has much less of a negative impact compared to locking.

 

If the dataset (files plus location data) can fit into RAM, this is entirely unnecessary. It will just magically work.
 

almost 3GB location data

Should fit in RAM on pretty much every server without even thinking about it. If it doesn't, try to normalize your data. 3GB for location data (as in, two coordinates, plus a name tag or something?) sounds like an awful lot.

 

each client [...] several thousands of requests a second

Ugh! Bandwidth! This is unaffordable.

 

Say that you have 100 clients doing 5,000 requests per second each. That's 500,000 requests per second. Let's say "request" doesn't mean fancy stuff with long strings and lots of data and options, but a mere "coordinate pair plus a 4-byte command code", so you'll have a request size of, say, 12 bytes.

 

This is 15.5TB of payload per month, which, assuming IPv6 TCP over Ethernet (there's practically no way around IPv6 in a datacenter nowadays) with maximum MTU and a 5% packet loss, that's 17.4TB of traffic (not counting TCP ACKs).

 

17.4TB and you haven't even sent a confirmation for a single request, nor a response or something.



#18 jeff8j   Members   -  Reputation: 767

Like
0Likes
Like

Posted 21 July 2014 - 10:30 AM


That is what the buffer cache handles in a near-optimal way. Every file that is accessed reasonably often will be served from RAM.

Its the files that arnt ever accessed the all the sudden lots of them do need to be accessed is the concern but thats why in the begining I was thinking I could just keep accessing them and doing something with them to keep everything in ram but im not sure what would be something that wouldnt have much overhead and not be optimized away im sure if I just read them and dont do anything with them it will probably be optimized out not sure havent tried yet

 


If you can predict a few milliseconds ahead of time which file you'll be accessing (e.g. if throughput matters but requests need not be processed in realtime, this can be done by stalling a request for that long) you can fadvise (if you read the files) or madvise (if you map the files), allowing the OS to asynchronously prefetch the data in case it has really been swapped out. Advising has much less of a negative impact compared to locking.

I probably could predict alot of what the client side would require thats a very good idea I didnt think of that but now to keep the processing low for the 8 cores

 


If the dataset (files plus location data) can fit into RAM, this is entirely unnecessary. It will just magically work.

lol I hope so thats what im putting together some profiling stats just taking me a bit to design some realistic player like tests

 


Ugh! Bandwidth! This is unaffordable.

Its over lan and im thinking the lan could be sectioned off if needed to run multiple gigabit connections

The actual packets on the line is less than 5000 because the client will batch requests and the server will batch responses lets say that the request asks for 1000 files which is about whats happening right now if each of those touch the hard drive thats lets say 10ms each thats 10 seconds then say we have lots of clients (10+) thats totaly unacceptable (of course im working on multiple areas of this problem some of which mentioned above)


Firefox youtube video and audio downloader MP3 MP4 OGG WEBM

https://addons.mozilla.org/en-US/firefox/addon/simple-youtube-converter/


#19 SeanMiddleditch   Members   -  Reputation: 6351

Like
0Likes
Like

Posted 21 July 2014 - 11:23 AM

Does paging affect your server resources?
Is there a way to prevent paging?


These are two very different questions. Trying to answer the second without concretely answering the first is a complete and utter waste of time.

#20 jeff8j   Members   -  Reputation: 767

Like
0Likes
Like

Posted 21 July 2014 - 11:40 AM


These are two very different questions. Trying to answer the second without concretely answering the first is a complete and utter waste of time.

True but now I have 3 different ways to profile so by asking "Is there a way to prevent paging?" I have a way to truly test the first question of "Does paging the resources"

 

The problems still arnt solved and the paging issue is probably more case specific than anything but I have more knowledge than I did before and a path to determine where to go from here and this isnt the only problem either so theres alot more to improve upon im happy with all the knowledge I have received thank you all!


Firefox youtube video and audio downloader MP3 MP4 OGG WEBM

https://addons.mozilla.org/en-US/firefox/addon/simple-youtube-converter/





Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS