Links to Pseudo-random Theory?

Started by
49 comments, last by taby 11 years, 11 months ago

So I can XOR two random numbers together and get a number at least as random as the less random one? And I can keep doing that as much as I want to get more and more random numbers?

No, XOR is a binary operation and has the property that a xor a = 0, so repeatedly XORing a pair of numbers together won't produce a very interesting sequence!
And same for addition, if you keep doing it then there will be correlation between successive values generated which means the sequence won't be very random either.
Example: say A = 6 and B = 9. So A + B = 15. Without knowing A and B, A + B would seem random. But if you keep adding B, you get 15, 24, 33, 42, 51, etc... which clearly is not random.[/quote]

The feature of the XOR operation I was talking about probably needs to be explained in more detail, but the idea is that if a sequence of bits is hard to predict, XORing it with numbers from other sources won't destroy that feature. Of course if another method comes up with the same number, their bits will annihilate each other, but that means that the bits were not that hard to guess.

Imagine a project needs 1MB of random bits. Each member of the team could come up with their 1MB block and then we'll XOR them together. One particular member might have used a crappy PRNG, another one might have used the first bits of a compressed Linux kernel, another one might have used /dev/urandom... If at least one of the methods is hard to predict, the XOR of all of them will be hard to predict.

About multiplication, what I had in mind is multiplying by an odd constant modulo 2^64 (if your machine supports that operation) as a method to mix low bits into higher bits. If you have access to the full 128 bits of the result of multiplying two 64-bit integers, the middle bits have a good mix of the original bits, that for instance can help you achieve good avalanche effect in a hash function. Modern 64-bit CPUs support this operation and perform it in something like 3 clock cycles, so it is a very fast way of getting bit mixing. Anyway, mixing bits can be done in a number of ways, and this is just one idea.
Advertisement
George Marsaglia posted several relatively simple generators to the old newsgroups many, many years ago. They're decent generators, and simple to implement. Alternatively, the boost library offers up good choices for high-quality generators, including lagged Fibonacci and Mersenne Twister. Knuth, of course, as you posted in your edit is always a good authority on PRNGs.

For games in general, you are mostly going to be looking for speed, as opposed to high-quality. Especially in noise applications, where you are layering many layers of noise together, you want your PRNG hash to be a fast as possible while still providing decent output. The reference implementation of Perlin noise provides a hash algorithm that makes use of a look-up table to "fold" bytes together to get the lattice hash for each point. The benefit of a look up table in this manner is relatively fast performance. As well, for a noise implementation you want to avoid any generator that requires internal state. Mersenne Twister, for example, stores internal tables and is complicated to seed, thus making it unsuitable for noise generation.

The link in my signature goes to the sourceforge page for my personal noise library. It's still (perpetually) under construction. As well, I have written quite a few journal posts regarding noise generation, seamless noise, etc... a few of which were published in Game Developer Magazine last year.
I would recommend you read Knuth for a solid background on pseudorandom number generation from a theoretical perspective. There is no source better.

For simple noise generation in a game you certainly don't need cryptographic quality PRNGs. You want something fast with broad spectral characteristics and possibly a long sequence. A lagged Fibonacci generator with block rejection is probably the best time-space tradeoff. The classic linear congruential generator has very poor spectral characteristics and makes a poor choice for noise. The Mersenne Twister has excellent spectral characteristics at the expense of greatly increased computation and memory footprint.

If you're using C++, the standard library (<random> in the current standard, or <tr1/random> for ancient compilers) provides a really excellent selection of pseudorandom number generators and useful distribution adapters [disclaimer: I wrote an implementation that comes with a popular compiler]. Easy to use, well documented, and well tested.

Stephen M. Webb
Professional Free Software Developer

http://en.wikipedia....g_(mathematics)

Oh and this is fun, though obviously platform dependent:
http://spectrum.ieee.org/computing/hardware/behind-intels-new-randomnumber-generator/0
Alright so it's faster to generate all of the random numbers at the start and then iterate through them one by one?
I'm not familiar with Hashes at all - I'll check my book and google for more info, of course.

So I could do this with two or three tables of different lengths and XOR the results and get a lot of randomness right?
And it would still be fast?
Or should I just create one really, really big table at the start?
Alright so it's faster to generate all of the random numbers at the start and then iterate through them one by one?
I'm not familiar with Hashes at all - I'll check my book and google for more info, of course.[/quote]
I suppose it would be faster in a sense but it's not practical, imagine you need trillions of random numbers for a very long-running simulation (this is not uncommon), you aren't gonna be able to precalculate them all at the beginning. And the performance gain is probably not worth the increased memory use.


So I could do this with two or three tables of different lengths and XOR the results and get a lot of randomness right?
And it would still be fast?
Or should I just create one really, really big table at the start?[/quote]
No you don't need to do this. You just call the PRNG each time you need a pseudorandom number, it will give you one quickly. Remember that while memory probably abounds on your laptops and desktops, embedded devices also need some form of pseudorandom number generator, and it probably can't afford kilobytes of memory. Pseudorandom theory has improved enough to let us generate a virtual infinity of very high quality pseudorandom numbers very efficiently on a computer in O(1) memory, we should use this to our advantage.

The basic idea is to start with an initial "seed" and mix/diffuse the state's bits around in a very unpredictable, nonrepeating (hopefully) way, and use that as the next random number. Then you start again with the new state, ad infinitum (or until the sequence starts repeating, anyway). There are other methods but this is pretty much the gold standard.

A fun exercise is to create your own PRNG with a small state (say, 32 bits) to facilitate analysis, and then run various tests on it (e.g. what it its period, etc...). You might be surprised! Then, modify some workings of the algorithm used and see how this modifies the results. It's quite fascinating.

EDIT: you can combine the outputs of two generators by XOR if you want but remember this will only result in a sequence that's as pseudorandom as the most pseudorandom of the two, and your speed is also halved, so in general it's best to just use a single, good pseudorandom number generator. There are exceptions, particularly in hardware (combining multiple shift registers, which are very fast, for instance), but these are special cases.

“If I understand the standard right it is legal and safe to do this but the resulting value could be anything.”

Yeah I thought that using an algorithm was right but it seemed like everyone was telling me I should use tables.

And thank you for reminding me that XOR requires two decent random numbers so it's twice the effort.
Would it typically be a bad idea to XOR x{n} with x{n-1} or x{n-2}? Then I won't need to generate more numbers but since they have something in common that's probably bad right?


So basically right now I'm only working with 4 bits(!) at a time and I'm trying to stay away from multiplication and modulus.

Looks like I don't have the math skills to grasp much of the theory, though. That's a shame.
Would it typically be a bad idea to XOR x{n} with x{n-1} or x{n-2}? Then I won't need to generate more numbers but since they have something in common that's probably bad right?[/quote]
No because then you'd have the sequence:
x{n}
x{n - 1}
x{n - 2}
x{n - 3}
...

And:
x{n} xor x{n - 1}
x{n} xor x{n - 2}
x{n} xor x{n - 3}
...

These two sequences are clearly correlated because if you XOR one with the other, you always obtain the same number: x{n}. Such a characteristic means that the numbers you thought were pseudorandom actually exhibit nonrandom behaviour, which makes them unsuitable for use (they don't satisfy all properties of random numbers so using them in a simulation or something would completely flaw it, as the correctness of the simulation relies on the numbers being random).

Here is a simple example of a very simple pseudorandom number generator (an algorithm). It takes as an input two bytes A and B and outputs two bytes A2 and B2 (and you can iterate it to generate a whole sequence of numbers):
A2 = A + B
B2 = (B <<< 3) xor A

Where <<< represents bit rotation (here, rotation three bits to the left). Try and feed it with two arbitrary numbers, say 44 and 171, and see what the output is. Then keep repeating it on the previous output and see when it eventually settles into a cycle (it will, but how big will the cycle be?). Does it produce the same output for two distinct inputs? Now what if you work on 16-bit words instead of bytes, or what if you change the rotation value to something else? Etc...

Looks like I don't have the math skills to grasp much of the theory, though. That's a shame.[/quote]
Yeah the math around pseudorandom numbers is pretty tough but much of it can be understood intuitively, and it's not like you need to know the theory to enjoy playing around with algorithms :)

“If I understand the standard right it is legal and safe to do this but the resulting value could be anything.”


Can anyone link me to some pseudo-random papers or websites?


http://people.seas.h...cnotes/list.htm

Harvard graduate course on the subject, taught by a expert, with free book/lecture notes! Kabam!
Okay I'm getting along with my RNG! It's kind of bloated so I'm shaving it down.
I know some of this stuff might be overkill but I'm going to be generating a fair amount of random numbers.
Besides, I'm only really doing this to learn and have fun. But I am using it in my work later on, if it's any good lol.

I want my RNG to be nearly as fast as a congruential generator like:
{ R = ((aR + b) mod c) }.
It doesn't have to be as lightweight or as good, of course.

1) How fast, relatively is addition and subtraction compared to and/or/xor?
How fast, relatively is multiplication, division, modulus compared to addition and subtraction?

1.5) Could someone ball park a number of bit operations that would be acceptable to meet my goals for speed?
I'm building the RNG entirely out of and, or, xor, not, <<, >>... so I need to know a good maximum number of these I'm allowed to have.

2) Are two bitshifts as fast as one of equal size?
Like is { a << 5 } faster than { a << 2; a << 3; } or is it done one at a time and thus equally fast?

Thoughts?

This topic is closed to new replies.

Advertisement