Misplaced Pages

Lehmer random number generator

Article snapshot taken from Wikipedia with creative commons attribution-sharealike license. Give it a read and then ask your questions in the chat. We can research this topic together.
Type of linear congruential generator with no additive constant

The Lehmer random number generator (named after D. H. Lehmer), sometimes also referred to as the Park–Miller random number generator (after Stephen K. Park and Keith W. Miller), is a type of linear congruential generator (LCG) that operates in multiplicative group of integers modulo n. The general formula is

X k + 1 = a ⋅ X k mod m , {\displaystyle X_{k+1}=a\cdot X_{k}{\bmod {m}},}

where the modulus m is a prime number or a power of a prime number, the multiplier a is an element of high multiplicative order modulo m (e.g., a primitive root modulo n), and the seed X0 is coprime to m.

Other names are multiplicative linear congruential generator (MLCG) and multiplicative congruential generator (MCG).

Parameters in common use

In 1988, Park and Miller suggested a Lehmer RNG with particular parameters m = 2 − 1 = 2,147,483,647 (a Mersenne prime M31) and a = 7 = 16,807 (a primitive root modulo M31), now known as MINSTD. Although MINSTD was later criticized by Marsaglia and Sullivan (1993), it is still in use today (in particular, in CarbonLib and C++11's minstd_rand0). Park, Miller and Stockmeyer responded to the criticism (1993), saying:

Given the dynamic nature of the area, it is difficult for nonspecialists to make decisions about what generator to use. "Give me something I can understand, implement and port... it needn't be state-of-the-art, just make sure it's reasonably good and efficient." Our article and the associated minimal standard generator was an attempt to respond to this request. Five years later, we see no need to alter our response other than to suggest the use of the multiplier a = 48271 in place of 16807.

This revised constant is used in C++11's minstd_rand random number generator.

The Sinclair ZX81 and its successors use the Lehmer RNG with parameters m = 2 + 1 = 65,537 (a Fermat prime F4) and a = 75 (a primitive root modulo F4). The CRAY random number generator RANF is a Lehmer RNG with the power-of-two modulus m = 2 and a = 44,485,709,377,909. The GNU Scientific Library includes several random number generators of the Lehmer form, including MINSTD, RANF, and the infamous IBM random number generator RANDU.

Choice of modulus

Most commonly, the modulus is chosen as a prime number, making the choice of a coprime seed trivial (any 0 < X0 < m will do). This produces the best-quality output, but introduces some implementation complexity, and the range of the output is unlikely to match the desired application; converting to the desired range requires an additional multiplication.

Using a modulus m which is a power of two makes for a particularly convenient computer implementation, but comes at a cost: the period is at most m/4, and the lower bits have periods shorter than that. This is because the lowest k bits form a modulo-2 generator all by themselves; the higher-order bits never affect lower-order bits. The values Xi are always odd (bit 0 never changes), bits 2 and 1 alternate (the lower 3 bits repeat with a period of 2), the lower 4 bits repeat with a period of 4, and so on. Therefore, the application using these random numbers must use the most significant bits; reducing to a smaller range using a modulo operation with an even modulus will produce disastrous results.

To achieve this period, the multiplier must satisfy a ≡ ±3 (mod 8), and the seed X0 must be odd.

Using a composite modulus is possible, but the generator must be seeded with a value coprime to m, or the period will be greatly reduced. For example, a modulus of F5 = 2 + 1 might seem attractive, as the outputs can be easily mapped to a 32-bit word 0 ≤ Xi − 1 < 2. However, a seed of X0 = 6700417 (which divides 2 + 1) or any multiple would lead to an output with a period of only 640.

Another generator with a composite modulus is the one recommended by Nakazawa & Nakazawa:

  • m = 134265023 × 134475827 = 18055400005099021 ≈ 2
  • a = 7759097958782935 (any of ±a (mod m) will do as well)

As both factors of the modulus are less than 2, it is possible to maintain the state modulo each of the factors, and construct the output value using the Chinese remainder theorem, using no more than 64-bit intermediate arithmetic.

A more popular implementation for large periods is a combined linear congruential generator; combining (e.g. by summing their outputs) several generators is equivalent to the output of a single generator whose modulus is the product of the component generators' moduli. and whose period is the least common multiple of the component periods. Although the periods will share a common divisor of 2, the moduli can be chosen so that is the only common divisor and the resultant period is (m1 − 1)(m2 − 1)···(mk − 1)/2. One example of this is the Wichmann–Hill generator.

Relation to LCG

While the Lehmer RNG can be viewed as a particular case of the linear congruential generator with c=0, it is a special case that implies certain restrictions and properties. In particular, for the Lehmer RNG, the initial seed X0 must be coprime to the modulus m, which is not required for LCGs in general. The choice of the modulus m and the multiplier a is also more restrictive for the Lehmer RNG. In contrast to LCG, the maximum period of the Lehmer RNG equals m − 1, and it is such when m is prime and a is a primitive root modulo m.

On the other hand, the discrete logarithms (to base a or any primitive root modulo m) of Xk in Z m {\displaystyle \mathbb {Z} _{m}} represent a linear congruential sequence modulo the Euler totient φ ( m ) {\displaystyle \varphi (m)} .

Implementation

A prime modulus requires the computation of a double-width product and an explicit reduction step. If a modulus just less than a power of 2 is used (the Mersenne primes 2 − 1 and 2 − 1 are popular, as are 2 − 5 and 2 − 59), reduction modulo m = 2 − d can be implemented more cheaply than a general double-width division using the identity 2 ≡ d (mod m).

The basic reduction step divides the product into two e-bit parts, multiplies the high part by d, and adds them: (ax mod 2) + d ⌊ax/2⌋. This can be followed by subtracting m until the result is in range. The number of subtractions is limited to ad/m, which can be easily limited to one if d is small and a < m/d is chosen. (This condition also ensures that d ⌊ax/2⌋ is a single-width product; if it is violated, a double-width product must be computed.)

When the modulus is a Mersenne prime (d = 1), the procedure is particularly simple. Not only is multiplication by d trivial, but the conditional subtraction can be replaced by an unconditional shift and addition. To see this, note that the algorithm guarantees that x ≢ 0 (mod m), meaning that x = 0 and x = m are both impossible. This avoids the need to consider equivalent e-bit representations of the state; only values where the high bits are non-zero need reduction.

The low e bits of the product ax cannot represent a value larger than m, and the high bits will never hold a value greater than a − 1 ≤ m − 2. Thus the first reduction step produces a value at most m + a − 1 ≤ 2m − 2 = 2 − 4. This is an (e + 1)-bit number, which can be greater than m (i.e. might have bit e set), but the high half is at most 1, and if it is, the low e bits will be strictly less than m. Thus whether the high bit is 1 or 0, a second reduction step (addition of the halves) will never overflow e bits, and the sum will be the desired value.

If d > 1, conditional subtraction can also be avoided, but the procedure is more intricate. The fundamental challenge of a modulus like 2 − 5 lies in ensuring that we produce only one representation for values such as 1 ≡ 2 − 4. The solution is to temporarily add d, so that the range of possible values is d through 2 − 1, and reduce values larger than e bits in a way that never generates representations less than d. Finally subtracting the temporary offset produces the desired value.

Begin by assuming that we have a partially reduced value y bounded so that 0 ≤ y < 2m = 2 − 2d. In this case, a single offset subtraction step will produce 0 ≤ y′ = ((y + d) mod 2) + d ⌊(y + d)/2⌋ − d < m. To see this, consider two cases:

0 ≤ y < m = 2 − d
In this case, y + d < 2 and y′ = y < m, as desired.
m ≤ y < 2m
In this case, 2 ≤ y + d < 2 is an (e + 1)-bit number, and ⌊(y + d)/2⌋ = 1. Thus, y′ = (y + d) − 2 + d − d = y − 2 + d = y − m < m, as desired. Because the multiplied high part is d, the sum is at least d, and subtracting the offset never causes underflow.

(For the case of a Lehmer generator specifically, a zero state or its image y = m will never occur, so an offset of d − 1 will work just the same, if that is more convenient. This reduces the offset to 0 in the Mersenne prime case, when d = 1.)

Reducing a larger product ax to less than 2m = 2 − 2d can be done by one or more reduction steps without an offset.

If ad ≤ m, then one additional reduction step suffices. Since x < m, ax < am ≤ (a − 1)2, and one reduction step converts this to at most 2 − 1 + (a − 1)d = m + ad − 1. This is within the limit of 2m if ad − 1 < m, which is the initial assumption.

If ad > m, then it is possible for the first reduction step to produce a sum greater than 2m = 2 − 2d, which is too large for the final reduction step. (It also requires the multiplication by d to produce a product larger than e bits, as mentioned above.) However, as long as d < 2, the first reduction will produce a value in the range required for the preceding case of two reduction steps to apply.

Schrage's method

If a double-width product is not available, Schrage's method, also called the approximate factoring method, may be used to compute ax mod m, but this comes at the cost:

  • The modulus must be representable in a signed integer; arithmetic operations must allow a range of ±m.
  • The choice of multiplier a is restricted. We require that m mod a ≤ ⌊m/a⌋, commonly achieved by choosing a ≤ √m.
  • One division (with remainder) per iteration is required.

While this technique is popular for portable implementations in high-level languages which lack double-width operations, on modern computers division by a constant is usually implemented using double-width multiplication, so this technique should be avoided if efficiency is a concern. Even in high-level languages, if the multiplier a is limited to √m, then the double-width product ax can be computed using two single-width multiplications, and reduced using the techniques described above.

To use Schrage's method, first factor m = qa + r, i.e. precompute the auxiliary constants r = m mod a and q = ⌊m/a⌋ = (m−r)/a. Then, each iteration, compute ax ≡ a(x mod q) − r ⌊x/q⌋ (mod m).

This equality holds because

a q = a ⋅ ( m − r ) / a = ( a / a ) ⋅ ( m − r ) = ( m − r ) ≡ − r ( mod m ) {\displaystyle {\begin{aligned}aq&=a\cdot (m-r)/a\\&=(a/a)\cdot (m-r)\\&=(m-r)\\&\equiv -r{\pmod {m}}\\\end{aligned}}}

so if we factor x = (x mod q) + q⌊x/q⌋, we get:

a x = a ( x mod q ) + a q ⌊ x / q ⌋ = a ( x mod q ) + ( m − r ) ⌊ x / q ⌋ ≡ a ( x mod q ) − r ⌊ x / q ⌋ ( mod m ) {\displaystyle {\begin{aligned}ax&=a(x{\bmod {q}})+aq\lfloor x/q\rfloor \\&=a(x{\bmod {q}})+(m-r)\lfloor x/q\rfloor \\&\equiv a(x{\bmod {q}})-r\lfloor x/q\rfloor {\pmod {m}}\\\end{aligned}}}

The reason it does not overflow is that both terms are less than m. Since x mod q < q ≤ m/a, the first term is strictly less than am/a = m and may be computed with a single-width product.

If a is chosen so that r ≤ q (and thus r/q ≤ 1), then the second term is also less than m: r ⌊x/q⌋ ≤ rx/q = x(r/q) ≤ x(1) = x < m. Thus, the difference lies in the range and can be reduced to with a single conditional add.

This technique may be extended to allow a negative r (−q ≤ r < 0), changing the final reduction to a conditional subtract.

The technique may also be extended to allow larger a by applying it recursively. Of the two terms subtracted to produce the final result, only the second (r ⌊x/q⌋) risks overflow. But this is itself a modular multiplication by a compile-time constant r, and may be implemented by the same technique. Because each step, on average, halves the size of the multiplier (0 ≤ r < a, average value (a−1)/2), this would appear to require one step per bit and be spectacularly inefficient. However, each step also divides x by an ever-increasing quotient q = ⌊m/a⌋, and quickly a point is reached where the argument is 0 and the recursion may be terminated.

Sample C99 code

Using C code, the Park-Miller RNG can be written as follows: 

uint32_t lcg_parkmiller(uint32_t *state)
{
	return *state = (uint64_t)*state * 48271 % 0x7fffffff;
}

This function can be called repeatedly to generate pseudorandom numbers, as long as the caller is careful to initialize the state to any number greater than zero and less than the modulus. In this implementation, 64-bit arithmetic is required; otherwise, the product of two 32-bit integers may overflow.

To avoid the 64-bit division, do the reduction by hand: 

uint32_t lcg_parkmiller(uint32_t *state)
{
	uint64_t product = (uint64_t)*state * 48271;
	uint32_t x = (product & 0x7fffffff) + (product >> 31);
	x = (x & 0x7fffffff) + (x >> 31);
	return *state = x;
}

To use only 32-bit arithmetic, use Schrage's method: 

uint32_t lcg_parkmiller(uint32_t *state)
{
	// Precomputed parameters for Schrage's method
	const uint32_t M = 0x7fffffff;
	const uint32_t A = 48271;
	const uint32_t Q = M / A;    // 44488
	const uint32_t R = M % A;    //  3399
	uint32_t div = *state / Q;	// max: M / Q = A = 48,271
	uint32_t rem = *state % Q;	// max: Q - 1     = 44,487
	int32_t s = rem * A;	// max: 44,487 * 48,271 = 2,147,431,977 = 0x7fff3629
	int32_t t = div * R;	// max: 48,271 *  3,399 =   164,073,129
	int32_t result = s - t;
	if (result < 0)
		result += M;
	return *state = result;
}

or use two 16×16-bit multiplies: 

uint32_t lcg_parkmiller(uint32_t *state)
{
	const uint32_t A = 48271;
	uint32_t low  = (*state & 0x7fff) * A;			// max: 32,767 * 48,271 = 1,581,695,857 = 0x5e46c371
	uint32_t high = (*state >> 15)    * A;			// max: 65,535 * 48,271 = 3,163,439,985 = 0xbc8e4371
	uint32_t x = low + ((high & 0xffff) << 15) + (high >> 16);	// max: 0x5e46c371 + 0x7fff8000 + 0xbc8e = 0xde46ffff
	x = (x & 0x7fffffff) + (x >> 31);
	return *state = x;
}

Another popular Lehmer generator uses the prime modulus 2−5: 

uint32_t lcg_rand(uint32_t *state)
{
    return *state = (uint64_t)*state * 279470273u % 0xfffffffb;
}

This can also be written without a 64-bit division: 

uint32_t lcg_rand(uint32_t *state)
{
	uint64_t product = (uint64_t)*state * 279470273u;
	uint32_t x;
	// Not required because 5 * 279470273 = 0x5349e3c5 fits in 32 bits.
	// product = (product & 0xffffffff) + 5 * (product >> 32);
	// A multiplier larger than 0x33333333 = 858,993,459 would need it.
	// The multiply result fits into 32 bits, but the sum might be 33 bits.
	product = (product & 0xffffffff) + 5 * (uint32_t)(product >> 32);
	product += 4;
	// This sum is guaranteed to be 32 bits.
	x = (uint32_t)product + 5 * (uint32_t)(product >> 32);
	return *state = x - 4;
}

Many other Lehmer generators have good properties. The following modulo-2 Lehmer generator requires 128-bit support from the compiler and uses a multiplier computed by L'Ecuyer. It has a period of 2: 

static unsigned __int128 state;
/* The state must be seeded with an odd value. */
void seed(unsigned __int128 seed)
{
	state = seed << 1 | 1;
}
uint64_t next(void)
{
	// GCC cannot write 128-bit literals, so we use an expression
	const unsigned __int128 mult =
		(unsigned __int128)0x12e15e35b500f16e << 64 |
		0x2e714eb2b37916a5;
	state *= mult;
	return state >> 64;
}

The generator computes an odd 128-bit value and returns its upper 64 bits.

This generator passes BigCrush from TestU01, but fails the TMFn test from PractRand. That test has been designed to catch exactly the defect of this type of generator: since the modulus is a power of 2, the period of the lowest bit in the output is only 2, rather than 2. Linear congruential generators with a power-of-2 modulus have a similar behavior.

The following core routine improves upon the speed of the above code for integer workloads (if the constant declaration is allowed to be optimized out of a calculation loop by the compiler): 

uint64_t next(void)
{
	uint64_t result = state >> 64;
	// GCC cannot write 128-bit literals, so we use an expression
	const unsigned __int128 mult =
		(unsigned __int128)0x12e15e35b500f16e << 64 |
		0x2e714eb2b37916a5;
	state *= mult;
	return result;
}

However, because the multiplication is deferred, it is not suitable for hashing, since the first call simply returns the upper 64 bits of the seed state.

References

  1. W. H. Payne; J. R. Rabung; T. P. Bogyo (1969). "Coding the Lehmer pseudo-random number generator" (PDF). Communications of the ACM. 12 (2): 85–86. doi:10.1145/362848.362860. S2CID 2749316.
  2. ^ L'Ecuyer, Pierre (June 1988). "Efficient and Portable Combined Random Number Generators" (PDF). Communications of the ACM. 31 (6): 742–774. doi:10.1145/62959.62969. S2CID 9593394.
  3. Park, Stephen K.; Miller, Keith W. (1988). "Random Number Generators: Good Ones Are Hard To Find" (PDF). Communications of the ACM. 31 (10): 1192–1201. doi:10.1145/63039.63042. S2CID 207575300.
  4. Marsaglia, George (1993). "Technical correspondence: Remarks on Choosing and Implementing Random Number Generators" (PDF). Communications of the ACM. 36 (7): 105–108. doi:10.1145/159544.376068. S2CID 26156905.
  5. Sullivan, Stephen (1993). "Technical correspondence: Another test for randomness" (PDF). Communications of the ACM. 36 (7): 108. doi:10.1145/159544.376068. S2CID 26156905.
  6. Park, Stephen K.; Miller, Keith W.; Stockmeyer, Paul K. (1988). "Technical Correspondence: Response" (PDF). Communications of the ACM. 36 (7): 108–110. doi:10.1145/159544.376068. S2CID 26156905.
  7. Vickers, Steve (1981). "Chapter 5. Functions". ZX81 Basic Programming (2nd ed.). Sinclair Research Ltd. Retrieved 2024-04-21. The ZX81 uses p=65537 & a=75 (Note that the ZX81 manual incorrectly states that 65537 is a Mersenne prime that equals 2 − 1. The ZX Spectrum manual fixed that and correctly states that it is a Fermat prime that equals 2 + 1.)
  8. Vickers, Steve (1983). "Chapter 11. Random numbers". Sinclair ZX Spectrum Basic Programming (2nd ed.). Sinclair Research Ltd. pp. 73–75. Retrieved 2022-05-26. The ZX Spectrum uses p=65537 and a=75, and stores some bi-1 in memory.
  9. ^ GNU Scientific Library: Other random number generators.
  10. Knuth, Donald (1981). Seminumerical Algorithms. The Art of Computer Programming. Vol. 2 (2nd ed.). Reading, MA: Addison-Wesley Professional. pp. 12–14.
  11. Bique, Stephen; Rosenberg, Robert (May 2009). Fast Generation of High-Quality Pseudorandom Numbers and Permutations Using MPI and OpenMP on the Cray XD1. Cray User Group 2009. The die is determined using modular arithmetic, e.g., lrand48() % 6 + 1, ... The CRAY RANF function only rolls three of the six possible outcomes (which three sides depends on the seed)!
  12. Greenberger, Martin (April 1961). "Notes on a New Pseudo-Random Number Generator". Journal of the ACM. 8 (2): 163–167. doi:10.1145/321062.321065. S2CID 17196519.
  13. ^ Nakazawa, Naoya; Nakazawa, Hiroshi (2025). "§7.1 The Present Best MC Generator #001". Random Number Generator on Computers. pp. 67–71. ISBN 978-1-003-41060-7. Note that the example implementation is suboptimal. Rather than maintaining state variables mz1 and mz2 and computing mz1a and mz2a each iteration, it is more efficient to maintain the latter as state variables. Also, the final reduction modulo m (called id in the book) is of a value that is less than 2m, so may consist of a single conditional subtraction.
  14. L'Ecuyer, Pierre; Tezuka, Shu (October 1991). "Structural Properties for Two Classes of Combined Random Number Generators" (PDF). Mathematics of Computation. 57 (196): 735–746. doi:10.2307/2938714. JSTOR 2938714.
  15. Schrage, Linus (June 1979). "A More Portable Fortran Random Number Generator" (PDF). ACM Transactions on Mathematical Software. 5 (2): 132–138. CiteSeerX 10.1.1.470.6958. doi:10.1145/355826.355828. S2CID 14090729.
  16. Jain, Raj (9 July 2010). "Computer Systems Performance Analysis Chapter 26: Random-Number Generation" (PDF). pp. 19–22. Retrieved 2017-10-31.
  17. ^ L'Ecuyer, Pierre; Côté, Serge (March 1991). "Implementing a random number package with splitting facilities". ACM Transactions on Mathematical Software. 17 (1): 98–111. doi:10.1145/103147.103158. S2CID 14385204. This explores several different implementations of modular multiplication by a constant.
  18. Fenerty, Paul (11 September 2006). "Schrage's Method". Retrieved 2017-10-31.
  19. L’Ecuyer, Pierre (January 1999). "Tables of linear congruential generators of different sizes and good lattice structure" (PDF). Mathematics of Computation. 68 (225): 249–260. CiteSeerX 10.1.1.34.1024. doi:10.1090/s0025-5718-99-00996-5.

External links

Categories: