Leveraging jumped() in NumPy's MT19937 RNG: Applications in Parallel Computing and Reproducible Sampling

Understanding Random Number Generators (RNGs)

• NumPy's random module provides various RNGs, including MT19937 (Mersenne Twister), a popular choice for its good balance of speed and randomness quality.
• In random sampling, we rely on RNGs to generate sequences of seemingly random numbers. These sequences are not truly random, but they exhibit statistical properties that make them suitable for simulations and statistical analysis.

The jumped() Method

• This technique allows you to obtain different, non-overlapping subsequences from the overall random number stream generated by the MT19937 object.
• The jumped() method is specific to the MT19937 RNG. It essentially advances the internal state of the generator as if a large number of random numbers (2 raised to the power of 128 times the jumps argument) have been generated.

Applications of jumped() in Random Sampling

1. • When running simulations or calculations in parallel across multiple processes, you want each process to use a distinct sequence of random numbers to avoid correlations and ensure statistical independence.
   • You can create a single MT19937 object with a fixed seed (the initial state) and then use jumped() on copies of this object to generate different starting points for each process within the same overall random number stream.

2. Reproducible Subsets
   

   • In some cases, you might want to reproduce a specific portion of a random number sequence for debugging or analysis purposes. By calling jumped() with a calculated value, you can effectively jump to that desired point in the sequence.

Example (Parallel Computing)

import numpy as np

def my_parallel_simulation(process_id):
    # Create a base RNG with a fixed seed
    base_rng = np.random.MT19937(seed=42)

    # Use jumped() to get a distinct stream for each process
    process_rng = base_rng.jumped(process_id)

    # Use process_rng for your random sampling within the simulation

# Run simulations in parallel (pseudocode)
for process_id in range(num_processes):
    my_parallel_simulation(process_id)

Key Points

• Consider using other techniques like Philox or PCG for better performance or specific needs in random sampling.
• After using jumped(), you still need to call methods like random() or rand() to generate actual random numbers from the advanced state.
• jumped() doesn't directly generate random numbers itself. It modifies the internal state of the MT19937 object.

---

Parallel Random Sampling (Multiple Processes)

import numpy as np
from multiprocessing import Pool  # Import for parallel processing

def generate_random_samples(seed, num_samples):
    # Create base RNG with fixed seed
    base_rng = np.random.MT19937(seed=seed)

    # Generate random samples using the base RNG
    samples = base_rng.rand(num_samples)
    return samples

def parallel_sampling(num_processes, num_samples_per_process):
    # Prepare seeds for each process
    seeds = np.random.randint(1000, size=num_processes)  # Example seed generation

    # Use Pool to run generate_random_samples in parallel
    with Pool(num_processes) as pool:
        results = pool.starmap(generate_random_samples, zip(seeds, [num_samples_per_process] * num_processes))

    # Combine results from all processes
    all_samples = np.concatenate(results)
    return all_samples

if __name__ == "__main__":
    num_processes = 4
    num_samples_per_process = 1000

    all_samples = parallel_sampling(num_processes, num_samples_per_process)
    print(all_samples.shape)  # Should be (total_samples,)

This code uses multiprocessing.Pool to demonstrate parallel sampling. It creates a base RNG with a fixed seed and then uses jumped() on copies of this object (via different seeds) to generate random samples in each process.

Reproducible Subset of Random Numbers

import numpy as np

def generate_reproducible_subset(base_seed, jump_value, num_samples):
    # Create base RNG
    base_rng = np.random.MT19937(seed=base_seed)

    # Jump to the desired position in the sequence
    rng = base_rng.jumped(jump_value)

    # Generate random samples from the advanced state
    samples = rng.rand(num_samples)
    return samples

# Example usage
base_seed = 42
jump_value = 2**64  # Jump halfway through the sequence (adjust as needed)
num_samples = 100

reproducible_subset = generate_reproducible_subset(base_seed, jump_value, num_samples)
print(reproducible_subset)

# Run again with the same arguments to get the same subset
same_subset = generate_reproducible_subset(base_seed, jump_value, num_samples)
print(np.allclose(reproducible_subset, same_subset))  # Should be True

This code showcases using jumped() to obtain a specific subset of random numbers from the MT19937 stream. It jumps to a calculated position based on the jump_value and then generates the desired number of samples.

---

Multiple RNG Instances

• The simplest alternative is to create multiple MT19937 instances with distinct seeds. This ensures independent random number streams without modifying the internal state of a single object.

import numpy as np

def generate_random_samples(num_samples_per_stream, num_streams):
    # Create multiple RNG instances with different seeds
    rngs = [np.random.MT19937(seed=i) for i in range(num_streams)]

    # Generate samples from each RNG
    all_samples = []
    for rng in rngs:
        samples = rng.rand(num_samples_per_stream)
        all_samples.append(samples)

    return np.concatenate(all_samples)

# Example usage
num_samples_per_stream = 1000
num_streams = 4

all_samples = generate_random_samples(num_samples_per_stream, num_streams)
print(all_samples.shape)  # Should be (total_samples,)

Splitting the Random Stream

• NumPy's random.split() method allows you to create new RNG objects that are statistically independent but share the same underlying algorithm and state as the original one. This can be useful if you need multiple streams derived from a common base.

import numpy as np

def generate_random_samples(base_seed, num_streams, num_samples_per_stream):
    # Create base RNG
    base_rng = np.random.MT19937(seed=base_seed)

    # Split the base RNG to create multiple streams
    rngs = [base_rng.split() for _ in range(num_streams)]

    # Generate samples from each stream
    all_samples = []
    for rng in rngs:
        samples = rng.rand(num_samples_per_stream)
        all_samples.append(samples)

    return np.concatenate(all_samples)

# Example usage (same as previous example)
all_samples = generate_random_samples(42, 4, 1000)
print(all_samples.shape)

• NumPy offers other RNG types besides MT19937. Consider Philox or PCG (both in random) for potential performance benefits or specific randomness requirements. These might not have a direct equivalent to jumped(), but they provide alternative approaches to managing the state.

import numpy as np

def generate_random_samples(num_samples_per_stream, num_streams, rng_type=np.random.MT19937):
    # Create multiple RNG instances with different seeds
    rngs = [rng_type(seed=i) for i in range(num_streams)]

    # Generate samples from each RNG
    all_samples = []
    for rng in rngs:
        samples = rng.rand(num_samples_per_stream)
        all_samples.append(samples)

    return np.concatenate(all_samples)

# Example usage with Philox RNG
all_samples = generate_random_samples(1000, 4, rng_type=np.random.Philox)
print(all_samples.shape)