Beyond torch.Tensor.uniform_: Using torch.distributions.uniform.Uniform for Advanced Randomness in PyTorch

Purpose

• Useful for generating random samples within that range for various applications in machine learning, such as initializing weights in neural networks or simulating random events.
• Represents the continuous uniform distribution, a probability distribution where all values within a specified range are equally likely.

Functionality

• Methods
  
  • sample(sample_shape=torch.Size([])): Generates random samples from the uniform distribution.
    • sample_shape (torch.Size, optional): Desired output shape. Defaults to a scalar.
  • log_prob(value): Computes the log-probability density of a given value under the uniform distribution.
    • value (torch.Tensor): The value to compute the log-probability for.
  • Other methods for distribution properties like cdf (cumulative distribution function), icdf (inverse cumulative distribution function), entropy, etc. (refer to PyTorch documentation for details).
• Parameters
  
  • low (float, tensor): Lower bound of the uniform distribution (inclusive). Defaults to 0.
  • high (float, tensor): Upper bound of the uniform distribution (exclusive). Defaults to 1.
  • validate_args (bool, optional): If True, checks for validity of arguments (e.g., low < high). Defaults to None (follows global PyTorch setting).
• Class
  torch.distributions.uniform.Uniform inherits from the base class torch.distributions.distribution.

Example

import torch
from torch.distributions import Uniform

# Create a uniform distribution between 2 and 5
dist = Uniform(low=2.0, high=5.0)

# Generate 5 random samples
samples = dist.sample(torch.Size([5]))
print(samples)

# Example output:
# tensor([4.8221, 3.1805, 2.5302, 3.7438, 4.2002])

# Calculate log-probability of a specific value (e.g., 3.5)
log_prob_value = dist.log_prob(torch.tensor(3.5))
print(log_prob_value)

# Example output:
# tensor(-0.3466)  (may vary due to randomness)

Key Points

• torch.distributions.uniform provides a more structured and customizable approach to generating uniform random samples compared to the in-place torch.Tensor.uniform_ method.
• Samples and probabilities are computed on tensors, enabling efficient vectorized operations.

• Explore other probability distributions offered by torch.distributions for various scenarios in deep learning and machine learning.
• For backpropagation through random samples, explore techniques like REINFORCE or pathwise derivative estimators (refer to PyTorch documentation on probability distributions).

---

Generating Samples with Different Shapes

import torch
from torch.distributions import Uniform

# Uniform distribution between 0 and 1
dist = Uniform(low=0.0, high=1.0)

# Generate 3 samples
samples_3 = dist.sample(torch.Size([3]))
print(samples_3)

# Generate a 2x2 grid of samples
samples_grid = dist.sample(torch.Size([2, 2]))
print(samples_grid)

This code shows how you can control the shape of the generated samples by passing a desired sample_shape argument to the sample method.

Calculating Probability Densities

import torch
from torch.distributions import Uniform

# Uniform distribution between -2 and 4
dist = Uniform(low=-2.0, high=4.0)

# Probability density for values -1, 0, and 2
values = torch.tensor([-1.0, 0.0, 2.0])
probs = dist.log_prob(values)
print(probs)

This code demonstrates using log_prob to calculate the log-probability density for specific values. Remember that the output is the log probability, not the actual probability itself.

Simulating Events with Rejection Sampling

import torch
from torch.distributions import Uniform

def is_in_circle(x, y):
  # Define a circle centered at (0.5, 0.5) with radius 0.25
  return (x - 0.5)**2 + (y - 0.5)**2 <= 0.25**2

# Uniform distribution between 0 and 1 for both x and y
dist_x = Uniform(low=0.0, high=1.0)
dist_y = Uniform(low=0.0, high=1.0)

# Number of samples to generate
num_samples = 1000

# Rejection sampling loop
points_inside = []
for _ in range(num_samples):
  x = dist_x.sample()
  y = dist_y.sample()
  if is_in_circle(x, y):
    points_inside.append([x, y])

# Convert points to a tensor
points_tensor = torch.tensor(points_inside)
print(points_tensor.shape)  # Should be torch.Size([number_of_points_inside, 2])

This code uses rejection sampling to simulate points uniformly distributed within a circle. The idea is to generate random points from a square (using two uniform distributions) and then reject points that fall outside the circle. This approach ensures that the final points are uniformly distributed within the circle's area.

---

torch.rand

• It's simpler and more concise for basic tasks.
• This is a built-in PyTorch function that directly generates random samples from a uniform distribution between 0 (inclusive) and 1 (exclusive).

Example

import torch

# Generate a tensor of random numbers between 0 and 1 (shape 3x4)
random_tensor = torch.rand(3, 4)
print(random_tensor)

torch.Tensor.uniform_ (In-Place Modification)

• It's efficient for modifying existing tensors, but it can be less readable than using a dedicated distribution class.
• This method modifies an existing tensor in-place, filling it with random samples from a uniform distribution.

Example

import torch

# Create a tensor and fill it with uniform samples between 2 and 5
tensor = torch.zeros(5)
tensor.uniform_(2.0, 5.0)
print(tensor)

Choosing the Right Approach

• torch.Tensor.uniform_ is useful when you want to directly modify an existing tensor with uniform samples, but be cautious about in-place modifications.
• torch.rand is a simpler option for basic scenarios where you just need to generate random values between 0 and 1.
• torch.distributions.uniform.Uniform is ideal when you need more control over the distribution parameters (like specifying bounds), want to calculate probabilities or log-probabilities, or require the flexibility of a dedicated class for advanced distribution operations within your deep learning model.

• Remember to consider the trade-off between simplicity and control when selecting your approach.
• For distributions beyond the uniform case, explore other classes within torch.distributions, such as Normal, Bernoulli, or Categorical.