Alternatives to torch.Tensor.new_empty() for Tensor Creation in PyTorch

Purpose

• Creates a new tensor with a specified size but filled with uninitialized (uncertain) values.

Functionality

1. Size
   You provide the desired dimensions of the tensor as a tuple in the size argument. For example, torch.Tensor.new_empty((3, 4)) creates a tensor with 3 rows and 4 columns.
2. Uninitialized Values
   The elements of the tensor will contain arbitrary values that haven't been explicitly set. These values can vary between runs and are not guaranteed to be consistent.
3. Inheriting Properties (Optional)
   
   • By default, the new tensor inherits the dtype (data type) and device (CPU or GPU) from the tensor it's called on (if provided).
   • You can optionally specify dtype and device arguments to create the tensor with a different data type or placement on hardware.

Example

import torch

# Create a tensor of size (2, 3) with uninitialized values (default dtype=float32, device=cpu)
empty_tensor = torch.Tensor.new_empty((2, 3))
print(empty_tensor)  # Output: tensor([[  1.9135e-41,  -1.1642e-38,  -7.4483e-41],
                        #        [  2.2250e-41,   8.9406e-39,  -4.2915e-41]])

# Create a tensor with different data type (int64) and on GPU (if available)
if torch.cuda.is_available():
    device = torch.device('cuda')
else:
    device = torch.device('cpu')
int_tensor = my_tensor.new_empty((2, 2), dtype=torch.int64, device=device)
print(int_tensor)  # Output will vary depending on device and execution

Key Points

• Consider using torch.zeros(size) to create a tensor filled with zeros for numerical computations.
• Avoid relying on the uninitialized values for calculations, as they can lead to unpredictable results.
• Use torch.empty(size) for a similar function without inheriting properties.

• You're working with dynamic data shapes that might not be known beforehand.
• You need a placeholder tensor with a specific size for later operations.

---

Creating a placeholder tensor for later operations (linear regression example)

import torch

# Define input and output sizes
input_size = 10
output_size = 1

# Create empty tensors for weights and bias (uninitialized)
weights = torch.Tensor.new_empty((input_size, output_size))
bias = torch.Tensor.new_empty(output_size)

# Later in your code, you might assign actual values to weights and bias
# based on calculations or training data
# ...

# Use weights and bias in your linear regression model
# ...

Creating a tensor with a specific data type and device (GPU example)

import torch

# Assuming you have a pre-existing tensor on CPU (my_tensor)
my_tensor = torch.randn(2, 3)

# Check for GPU availability
if torch.cuda.is_available():
    device = torch.device('cuda')
else:
    device = torch.device('cpu')

# Create a new empty tensor on GPU (if available) with long data type (int64)
new_tensor = my_tensor.new_empty((4, 4), dtype=torch.int64, device=device)
print(new_tensor)  # This will print a tensor with uninitialized values on GPU (if available)

import torch

# Function that takes a list of data points and creates a tensor
def create_data_tensor(data_list):
  # Get the number of data points and their features (assuming all have same number of features)
  num_data, num_features = len(data_list), len(data_list[0])
  # Create an empty tensor with the dynamic size based on the data
  data_tensor = torch.Tensor.new_empty((num_data, num_features))
  # Later, you can populate the data_tensor based on the actual data points
  # ...
  return data_tensor

# Example usage
data_list = [[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]]
data_tensor = create_data_tensor(data_list)
print(data_tensor.size())  # Output: torch.Size([2, 3])

---

torch.empty(size)

• However, torch.empty does not inherit properties (dtype and device) from an existing tensor. It uses the default settings (usually float32 on CPU).
• This function creates a tensor of the specified size with uninitialized values similar to torch.Tensor.new_empty().

torch.zeros(size, dtype=None, device=None, requires_grad=False)

• It offers optional arguments for dtype (data type), device (CPU or GPU), and requires_grad (whether to track gradients for autograd).
• This function creates a tensor of the specified size filled with zeros.

torch.ones(size, dtype=None, device=None, requires_grad=False)

• It has the same optional arguments as torch.zeros.
• This function creates a tensor of the specified size filled with ones.

NumPy Integration

• This can be convenient for interoperability between the two libraries.
• If you're already working with NumPy arrays, you can convert them to PyTorch tensors using torch.from_numpy(numpy_array).

Choosing the Right Alternative

• Working with NumPy arrays? Use torch.from_numpy(numpy_array).
• Need a tensor filled with ones? Use torch.ones(size).
• Need a tensor filled with zeros (numerical computations)? Use torch.zeros(size).
• Need a tensor with uninitialized values and inherit properties? Use torch.empty(size).

• For numerical computations, it's generally better to use torch.zeros to ensure consistent starting values.
• Avoid relying on uninitialized values in calculations, as they can lead to unexpected results.