PyTorch Tensor Basics¶
This section covers: * Converting NumPy arrays to PyTorch tensors * Creating tensors from scratch
Perform standard imports¶
import torch
import numpy as np
Confirm you're using PyTorch version 1.1.0+
torch.__version__
Converting NumPy arrays to PyTorch tensors¶
A torch.Tensor is a multi-dimensional matrix containing elements of a single data type.
Calculations between tensors can only happen if the tensors share the same dtype.
In some cases tensors are used as a replacement for NumPy to use the power of GPUs (more on this later).
arr = np.array([1,2,3,4,5])
print(arr)
print(arr.dtype)
print(type(arr))
x = torch.from_numpy(arr)
# Equivalent to x = torch.as_tensor(arr)
print(x)
# Print the type of data held by the tensor
print(x.dtype)
# Print the tensor object type
print(type(x))
print(x.type()) # this is more specific!
arr2 = np.arange(0.,12.).reshape(4,3)
print(arr2)
x2 = torch.from_numpy(arr2)
print(x2)
print(x2.type())
Here torch.DoubleTensor refers to 64-bit floating point data.
Tensor Datatypes¶
| TYPE | NAME | EQUIVALENT | TENSOR TYPE |
|---|---|---|---|
| 32-bit integer (signed) | torch.int32 | torch.int | IntTensor |
| 64-bit integer (signed) | torch.int64 | torch.long | LongTensor |
| 16-bit integer (signed) | torch.int16 | torch.short | ShortTensor |
| 32-bit floating point | torch.float32 | torch.float | FloatTensor |
| 64-bit floating point | torch.float64 | torch.double | DoubleTensor |
| 16-bit floating point | torch.float16 | torch.half | HalfTensor |
| 8-bit integer (signed) | torch.int8 | CharTensor | |
| 8-bit integer (unsigned) | torch.uint8 | ByteTensor |
Copying vs. sharing¶
torch.from_numpy()
torch.as_tensor()
torch.tensor()
There are a number of different functions available for creating tensors. When using torch.from_numpy() and torch.as_tensor(), the PyTorch tensor and the source NumPy array share the same memory. This means that changes to one affect the other. However, the torch.tensor() function always makes a copy.
# Using torch.from_numpy(), shares same memory
arr = np.arange(0,5)
t = torch.from_numpy(arr)
print(t)
arr[2]=77
print(t)
# Using torch.tensor(), makes a copy
arr = np.arange(0,5)
t = torch.tensor(arr)
print(t)
arr[2]=77
print(t)
Class constructors¶
torch.Tensor()
torch.FloatTensor()
torch.LongTensor(), etc.
There's a subtle difference between using the factory function torch.tensor(data) and the class constructor torch.Tensor(data).
The factory function determines the dtype from the incoming data, or from a passed-in dtype argument.
The class constructor torch.Tensor()is simply an alias for torch.FloatTensor(data). Consider the following:
data = np.array([1,2,3])
a = torch.Tensor(data) # Equivalent to cc = torch.FloatTensor(data)
print(a, a.type())
b = torch.tensor(data)
print(b, b.type())
c = torch.tensor(data, dtype=torch.long)
print(c, c.type())
Creating tensors from scratch¶
Uninitialized tensors with .empty()¶
torch.empty() returns an uninitialized tensor. Essentially a block of memory is allocated according to the size of the tensor, and any values already sitting in the block are returned. This is similar to the behavior of numpy.empty().
x = torch.empty(4, 3)
print(x)
Initialized tensors with .zeros() and .ones()¶
torch.zeros(size)
torch.ones(size)
It's a good idea to pass in the intended dtype.
x = torch.zeros(4, 3, dtype=torch.int64)
print(x)
Tensors from ranges¶
torch.arange(start,end,step)
torch.linspace(start,end,steps)
Note that with .arange(), end is exclusive, while with linspace(), end is inclusive.
x = torch.arange(0,18,2).reshape(3,3)
print(x)
x = torch.linspace(0,18,12).reshape(3,4)
print(x)
Tensors from data¶
torch.tensor() will choose the dtype based on incoming data:
x = torch.tensor([1, 2, 3, 4])
print(x)
print(x.dtype)
print(x.type())
# Converting type
x = x.type(torch.int16)
print(x.dtype)
print(x.type())
Alternatively you can set the type by the tensor method used. For a list of tensor types visit https://pytorch.org/docs/stable/tensors.html
x = torch.FloatTensor([5,6,7])
print(x)
print(x.dtype)
print(x.type())
You can also pass the dtype in as an argument. For a list of dtypes visit https://pytorch.org/docs/stable/tensor_attributes.html#torch.torch.dtype
x = torch.tensor([8,9,-3], dtype=torch.int)
print(x)
print(x.dtype)
print(x.type())
Changing the dtype of existing tensors¶
Don't be tempted to use x = torch.tensor(x, dtype=torch.type) as it will raise an error about improper use of tensor cloning.
Instead, use the tensor .type() method.
print('Old:', x.type())
x = x.type(torch.int64)
print('New:', x.type())
Random number tensors¶
torch.rand(size) returns random samples from a uniform distribution over [0, 1)
torch.randn(size) returns samples from the "standard normal" distribution [σ = 1]
Unlike rand which is uniform, values closer to zero are more likely to appear.
torch.randint(low,high,size) returns random integers from low (inclusive) to high (exclusive)
x = torch.rand(4, 3)
print(x)
x = torch.randn(4, 3)
print(x)
x = torch.randint(0, 5, (4, 3))
print(x)
Random number tensors that follow the input size¶
torch.rand_like(input)
torch.randn_like(input)
torch.randint_like(input,low,high)
these return random number tensors with the same size as input
x = torch.zeros(2,5)
print(x)
x2 = torch.randn_like(x)
print(x2)
The same syntax can be used with
torch.zeros_like(input)
torch.ones_like(input)
x3 = torch.ones_like(x2)
print(x3)
Setting the random seed¶
torch.manual_seed(int) is used to obtain reproducible results
torch.manual_seed(42)
x = torch.rand(2, 3)
print(x)
torch.manual_seed(42)
x = torch.rand(2, 3)
print(x)
Tensor attributes¶
Besides dtype, we can look at other tensor attributes like shape, device and layout
x.shape
x.size() # equivalent to x.shape
x.device
PyTorch supports use of multiple devices, harnessing the power of one or more GPUs in addition to the CPU. We won't explore that here, but you should know that operations between tensors can only happen for tensors installed on the same device.
x.layout
PyTorch has a class to hold the memory layout option. The default setting of strided will suit our purposes throughout the course.
