Creating tensors: shape, dtype and device

Every tensor has a shape, a dtype and a device

A tensor is an n-dimensional array whose elements all share one data type. Three attributes describe it: shape (size along each dimension), dtype (for example torch.float32 or torch.int64) and device (cpu, cuda, mps).

torch.tensor(data) copies Python data and infers the dtype: Python floats become float32, Python ints become int64. Factory functions build tensors of a given shape: torch.zeros, torch.ones, torch.rand (uniform in [0, 1)), torch.randn (standard normal), torch.arange and torch.linspace.

torch.from_numpy(arr) does not copy: the tensor shares memory with the NumPy array, so changing one changes the other. Note that NumPy's default float type is float64, while PyTorch models use float32 because it is faster on GPUs and takes half the memory. Mixing them causes errors like expected scalar type Float but found Double. Classification labels, on the other hand, are usually int64 (also called long).

From raw numbers to batched tensors

Data starts as Python lists or NumPy arrays, becomes a tensor with a fixed dtype, gets reshaped or indexed into the layout a model expects, and is finally combined with other tensors through broadcasting and matrix operations.

Creating tensors

The comments state what each line produces. Watch the last line: the NumPy-backed tensor is float64 and changes when the array changes.

import numpy as np
import torch

a = torch.tensor([1, 2, 3])          # dtype torch.int64
b = torch.tensor([1.0, 2.0, 3.0])    # dtype torch.float32
z = torch.zeros(2, 3)                # shape (2, 3), all zeros
r = torch.randn(4, 5)                # samples from a standard normal
s = torch.arange(0, 10, 2)           # tensor([0, 2, 4, 6, 8])

arr = np.array([1.0, 2.0])
t = torch.from_numpy(arr)            # float64, shares memory with arr
arr[0] = 99.0
print(t[0].item())                   # 99.0: same memory
print(a.dtype, b.dtype, t.dtype, z.shape)

Converting dtypes on purpose

Use .float(), .long() or .to(torch.float32) to convert. A common pattern after loading NumPy data is torch.from_numpy(arr).float(). Each conversion returns a new tensor; it does not change the original in place.

त्वरित जाँच: What is the dtype of `torch.tensor([1, 2, 3])`?

• torch.float32
• torch.int64
• torch.int32
• torch.float64