Indexing, views and reshaping

Storage, strides and views

Indexing works like NumPy: x[0], x[:, 1], x[1:3], and boolean masks such as x[x > 0]. Underneath, a tensor is metadata (shape, strides, offset) pointing into a block of memory called its storage. Many operations only change the metadata and return a view that shares storage with the original. Writing into a view therefore changes the original too.

view(new_shape) always returns a view and never copies; it raises an error if the memory layout does not allow it. reshape(new_shape) returns a view when possible and silently copies otherwise. transpose, t() and permute return views with swapped strides, which makes the result non-contiguous; flattening it with .view(-1) fails, while .reshape(-1) or .contiguous().view(-1) works. A -1 in a shape means "infer this size".

unsqueeze(d) inserts a size-1 dimension at position d (for example, adding a batch dimension), and squeeze(d) removes one.

Views share memory

Changing m changes x, because view only re-describes the same storage. After a transpose, reshape works but view would raise a RuntimeError.

import torch

x = torch.arange(12)          # 0..11, shape (12,)
m = x.view(3, 4)              # shares storage with x
m[0, 0] = 100
print(x[0].item())            # 100: same memory

t = m.t()                     # transpose -> shape (4, 3), non-contiguous
print(t.is_contiguous())      # False
flat = t.reshape(-1)          # works (copies because it must)
# t.view(-1) would raise a RuntimeError here

img = torch.rand(3, 28, 28)
batch = img.unsqueeze(0)      # add a batch dimension: (1, 3, 28, 28)
print(batch.shape)
print(batch.squeeze(0).shape) # back to torch.Size([3, 28, 28])

Same students, new seating chart

A view is like printing a new seating chart for the same classroom. Arranging 12 students as 3 rows of 4 or 4 rows of 3 does not move anyone; it only changes how you read the chart. If you give one student a prize using the new chart, the same student has it on the old chart too. A copy, by contrast, is a different classroom.

Quick check: Which statement about `view` and `reshape` is correct?

• `view` always copies the data, `reshape` never does
• Both always copy, so changing the result never affects the original
• `reshape` only works on contiguous tensors
• `view` never copies and errors if it cannot, while `reshape` copies when a view is impossible