Tauseef FayyazA to Z
Glossary
Every concept in this sheet, plus the smaller terms it uses. Select a concept to open its card.
B
- backward and .grad
- Calling
.backward()on a scalar computes the gradient of it with respect to every leaf tensor that requires grad, and adds the result into each tensor's.grad. - Broadcasting
- When two tensors of different shapes meet in an operation, PyTorch lines up their shapes from the right and stretches size-one dimensions to match, without copying data.
C
- Creating tensors
- A tensor is a typed, n-dimensional array.
D
- Data types
- Every tensor has one dtype.
- detach
tensor.detach()returns a tensor that shares the data but is cut off from the graph, so no gradient flows through it.- Devices
- A tensor lives on one device: the CPU, an NVIDIA GPU (
cuda), Apple silicon (mps) or others.
I
- In-place operations
- Methods ending in an underscore, such as
add_andzero_, change a tensor in place instead of returning a new one. - Indexing and masking
- Tensors support NumPy-style slicing, integer index lists and boolean masks.
N
- no_grad and inference_mode
torch.no_grad()turns off graph recording inside its block.
R
- Reductions and matrix maths
- Reductions such as
sum,mean,maxandargmaxtake adimto reduce along. - requires_grad
- A flag on a tensor that tells autograd to track operations on it.
S
- Shapes and reshaping
shapelists the size of each dimension.
T
- The computation graph
- As tracked operations run, autograd builds a graph linking each result to its inputs through a
grad_fn.