Matrix multiplication and shapes

You can multiply a matrix of shape (m, n) by one of shape (n, p), and the answer has shape (m, p). The two inner numbers must match. Each cell of the answer is the dot product of one row from the first matrix and one column from the second.

This is how a whole batch of predictions is made at once: X of shape (1000, 3) times weights of shape (3, 1) gives 1,000 predictions, shape (1000, 1). Every neural network layer later on is a matrix multiplication plus a bias. Reading shapes here is what lets you fix PyTorch errors like 'mat1 and mat2 shapes cannot be multiplied'. Also learn the transpose, which swaps rows and columns.