The maths you need

Vectors, matrices and the dot product

Also called linear algebra, dot product

What

A vector is an ordered list of numbers, such as one row of features; a matrix is a grid, such as a dataset with n rows and d columns. The dot product multiplies matching entries and sums them.

Why

Data, weights and embeddings are vectors and matrices, and a linear model prediction is just a dot product plus a bias.

w = np.array([0.4, -1.2, 2.0])
b = 0.5
x = np.array([1.0, 0.0, 3.0])
y_hat = w @ x + b          # one prediction: 6.9
X = np.random.rand(100, 3)
preds = X @ w + b          # all 100 predictions at once

Matrix multiplication and shapes

Also called matmul, shapes

What

Multiplying an (n, k) matrix by a (k, m) matrix gives an (n, m) matrix, where each entry is the dot product of a row and a column. The inner dimensions must match.

Why

Every dense neural network layer is a matrix multiplication plus a bias and an activation, and most bugs in model code are shape mismatches.

Example

A batch of 32 images flattened to 784 features, shape (32, 784), multiplied by a weight matrix of shape (784, 128) gives hidden activations of shape (32, 128).

Derivatives and gradients

Also called gradient, derivatives, partial derivative

What

A derivative is the rate at which a function changes as its input changes. The gradient is the vector of partial derivatives with respect to every parameter, and it points in the direction of steepest increase.

Why

Training minimises a loss, and the gradient says how to nudge each weight to make the loss smaller.

Example

For f(w) = w² the derivative is 2w: at w = 3 the slope is 6, so decreasing w lowers f, and the minimum is where the slope is 0.

Gradient descent

Also called SGD, stochastic gradient descent, mini-batch gradient descent

What

Repeatedly moving the parameters a small step against the gradient: w = w - lr * grad. The learning rate sets the step size; stochastic and mini-batch versions estimate the gradient from a sample of the data.

Why

It is how almost every model from logistic regression to transformers is trained.

rng = np.random.default_rng(0)
X, y = rng.random((200, 3)), rng.random(200)
w, lr = np.zeros(3), 0.1
for step in range(500):
    error = X @ w - y
    grad = 2 * X.T @ error / len(y)    # gradient of the MSE
    w -= lr * grad

The chain rule

What

The derivative of a function of a function is the product of their derivatives: if y = f(g(x)), then dy/dx = f'(g(x)) times g'(x).

Why

A neural network is a long chain of functions, and backpropagation applies the chain rule layer by layer to get the gradient of every weight.

Example

For loss = (wx - y)², let e = wx - y. Then d(loss)/de = 2e and de/dw = x, so d(loss)/dw = 2e times x.

Probability and Bayes' theorem

Also called Bayes, conditional probability, Bayes rule

What

Conditional probability P(A|B) is the chance of A given that B happened. Bayes' theorem reverses it: P(A|B) = P(B|A) P(A) / P(B), updating a prior belief with evidence.

Why

It underlies naive Bayes classifiers, probabilistic models and reasoning about base rates when judging a classifier on rare events.

Example

A test with 99% sensitivity and 95% specificity for a condition 1% of people have: P(condition | positive) = 0.0099 / (0.0099 + 0.0495), about 17%.

Probability distributions

Also called normal distribution, Gaussian, binomial, Poisson

What

Descriptions of how likely each value is: normal (the bell curve, set by mean and standard deviation), Bernoulli and binomial (yes or no outcomes), Poisson (counts per interval), uniform, and skewed ones such as log-normal.

Why

They guide the choice of loss, show when a log transform is needed, let you simulate data and explain what model outputs mean as probabilities.

Descriptive statistics

Also called mean, median, standard deviation, variance, correlation

What

Numbers that summarise data: mean and median for the centre, variance and standard deviation for spread, percentiles for shape, and correlation for the linear relationship between two variables.

Why

They are the first check on every dataset and feature, revealing skew, outliers and very different scales.

Hypothesis tests and p-values

Also called p-value, A/B test, t-test, statistical significance, confidence interval

What

A test asks how surprising the data would be if there were no real effect (the null hypothesis); the p-value is that probability. A confidence interval gives a plausible range for the size of the effect.

Why

They tell you whether a new model or product change beat the old one by more than noise, as in A/B tests.