Data science: Towards ML engineering

Sefism.

A to Z

Glossary

Every concept in this sheet, plus the smaller terms it uses. Select a concept to open its card.

A
Arrays and vectors
A block of values of one type: a NumPy array in Python, an atomic vector in R.
C
Common distributions
Shapes that describe how a random quantity behaves: normal for sums of many small effects, binomial for counts of successes in n tries, Poisson for counts of events per interval.
Conditional probability and Bayes' rule
P(A|B) is the probability of A given that B happened.
Confidence interval
A range built so that 95% of intervals made this way would contain the true value.
Correlation versus causation
Correlation measures how two variables move together, from -1 to 1.
D
DataFrame and tibble
A table in memory: named columns, each with one type, and rows that line up across them. pandas in Python, tibbles in R.
G
Group by and aggregate
Splitting rows into groups by one or more columns, computing a summary for each group and combining the results into a new table.
H
Hypothesis test and p-value
Assume no effect (the null hypothesis) and ask how surprising your data would be.
L
Long and wide format
Wide data has one column per period or category; long data has one row per observation with a column saying which one.
M
Mean, median, standard deviation and IQR
The mean is the sum over n; the median is the middle value; the standard deviation is the typical distance from the mean; the IQR is Q3 minus Q1, the spread of the middle half.
Merging tables
Combining two data frames on a key column, like a SQL join: merge in pandas, left_join and friends in dplyr.
N
Notebook workflow
Code, output, charts and notes in one document you run cell by cell: Jupyter or Colab for Python, RStudio with Quarto or R Markdown for R.
R
Reproducible environment
A per-project record of the exact package versions an analysis used, so it runs the same on another machine or next year.
S
Sampling distribution and the CLT
If you repeated your sample many times, the sample mean would vary; that spread is its sampling distribution.
Selecting and filtering rows
Keeping the columns you need and the rows that meet a condition: loc and boolean masks in pandas, select and filter in dplyr.
Standard error
The standard deviation of an estimate across repeated samples.
T
Type I, type II errors and power
A type I error is finding an effect that is not there (rate alpha, usually 5%).
V
Vectorised operations
Writing one expression over a whole column instead of looping over rows, so the work runs in fast compiled code.
W
Which test when
Two group means: Welch t-test.
11 · Glossary, A to Z11 / 11