Data Analysis Best Practices

Chapter 2 · Practice 1 of 4

State what one row represents before you query a table

From Read the data correctly

Why it matters

The grain of a table is what one row means: one order, one order line, one user per day. Most join and aggregation mistakes come from guessing it wrong.

Check it rather than assume it. If you believe order_items has one row per order line, the key should be unique:

sql
select order_id, line_number, count(*) as rows_per_key
from order_items
group by order_id, line_number
having count(*) > 1;   -- expect zero rows

The same check in pandas is df.duplicated(subset=["order_id", "line_number"]).sum(), and in Polars df.select(pl.struct("order_id", "line_number").is_duplicated().sum()). Write the confirmed grain in a comment next to the query that reads the table.

Do

  • Write the grain of each input table at the top of the analysis