Machine learning

Stage 1 · Lesson 6 of 9

groupby, merge and reshaping

Python for data

Most analysis questions follow one pattern: split the rows into groups, work out something for each group, then combine the answers. In pandas that is groupby, for example df.groupby('region')['sales'].agg(['sum', 'mean']). Joining two tables is merge, which works like a : inner, left, right or outer, matched on one or more key columns.

Reshaping uses pivot_table to turn rows into columns and melt to turn columns back into rows. Together these answer questions like 'average order value per customer per month'.

Common trapThe trap with merge is duplicate keys. If a customer appears twice in the right table, every matching left row is doubled and your totals quietly grow. Check the row count before and after every merge, or pass validate='many_to_one' so pandas stops with an error instead.

A question to check yourself, an AI practice session and 1 cheat sheet entry come with the full roadmap. Unlock

Resources for this stage

Start with· follow one of these

Practise

  • The Python, Pandas and Data Visualization micro courses, in browser notebooks

Keep handy

  • Start with 10 minutes to pandas, then look up indexing, groupby and reshaping

  • Quick NumPy, pandas and Matplotlib lookups with a try-it editor

Mark this stage as not done?