Chapter 2 · Practice 3 of 5
Split by group or by time when rows are not independent
Why it matters
Random splits assume rows are independent and identically distributed. scikit-learn's cross-validation guide warns that this rarely holds in practice, and names two common breaks:
| Data shape | Problem with a random split | Use |
|---|---|---|
| Several rows per patient, user or device | The model memorises the entity, not the pattern | GroupKFold, StratifiedGroupKFold, GroupShuffleSplit |
| Rows ordered in time | The model trains on the future to predict the past | TimeSeriesSplit |
For groups, the guide says every sample in the validation fold must come from groups not represented at all in the training fold. For time, it says to evaluate on future observations, and warns that shuffling time-ordered data such as news articles is likely to inflate the validation score.
Google's Rule #33 says the same for the final check: if you train on data up to 5 January, test on data from 6 January onwards.
Example
from sklearn.model_selection import GroupKFold, cross_val_score
cv = GroupKFold(n_splits=5)
scores = cross_val_score(model, X, y, groups=df["customer_id"], cv=cv)