Remove duplicates across splits and keep the test set for a final check

Google's ML Crash Course recommends three sets: training, validation for decisions during development, and a test set held back for the end. It gives two warnings:

• Duplicates inflate scores. Remove any validation or test example that also appears in training. The course describes a spam model whose suspiciously high test precision came from duplicated emails.
• Test and validation sets wear out. The more often you use the same test set to make decisions, the more the model fits it. Refresh them with new data when you can.

In practice, deduplicate on a stable key, or on normalised content, before splitting, and look at the test set as rarely as possible: ideally once, after model selection is finished.