What AI data work is

Why AI models need people: labels for traditional models, demonstrations and preferences for chatbots, and human testing for quality and safety. Plus who pays and what the 2026 demand data says.

Labelled data

Also called labels, supervised learning, training data, ground truth

What: Examples with the correct answer attached, such as a photo marked "bus" or a review marked "negative". Supervised learning trains models on these pairs.
Why: A model can only be as good as its labels. Inconsistent labels teach it confusion, which is why companies pay for careful human labelling and check it.

Watch out

Thinking labelling is mindless clicking; it is rule-following work judged on quality.

Demonstrations (SFT data)

Also called SFT, supervised fine-tuning, ideal answers

What: A prompt paired with the ideal answer, written by a person. Fine-tuning a model on many of these teaches it to follow instructions: supervised fine-tuning.
Why: Writing and fixing demonstrations is a core AI trainer task. The model copies their quality, errors included.

Preference data and RLHF

Also called RLHF, reinforcement learning from human feedback, human feedback, comparisons

What: Records of which of two or more answers a person preferred. In RLHF, a reward model learns from them and the chatbot is tuned to score well against it.
Why: Side-by-side and ranking tasks produce this data. Careless choices push models towards answers that only look good.
Example: InstructGPT (OpenAI, 2022) used hired labellers to write demonstrations and rank model outputs, the pattern most chatbot training still follows.

Reward model

What: A model trained on human preference data to predict which answer people would prefer, giving each answer a score.
Why: It stands in for human raters during training, so it inherits every bias and mistake in the ratings it learned from.

Eval (evaluation)

Also called evals, evaluation, benchmark

What: A set of test prompts plus a way to score the answers, by code or by people using a rubric, to measure how good a model is at something.
Why: Companies use evals to compare models and catch regressions. Human rating projects are often evals, and designing them is a growing job.

Red teaming (safety testing)

Also called safety testing, adversarial testing

What: Deliberately trying to make a model misbehave, such as giving harmful instructions or leaking data, under project rules, and reporting what worked.
Why: Problems found before release can be fixed. The useful output is a clear, reproducible failure report.

Watch out

Treating it as a game of breaking the model instead of documenting each failure precisely.

The AI data supply chain

Also called data vendors

What: AI labs hire data vendors (Scale AI, Appen, TELUS Digital, Toloka, Labelbox, Turing), which recruit workers through platforms such as Outlier, CrowdGen, Mindrift and Alignerr.
Why: It explains why projects stop suddenly, why you sign confidentiality terms, and why the platform, not the lab, pays you.

Watch out

Trusting an "agent" who claims to sit between you and a platform; real platforms recruit directly.

2026 demand evidence

What: Upwork In-Demand Skills 2026: AI data annotation and labelling demand up 154 percent year over year. LinkedIn Jobs on the Rise 2026 (US): data annotators among the fastest-growing roles.
Why: Real evidence of demand, but it describes marketplaces and the US, often for specialists and languages, and says nothing about work where you live.

Watch out

Reading a growth headline as a promise of steady income.

Go deeper