Curation is not cleaning and it is not labeling
The three are routinely conflated, and the conflation obscures where the leverage actually sits.
A perfectly cleaned, perfectly labeled dataset can still produce a poor model, because the composition was wrong: the tails were underrepresented, one class dominated, the same scenario appeared in ten thousand near-identical variations, or the distribution failed to match production.
Cleaning and labeling are necessary. Curation is what determines the ceiling.
Why scale stopped solving this
For a period, the reliable path to a better model was more data. That relationship has weakened substantially, for reasons that are structural rather than temporary.
Redundancy dominates large corpora. In video, consecutive frames are near-identical — a thirty-second clip at thirty frames per second contains nine hundred frames and perhaps forty genuinely distinct moments. In text, common phrasings recur at enormous multiples. Adding redundant records increases training cost and dataset size without adding information.
Marginal value declines sharply. The ten-thousandth typical example teaches a model almost nothing. The first example from an underrepresented tail can meaningfully change behaviour on an entire segment of production traffic.
Errors scale with volume. A 3% label error rate on 100,000 records is 3,000 wrong labels. At scale, error volume grows faster than the capacity to find it, and errors concentrated in a specific slice are functionally invisible in aggregate metrics.
Compute is not free. Training cost scales with dataset size. Spending it on redundant records is a direct transfer from budget to no outcome.
The consequence is that the composition question — what should be in this dataset — has become more important than the volume question.
The curation decisions that matter
- Coverage against production distribution. The training set should represent what the model will encounter, including the tails. This requires knowing the production distribution, which requires observability. Curation performed without production feedback is guesswork.
- Deduplication beyond exact match. Near-duplicates — the same scene at a slightly different angle, the same question phrased differently, consecutive video frames — are where the volume is. Semantic deduplication is materially harder and materially more valuable.
- Class balance, deliberately chosen. Natural distributions are usually imbalanced, and the imbalance usually favours the class that matters least. Whether to rebalance must be a decision rather than an accident.
- Edge case identification and retention. The rare, ambiguous, and difficult records are the most informative per unit of annotation cost, and the most likely to be discarded by naive filtering because they look like noise.
- Quality-weighted inclusion. Not every record deserves equal weight. Verified labels, gold-standard items, and high-agreement records carry more signal than unverified automation output.
What multimodal adds
Multimodal training compounds every curation problem and introduces several that are specific to it.
Cross-modal alignment. An image-caption pair requires the caption to correspond to the image. Misalignment is common in scraped data, hard to detect automatically, and directly harmful — the model learns an incorrect association rather than simply failing to learn a correct one.
Divergent quality standards by modality. Image quality is resolution and composition. Audio quality is signal-to-noise and speaker separation. Video adds temporal consistency. Text adds coherence and factuality. A single quality score across modalities is not meaningful.
Volume asymmetry. One minute of video at thirty frames per second is eighteen hundred frames. One minute of audio is a few hundred words of transcript. Naive combination produces a dataset where one modality overwhelms the others by orders of magnitude.
Preference and evaluation data. For LLM alignment, curation extends to the selection of prompts, the diversity of response pairs, and the coverage of failure modes the model is expected to handle. Preference datasets that oversample easy comparisons produce alignment that fails on the hard ones.
Curation is continuous, not initial
The most consequential structural point: curation is usually treated as a one-time step before the first training run, and it is not.
Production distribution shifts. New categories appear. Edge cases that were rare become common. A dataset curated correctly for the world of twelve months ago is miscurated for today, and nothing in a static pipeline will surface that.
Continuous curation requires a feedback path from production back to the dataset — a way to observe which slices are failing, determine whether the failure reflects a coverage gap or a labeling error, and adjust composition accordingly. Without that path, curation decays from the moment it is made.
How Concave AI approaches this
Datalier treats curation as an ongoing operation informed by production behaviour rather than a preprocessing step.
Composition measured at every version. Class distribution, coverage metrics, duplicate rate, and edge-case representation are computed and reported for each frozen version. Distribution changes appear explicitly in the version delta, so composition drift across iterations is visible rather than accumulating unnoticed.
Deduplication with near-match detection. Exact and near-duplicate identification during transformation, with counts and examples reported rather than records silently removed.
Modality-aware handling. Dozens of annotation types across text, image, video, and audio, each with its own quality criteria and processing path. Video uses keyframe propagation rather than treating every frame as an independent image, addressing temporal redundancy at the architectural level.
Uncertainty-first task ordering. The human review queue prioritises low-confidence and high-disagreement items, concentrating annotation effort where it produces the most information per unit of cost.
Coverage informed by production. The observability layer identifies which production slices are underperforming and whether the corresponding training coverage is thin. Curation decisions are made against observed production behaviour rather than assumed distribution.
Continuous correction. Failing slices generate targeted correction datasets through four selection strategies — failing slices, low confidence, recent errors, or drift-affected records. Corrections route through labeling and human verification, are frozen as a new version with full lineage, and export for retraining.
Curation stops being a decision made once at the beginning and becomes a property of the pipeline that is maintained.
Conclusion
More data was never really the strategy — it was a proxy for it, and one that stopped working once redundancy, marginal value, and compute cost caught up with it. What actually moves model quality now is the discipline of deciding, continuously and with production feedback, what belongs in the training set and what does not. Teams that treat curation as a one-time preprocessing step will keep re-learning that lesson on every retrain.
