Why Git does not work for training data
The instinct to reach for Git is correct in spirit and wrong in mechanics.
Git is built for text files, line-based diffs, and merge resolution by a human reading a conflict. Training data is binary, large, and changes in ways that produce no meaningful line diff. A ten-gigabyte image directory does not diff. A re-annotated segmentation mask does not merge.
The ecosystem has responded — DVC, lakeFS, Pachyderm, Delta Lake, Nessie, Dolt, Git LFS, with consolidation now underway following lakeFS's acquisition of DVC in late 2025. These tools solve the storage and reproducibility problem competently: content-addressed storage, commit history, branch and merge on data.
What they do not solve is annotation state.
The gap: versioning files versus versioning labels
Existing data versioning tools treat a dataset as a collection of files. Commit, branch, roll back, reproduce. The unit of change is a file.
For AI training data, the unit of change is usually not the file. It is the annotation layer sitting on top of the file.
Consider a realistic sequence on a single video dataset: the raw video is ingested and never changes again. An object detection pass runs — bounding boxes are produced, reviewed, corrected, frozen. That is one version. Weeks later, the same footage is annotated for temporal segmentation — action boundaries across the timeline. Different task, different output, same underlying asset. Second version. Later still, pose tracking runs across the same clips for a separate model. Third version. Then production drift is detected on the detection model, 300 clips are re-labeled, and a corrected detection version is frozen. Fourth version — a descendant of the first, not the third.
A file-based versioning tool sees one unchanging video directory and several sets of annotation files. It can store them. It cannot express that they are parallel annotation layers over a shared asset, that two of them are unrelated tasks, and that the fourth supersedes the first.
That relationship is exactly what an ML team needs to reason about.
What annotation-aware versioning must record
One more property matters as much as any of the six above: reproducibility. Enough recorded state that the exact dataset can be reconstructed. Silent breaking changes, missing environment pins, and unversioned datasets cause more outages than model drift ever will — and irreproducible training runs make every subsequent experiment uninterpretable, because an architecture improvement cannot be distinguished from a data change.
Why this matters more in multimodal work
Multimodal datasets multiply every versioning problem.
Layers evolve at different rates. The transcript for a video may be corrected three times while the object tracks are untouched. Versioning them as a single monolithic snapshot forces unnecessary re-freezing and obscures what actually changed.
Modalities have different quality criteria. Transcription quality is word error rate. Segmentation quality is mask IoU. Classification quality is agreement and calibration. A single composite score across all of them is not meaningful, so quality must be recorded per layer.
Cross-modal consistency is itself a versioned property. If the transcript is corrected but the video annotations referencing it are not, the dataset is internally inconsistent — and that inconsistency is a property of a particular version combination, not of any single layer.
Rollback must be selective. Reverting a bad transcription pass should not revert three weeks of correct segmentation work performed in parallel.
Versioning as the substrate for everything else
Versioning is often filed under governance and treated as overhead. In practice it is load-bearing for three operations teams perform constantly.
Experiment attribution. When model v3 outperforms v2, the question is whether the architecture changed or the data did. Only immutable, hash-verified dataset versions answer this.
Safe correction cycles. When drift triggers re-labeling, the corrected data becomes a new version with a recorded delta from its parent. Versioning both datasets and models ensures stability and allows safe rollbacks if the new model underperforms. Without it, correction is an irreversible mutation.
Audit and reproduction. The question how was this model trained has an answer only if the exact training state was preserved at the moment of training.
How Concave AI approaches this
Datalier versions annotation state over shared assets, with quality and lineage attached to every snapshot.
Assets stored once, annotated many times. Raw media is ingested through a connected S3, GCS, Azure, Hugging Face, PostgreSQL, or Snowflake source and stored once. Multiple annotation tasks run against the same asset without duplicating it — a video dataset can carry object tracking, temporal segmentation, and pose tracking layers simultaneously.
Semantic version numbering with recorded intent. Major for schema change or reprocessing. Minor for new labels, corrections, or quality improvement. Patch for small fixes. Every version records its parent, its delta, and the stated reason for the bump.
Branching for parallel annotation strategies. Alternative label taxonomies or threshold settings can be explored on the same base data without disturbing the main line, then compared before one is promoted.
Immutable, hash-verified snapshots. Freezing produces a SHA-256 hash covering data, labels, and metadata. A single changed row changes the hash — cryptographic proof that an export matches the version that passed review.
Quality computed per version. Verified agreement, gold-standard accuracy, calibration error, class balance, and edge coverage, resolving to a readiness verdict: ready, review recommended, or not ready. Versions failing configured gates are blocked from export rather than silently shipped.
Lineage reports generated automatically. Nine sections per version — identity, source provenance, transformation chain, labeling history, version changes, quality summary, lineage path, export evidence, and risk assessment — written for three audiences in one document: an executive summary with the verdict, a technical section for engineers, and export evidence for audit.
Version comparison. Any two versions can be diffed: rows added and removed, labels changed, class distribution shift, quality delta. Composition drift across iterations becomes visible rather than accumulating unnoticed.
Export in each modality's native format, with provenance attached. COCO JSON, YOLO, DPO JSONL, CoNLL, SQuAD, ImageFolder, mask pairs, MOT CSV, SRT, RTTM. Data files, manifest, and lineage report ship together as one package.
Push directly to connected storage. A frozen version exports to the customer's own S3, GCS, Azure, or Hugging Face Hub, landing as three artifacts: the training data, the manifest, and the lineage report.
Conclusion
Git changed software engineering not because it stored files but because it made every state of a codebase recoverable, comparable, and attributable. Training data has needed the same thing for years and has been served by tools that solved only the storage half.
The missing half is annotation state — the layers of meaning applied over an asset, evolving at different rates, across modalities, with quality that must be measured at the moment of freezing rather than asserted afterwards.
When every annotation state is immutable, hash-verified, semantically tagged, and accompanied by its own quality report and lineage, dataset management stops being an act of discipline and becomes a property of the system. Teams stop asking which version we trained on, and start asking better questions.
