Koch et al. (2021) found that more than 70% of benchmark datasets used in prominent computer-vision papers had been reused from other domains.
Notes on verification
Directly confirmed by the peer-reviewed NeurIPS 2021 paper (71.9% figure) and corroborated by an independent author interview describing the same qualitative finding. [tier=unverified indep_score=0.3 clusters=2 claim_tier=notable] [rescored 2026-09-15: curated origin-host map (PR #65); unclassified hosts no longer scored as aggregators]
Sources
- Can We Trust AI Benchmarks? An Interdisciplinary Review ... (seed:technology_and_ai)
- https://datasets-benchmarks-proceedings.neurips.cc/paper/2021/file/3b8a614226a953a8cd9526fca6fe9ba5-Paper-round2.pdf (corroboration)
- https://aihub.org/2022/02/17/the-life-of-a-dataset-in-machine-learning-research-interview-with-bernard-koch/ (corroboration)