40th Annual Conference on Neural Information Processing Systems (NeurIPS 2026), NeurIPS’26 Workshop on AI-Native Academia: Authorship, Peer Review, and Conference Governance under AI, Atlanta, USA, December 9-13, 2026.
Scientific peer review increasingly struggles to assess reproducibility at the scale and complexity of modern research output. Evaluating reproducibility requires reconstructing experimental dependencies, methodological choices, data flows, and result-generating procedures, which often exceeds what human reviewers can provide given the growing number of submissions and time constraints. Agentic Reproducibility Assessment (ARA) formalizes reproducibility assessment as a structured reasoning task over scientific documents. Given a paper, ARA extracts a directed workflow graph linking sources, methods, experiments, and outputs, then evaluates its reconstructability using structural and content-based scores for reproducibility assessments. Experiments on 213 ReScience C articles – the largest cross-domain benchmark of human-validated computational reproducibility studies considered to date – demonstrate ARA’s generalizability and consistent workflow reconstruction and assessment across LLMs, model temperatures, and scientific domains. These results highlight ARA’s potential to complement human review at scale and support next-generation peer review. ARA consistently achieves 61% accuracy on three reproducibility assessment benchmarks, highlighting its potential to complement human review at scale and enabling next-generation peer review.
Kevin Riehl and Andres L. Marin contributed equally to this work and share first authorship.
Kevin Riehl
Doctoral Researcher & Scientist
My name is Kevin Riehl, and I am a cosmopolitan, technology enthusiast and philantrop. I believe, that technology is the key to make the world a better place, and that learning, self-improvement, collaboration and criticial thinking are our duty as gifted minds.