What Are Automated Replicability Predictions For? | Lukas Röseler

Lukas Röseler asks what automated replicability predictions are for, given that replications are costly and poorly rewarded. The talk first asks whether prediction can work: there is no consensus on how to define replication success, and the six criteria in the FReD explorer give different success rates; average success rates are imprecise; and published replications are not representative, with differences by journal, by registered reports and by author overlap. Röseler then reviews current approaches (human prediction markets, algorithms such as z-curve, and LLMs) and gives reasons why prediction might fail, from effect heterogeneity to weak theories. If it does work, predictions should serve as a non-evaluative metric for self-awareness, meta-science and choosing which findings to replicate, not for research assessment, as a substitute for replication or to discredit fields. https://t1p.de/roeseloer Speaker: Lukas Röseler (University of Münster) Talk title: What Are Automated Replicability Predictions For? Session: Session 6 · Predicting replicability Recorded at AI in Metascience 2026, an online conference organised by FORRT (Framework for Open and Reproducible Research Training), 28–30 September 2026. Day 2, Tuesday 29 September: Replications and reproductions. Programme and abstracts: https://forrt.org/ai-metascience/ FORRT: https://forrt.org