Physical Sciences and Mathematics Commons^™

A Synthetic Prediction Market For Estimating Confidence In Published Work, Sarah Rajtmajer, Christopher Griffin, Jian Wu, Robert Fraleigh, Laxmann Balaji, Anna Squicciarini, Anthony Kwasnica, David Pennock, Michael Mclaughlin, Timothy Fritton, Nishanth Nakshatri, Arjun Menon, Sai Ajay Modukuri, Rajal Nivargi, Xin Wei, Lee Giles

Computer Science Faculty Publications

[First paragraph] Concerns about the replicability, robustness and reproducibility of findings in scientific literature have gained widespread attention over the last decade in the social sciences and beyond. This attention has been catalyzed by and has likewise motivated a number of large-scale replication projects which have reported successful replication rates between 36% and 78%. Given the challenges and resources required to run high-powered replication studies, researchers have sought other approaches to assess confidence in published claims. Initial evidence has supported the promise of prediction markets in this context. However, they require the coordinated, sustained effort of collections of human experts …