Medicine and Health Sciences Commons^™

In Search Of Justification For The Unpredictability Paradox, Jeremy Howick, Alexander Mebius

Alexander Mebius

A 2011 Cochrane Review found that adequately randomized trials sometimes revealed larger, sometimes smaller, and often similar effect sizes to inadequately randomized trials. However, they found no average statistically significant difference in effect sizes between the two study types. Yet instead of concluding that adequate randomization had no effect the review authors postulated the “unpredictability paradox”, which states that randomized and non-randomized studies differ, but in an unpredictable direction. However, stipulating the unpredictability paradox is problematic for several reasons: 1) it makes the authors’ conclusion that adequate randomization makes a difference unfalsifiable—if it turned out that adequately randomized trials had …