Social and Behavioral Sciences Commons^™

An Empirical Demonstration Of The Need For Exact Tests, Vance W. Berger

Journal of Modern Applied Statistical Methods

The robustness of parametric analyses is rarely questioned or qualified. Robustness, generally understood, means the exact and approximate p-values will lie on the same side of alpha for any reasonable data set; and 1) any data set would qualify as reasonable and 2) robustness holds universally, for all alpha levels and approximations. For this to be true, the approximation would need to be perfect all of the time. Any discrepancy between the approximation and the exact p-value, for any combination of alpha level and data set, would constitute a violation. Clearly, this is not true, and when confronted with this …