Physical Sciences and Mathematics Commons^™

Bootstrap P-Values In Discrete Models: Asymptotic And Non-Asymptotic Effects, Chris Lloyd

Chris J. Lloyd

(This paper is a major revision of http://works.bepress.com/chris_lloyd/15/.) Standard first order P-values suffer from two important drawbacks. First, even for quite large sample sizes they can misrepresent the exact significance which depends on nuisance parameters unspecified under the null. For most discrete models is that accuracy is variable and breaks down completely at the boundary. Second, different test statistics can give practically different results.

The bootstrap P-value is the exact significance with the null maximum estimate (ML) of the nuisance parameter substituted. We show that bootstrap P-values based on different first order statistics differ to second order. We also show …

Bootstrap And Second Order Tests Of Risk Difference, Chris Lloyd

Chris J. Lloyd

Standard approximate tests of the difference of two probabilities have type 1 error that can differ significantly from nominal, even for quite large sample sizes. There are two modern methods of reducing type 1 error. One is to use so-called higher order asymptotics (Reid, 2003) to provide an explicit adjustment to the likelihood ratio statistic. The second is to replace the nuisance parameter in an exact calculation with a null estimate (Young and Lee, 2005), which is a kind of bootstrap. The purpose of this paper is to explain and evaluate these two methods, for testing whether a difference in …