Physical Sciences and Mathematics Commons^™

Computing Highly Accurate Or Exact P-Values Using Importance Sampling, Chris Lloyd

Chris J. Lloyd

Especially for discrete data, standard first order P-values can suffer from poor accuracy, even for quite large sample sizes. Moreover, different test statistics can give practically different results. There are several approaches to computing P-values which do not suffer these defects, such as parametric bootstrap P-values or the partially maximised P-values of Berger and Boos (1994).

Both these methods require computing the exact tail probability of the approximate P-value as a function of the nuisance parameter/s, known as the significance profile. For most practical problems, this is not computationally feasible. I develop an importance sampling approach to this problem. A …

Estimated P-Values In Discrete Models: Asymptotic And Non-Asymptotic Effects, Chris Lloyd

Chris J. Lloyd

The exact null distribution of a P-value typically depends on nuisance parameters unspecified under the null. For discrete models and standard approximate P-values, this dependence can be quite strong. The estimated (or bootstrap) P-value is the exact probability of the P-value being no larger than its observed value, with the null estimate of the nuisance parameter substituted. For continuous models, it is known that such `bootstrap' P-values deviate from uniformity by terms of order m^{-3/2}, where m is a measure of sample size. The main difficulty with discrete models is the breakdown of asymptotics near the boundary. The aim of …

Exact Confidence Bounds For The Risk Ratio In 2x2 Tables With Structural Zero, Chris J. Lloyd, Max Moldovan

Chris J. Lloyd

This paper examines exact one-sided confidence limits for the risk ratio in a 2x2 table with structural zero. Starting with four approximate lower and upper limits, we adjust each using the algorithm of Buehler (1957) to arrive at lower (upper) limits that have exact coverage properties and are as large (small) as possible subject to coverage, as well as an ordering, constraint. Different Buehler limits are compared by their mean size, since all are exact in their coverage. Buehler limits based on the signed root likelihood ratio statistic are found to have the best performance and recommended for practical use.

Efficient And Exact Tests Of The Risk Ratio In A Correlated 2x2 Table With Structural Zero, Chris Lloyd

Chris J. Lloyd

For a correlated 2x2 table where the (01) cell is empty by design, the parameter of interest is typically the ratio of the probability of secondary response conditional on primary response to the probability of primary response, also known as a risk ratio. It is common to test whether or not the risk ratio equals one. One method of obtaining an exact P-value is to maximise the tail probability of the test statistic over the nuisance parameter. It is argued that better results are obtained by first replacing the nuisance parameter by its profile estimate in the calculation of its …