Physical Sciences and Mathematics Commons^™

A Review Of Frequentist Tests For The 2x2 Binomial Trial, Chris Lloyd

Chris J. Lloyd

The 2x2 binomial trial is the simplest of data structures yet its statistical analysis and the issues it raises have been debated and revisited for over 70 years. Which analysis should biomedical researchers use in applications? In this review, we consider frequentist tests only, specifically tests with control size either exactly or very close to exactly. These procedures can be classified as conditional and unconditional. Amongst tests motivated by a conditional model, Lancaster’s mid-p and Liebermeister’s test are less conservative than Fisher’s classical test, but do not control type 1 error. Within the conditional framework, only Fisher’s test can be …

On The Exact Size Of Multiple Comparison Tests, Chris Lloyd

Chris J. Lloyd

No abstract provided.

On The Size Accuracy Of Combination Tests, Chris Lloyd

Chris J. Lloyd

One element of the analysis of adaptive clinical trials is combining the evidence from several (often two) stages. When the endpoint is binary, standard single stage tests statistics do not control size well. Yet the combined test might not be valid if the single stage tests are not. The purpose of this paper is to numerically and theoretically examine the extent to which combining basic tests statistics mitigates or magnifies the size violation of the final test.

Some Non-Asymptotic Properties Of Parametric Bootstrap P-Values, Chris Lloyd

Chris J. Lloyd

The bootstrap P-value is the exact tail probability of a test statistic, cal-culated assuming the nuisance parameter equals the null maximum likelihood (ML) estimate. For discrete data, bootstrap P-values perform amazingly well even for small samples, even as standard first order methods perform surprisingly poorly. Why is this? Detailed numerical calculations in Lloyd (2012a) strongly suggest that the good performance of bootstrap is not explained by asymptotics. In this paper, I establish several desirable non-asymptotic properties of bootstrap P-values. The most important of these is that bootstrap will correct ‘bad’ ordering of the sample space which leads to a more …

Computing Highly Accurate Confidence Limits From Discrete Data Using Importance Sampling, Chris Lloyd

Chris J. Lloyd

For discrete parametric models, approximate confidence limits perform poorly from a strict frequentist perspective. In principle, exact and optimal confidence limits can be computed using the formula of Buehler (1957), Lloyd and Kabaila (2003). So-called profile upper limits (Kabaila \& Lloyd, 2001) are closely related to Buehler limits and have extremely good properties. Both profile and Buehler limits depend on the probability of a certain tail set as a function of the unknown parameters. Unfortunately, this probability surface is not computable for realistic models. In this paper, importance sampling is used to estimate the surface and hence the confidence limits. …

Computing Highly Accurate Or Exact P-Values Using Importance Sampling (Revised), Chris Lloyd

Chris J. Lloyd

Especially for discrete data, standard first order P-values can suffer from poor accuracy, even for quite large sample sizes. Moreover, different test statistics can give practically different results. There are several approaches to computing P-values which do not suffer these defects, such as parametric bootstrap P-values or the partially maximised P-values of Berger & Boos (1994).

Both these methods require computing the exact tail probability of the approximate P-value as a function of the nuisance parameter/s, known as the significance profile. For most practical problems this is not computationally feasible. I develop an importance sampling approach to this problem. A …

Bootstrap P-Values In Discrete Models: Asymptotic And Non-Asymptotic Effects, Chris Lloyd

Chris J. Lloyd

(This paper is a major revision of http://works.bepress.com/chris_lloyd/15/.) Standard first order P-values suffer from two important drawbacks. First, even for quite large sample sizes they can misrepresent the exact significance which depends on nuisance parameters unspecified under the null. For most discrete models is that accuracy is variable and breaks down completely at the boundary. Second, different test statistics can give practically different results.

The bootstrap P-value is the exact significance with the null maximum estimate (ML) of the nuisance parameter substituted. We show that bootstrap P-values based on different first order statistics differ to second order. We also show …

More Powerful Unconditional Tests Of No Treatment Effect From Binary Matched Pairs, Chris Lloyd

Chris J. Lloyd

This is the workign paper version that preceeded the paper "A New Exact and More Powerful Unconditional Test of no Treatment Effect from Binary Matched Pairs" published in Biometrics 76 (also on this site:http://works.bepress.com/chris_lloyd/3/