Statistics and Probability Commons^™

Notes On Hypothesis Testing Under A Single-Stage Design In Phase Ii Trial, Kung-Jong Lui

Journal of Modern Applied Statistical Methods

A primary objective of a phase II trial is to determine future development is warranted for a new treatment based on whether it has sufficient activity against a specified type of tumor. Limitations exist in the commonly-used hypothesis setting and the standard test procedure for a phase II trial. This study reformats the hypothesis setting to mirror the clinical decision process in practice. Under the proposed hypothesis setting, the critical points and the minimum required sample size for a desired power of finding a superior treatment at a given α -level are presented. An example is provided to illustrate how …

Generalized Variances Ratio Test For Comparing K Covariance Matrices From Dependent Normal Populations, Marcelo Angelo Cirillo, Daniel Furtado Ferreira, Thelma Sáfadi, Eric Batista Ferreira

Journal of Modern Applied Statistical Methods

New tests based on the ratio of generalized variances are presented to compare covariance matrices from dependent normal populations. Monte Carlo simulation concluded that the tests considered controlled the Type I error, providing empirical probabilities that were consistent with the nominal level stipulated.

On Testing For Significant Quantitative Trait Loci (Qtl) Effects When Variances Are Unequal, Pradeep Singh, Shesh N. Rai

Conference on Applied Statistics in Agriculture

The basic theory of QTL (Quantitative Trait Loci) mapping is to score a population for a quantitative trait according to the marker genotype, and then to use statistics to identify differences associated with the markers and the quantitative trait of interest. Permutation based methods have been used to estimate threshold values for quantitative mapping. The permutation test based on the Student t-test for equality of means does not control Type I error rate to its nominal value when variances are unequal. In this study we propose a modification of the Student t-test based on the jackknife estimator of population variance. …

Type Ii Robustness Of The Null Hypothesis Rho = 0 For Non-Normal Distributions, Stephanie Wren

Wayne State University Dissertations

Is the t test statistic for the Pearson Product Moment Correlation Coefficient robust to errors of the second kind? This investigation indirectly measured the effects of power through a type 2 error rate robustness study. The results were revealing.

Detecting Lag-One Autocorrelation In Interrupted Time Series Experiments With Small Datasets, Clare Riviello, S. Natasha Beretvas

Journal of Modern Applied Statistical Methods

The power and type I error rates of eight indices for lag-one autocorrelation detection were assessed for interrupted time series experiments (ITSEs) with small numbers of data points. Performance of Huitema and McKean’s (2000) z_HM statistic was modified and compared with the z_HM, five information criteria and the Durbin-Watson statistic.

Quel Test For Two Linear Restrictions In The Nonlinear Models, Krishna K. Saha

Journal of Modern Applied Statistical Methods

An alternative Wald type test called the quel test is developed for two linear restrictions by finding the critical region based on the quel utilizing the repeated values of estimated parameters of interest under the null. Simulation shows evidence that the full quel test performs best in that it holds nominal level well and shows monotonic increasing power properties.

Comparative Power Of The Independent T, Permutation T, And Wilcoxontests, Michèle Weber, Shlomo Sawilowsky

Journal of Modern Applied Statistical Methods

The nonparametric Wilcoxon Rank Sum (also known as the Mann-Whitney U) and the permutation t-tests are robust with respect to Type I error for departures from population normality, and both are powerful alternatives to the independent samples Student’s t-test for detecting shift in location. The question remains regarding their comparative statistical power for small samples, particularly for non-normal distributions. Monte Carlo simulations indicated the rank-based Wilcoxon test was found to be more powerful than both the t and the permutation t-tests.

Robustness To Non-Independence And Power Of The I Test For Trend In Construct Validity, John L. Cuzzocrea, Shlomo Sawilowsky

Journal of Modern Applied Statistical Methods

The Multitrait-Multimethod Matrix is used to evaluate construct validity; Sawilowsky (2002) created the I test to analyze the matrix. This article examined the robustness and power of the Sawilowsky I test. Ad hoc critical values were determined to improve the statistical power of the technique for analyzing the Multitrait-Multimethod Matrix.

Comparing Different Methods For Multiple Testing In Reaction Time Data, Massimiliano Pastore, Massimo Nucci, Giovanni Galfano

Journal of Modern Applied Statistical Methods

Reaction times were simulated for examining the power of six methods for multiple testing, as a function of sample size and departures from normality. Power estimates were low for all methods for non-normal distributions. With normal distributions, even for small sample sizes, satisfactory power estimates were observed, especially for FDR-based procedures.

Reflecting On The Experience Sampling Method In The Qualitative Research Context: Focus On Knowledge Production And Power During The Data-Collection Process

Fredline MCormack-Hale

Tests For 2 X 2 Tables In Clinical Trials, Vic Hasselblad, Yulia Lokhnygina

Journal of Modern Applied Statistical Methods

Five standard tests are compared: chi-squared, Fisher's exact, Yates’ correction, Fisher’s exact mid-p, and Barnard’s. Yates’ is always inferior to Fisher’s exact. Fisher’s exact is so conservative that one should look for alternatives. For certain sample sizes, Fisher’s mid-p or Barnard’s test maintain the nominal alpha and have superior power.

Sample Size Determination In Animal Health Studies, Zhanglin Cui, Alan G. Zimmermann, Daniel H. Mowrey

Conference on Applied Statistics in Agriculture

Oftentimes in animal health studies, a treatment group is randomly assigned to a pen of animals, and the pen of animals as a whole is treated (fed the same medicated feed or water) together. In this scenario, the pen of animals is the experimental unit and the individual animal may be an observational unit. In addition to having the pen as the experimental unit, if multiple sites are used and site is treated as a random factor, this adds complexity to the study. To properly design the study, it is necessary to determine the number of animals in a pen, …

Reliability, Effect Size, And Responsiveness And Intraclass Correlation Of Health Status Measures Used In Randomized And Cluster-Randomized Trials, Paula Diehr, Lu Chen, Donald L. Patrick, Ziding Feng, Yutaka Yasui

UW Biostatistics Working Paper Series

Background: New health status instruments are described by psychometric properties, such as Reliability, Effect Size, and Responsiveness. For cluster-randomized trials, another important statistic is the Intraclass Correlation for the instrument within clusters. Studies using better instruments can be performed with smaller sample sizes, but better instruments may be more expensive in terms of dollars, lost opportunities, or poorer data quality due to the response burden of longer instruments. Investigators often need to estimate the psychometric properties of a new instrument, or of an established instrument in a new setting. Optimal sample sizes for estimating these properties have not been studied …

Misconceptions Leading To Choosing The T Test Over The Wilcoxon Mann-Whitney Test For Shift In Location Parameter, Shlomo S. Sawilowsky

Journal of Modern Applied Statistical Methods

There exist many misconceptions in choosing the t over the Wilcoxon Rank-Sum test when testing for shift. Examples are given in the following three groups: (1) false statement, (2) true premise, but false conclusion, and (3) true statement irrelevant in choosing between the t test and the Wilcoxon Rank Sum test.

Power Of The T Test For Normal And Mixed Normal Distributions, Marilyn S. Thompson, Samuel B. Green, Yi-Hsin Chen, Shawn Stockford, Wen-Juo Lo

Journal of Modern Applied Statistical Methods

Previous research suggests that the power of the independent-samples t test decreases when population distributions are mixed normal rather than normal, and that robust methods have superior power under these conditions. However, under some conditions, the power for the independent-samples t test can be greater when the population distributions for the independent groups are mixed normal rather than normal. The implications of these results are discussed.

Sample Size Selection For Pair-Wise Comparisons Using Information Criteria, Xuemei Pan, C. Mitchell Dayton

Journal of Modern Applied Statistical Methods

This article provides results for rates of correct identifications of paired-comparison information criteria and Tukey HSD as functions of the pattern of mean differences and of sample size. Therefore, the tables provided are useful for selecting sample sizes in real world applications.

Test Statistics Null Distributions In Multiple Testing: Simulation Studies And Applications To Genomics, Katherine S. Pollard, Merrill D. Birkner, Mark J. Van Der Laan, Sandrine Dudoit

U.C. Berkeley Division of Biostatistics Working Paper Series

Multiple hypothesis testing problems arise frequently in biomedical and genomic research, for instance, when identifying differentially expressed or co-expressed genes in microarray experiments. We have developed generally applicable resampling-based single-step and stepwise multiple testing procedures (MTP) for control of a broad class of Type I error rates, defined as tail probabilities and expected values for arbitrary functions of the numbers of false positives and rejected hypotheses (Dudoit and van der Laan, 2005; Dudoit et al., 2004a,b; Pollard and van der Laan, 2004; van der Laan et al., 2005, 2004a,b). As argued in the early article of Pollard and van der …

Multiple Testing Procedures And Applications To Genomics, Merrill D. Birkner, Katherine S. Pollard, Mark J. Van Der Laan, Sandrine Dudoit

U.C. Berkeley Division of Biostatistics Working Paper Series

This chapter proposes widely applicable resampling-based single-step and stepwise multiple testing procedures (MTP) for controlling a broad class of Type I error rates, in testing problems involving general data generating distributions (with arbitrary dependence structures among variables), null hypotheses, and test statistics (Dudoit and van der Laan, 2005; Dudoit et al., 2004a,b; van der Laan et al., 2004a,b; Pollard and van der Laan, 2004; Pollard et al., 2005). Procedures are provided to control Type I error rates defined as tail probabilities for arbitrary functions of the numbers of Type I errors, V_n, and rejected hypotheses, R_n. These error rates include: …

On Optimizing Multi-Level Designs: Power Under Budget Constraints, Todd C. Headrick, Bruno D. Zumbo

Todd Christopher Headrick

This paper derives a procedure for efficiently allocating the number of units in multi-level designs given prespecified power levels. The derivation of the procedure is based on a constrained optimization problem that maximizes a general form of a ratio of expected mean squares subject to a budget constraint. The procedure makes use of variance component estimates to optimize designs during the budget formulating stages. The method provides more general closed form solutions than other currently available formulae. As such, the proposed procedure allows for the determination of the optimal numbers of units for studies that involve more complex designs. A …

Multiple Testing Procedures For Controlling Tail Probability Error Rates, Sandrine Dudoit, Mark J. Van Der Laan, Merrill D. Birkner

U.C. Berkeley Division of Biostatistics Working Paper Series

The present article discusses and compares multiple testing procedures (MTP) for controlling Type I error rates defined as tail probabilities for the number (gFWER) and proportion (TPPFP) of false positives among the rejected hypotheses. Specifically, we consider the gFWER- and TPPFP-controlling MTPs proposed recently by Lehmann & Romano (2004) and in a series of four articles by Dudoit et al. (2004), van der Laan et al. (2004b,a), and Pollard & van der Laan (2004). The former Lehmann & Romano (2004) procedures are marginal, in the sense that they are based solely on the marginal distributions of the test statistics, i.e., …

Multiple Testing Procedures: R Multtest Package And Applications To Genomics, Katherine S. Pollard, Sandrine Dudoit, Mark J. Van Der Laan

U.C. Berkeley Division of Biostatistics Working Paper Series

The Bioconductor R package multtest implements widely applicable resampling-based single-step and stepwise multiple testing procedures (MTP) for controlling a broad class of Type I error rates, in testing problems involving general data generating distributions (with arbitrary dependence structures among variables), null hypotheses, and test statistics. The current version of multtest provides MTPs for tests concerning means, differences in means, and regression parameters in linear and Cox proportional hazards models. Procedures are provided to control Type I error rates defined as tail probabilities for arbitrary functions of the numbers of false positives and rejected hypotheses. These error rates include tail probabilities …

A Different Future For Social And Behavioral Science Research, Shlomo S. Sawilowsky

Journal of Modern Applied Statistical Methods

The dissemination of intervention and treatment outcomes as effect sizes bounded by conf idence intervals in order to think meta-analytically was promoted in a recent article in Educational Researcher. I raise concerns with unfettered reporting of effect sizes, point out the con in confidence interval, and caution against thinking meta-analytically. Instead, cataloging effect sizes is recommended for sample size estimation and power analysis to improve social and behavioral science research.

Small Sample Power Characteristics Of Generalized Mixed Model Procedures For Binary Repeated Measures Data Using Sas, Matthew Beckman, Walter W. Stroup

Conference on Applied Statistics in Agriculture

Researchers in the agricultural and biological sciences often conduct experiments with repeated measures and categorical response variables. Recent advances in statisticalcomputing have made several options available to analyze data from these experiments. For example, SAS has several procedures based on generalized mixed model theory. These include PROC GENMOD, MIXED, NLMIXED, and the GLIMMIX macro. Inference for these procedures depends on asymptotic theory. While statistics literature contains some information about the small-sample behavior, there is much that remains unknown. This presentation will focus on Bernoulli response variables. Power characteristics are compared via simulation for several scenarios involving relatively small repeated measures …

A Longitudinal Follow-Up Of Discrete Mass At Zero With Gap, Joseph L. Musial, Patrick D. Bridge, Nicol R. Shamey

Journal of Modern Applied Statistical Methods

The first part of this paper discusses a five-year systematic review of the Journal of Consulting and Clinical Psychology following the landmark power study conducted by Sawilowsky and Hillman (1992). The second part discusses a five-year longitudinal follow-up of a radically nonnormal population distribution: discrete mass at zero with gap. This distribution was based upon a real dataset.

Exact Level And Power Of Permutation, Bootstrap, And Asymptotic Tests Of Trend, Christopher D. Corcoran, Cyrus R. Mehta

Journal of Modern Applied Statistical Methods

We develop computational tools that can evaluate the exact size and power of three tests of trend (e.g., permutation, bootstrap and asymptotic) without resorting to large-sample theory or simulations. We then use these tools to compare the operating characteristics of the three tests. It is seen that the bootstrap test is ultra-conservative relative to the other two tests and as a result suffers from a severe deterioration in power. The power of the asymptotic test is uniformly larger than that of the other two tests, but it fails to preserve the Type I error for most of the range of …

An Investigation Of The Rank Transformation In Multple Regression, Todd C. Headrick, Ourania Rotou

Todd Christopher Headrick

Real world data often fail to meet the underlying assumptions of normal statistical theory. The rank transformation (RT) procedure is recommended and used in the context of multiple regression analysis when the assumption of normality is violated. There is no general supporting theory of the RT. In view of this, the current study examined the Type I error and power properties of the RT in terms of multiple regression. The investigation included both additive and nonadditive models. Results indicated that there were severely inflated Type I error rates associated with the RT procedure under both normal and nonnormal distributions (e.g., …

Empirical Estimates Of Power For Binomial Data With Mixed Models, R. K. Splan, L. D. Van Vleck, H. D. Hafs

Conference on Applied Statistics in Agriculture

Observations on return to estrus from anestrus postpartum beef cows were used as the basis for a simulation study to develop a method to determine numbers of locations and animals per treatment per location to achieve a specified power of test. Estimates of among location and total variance were obtained by REML from the data set and then used to generate simulated data for the binomial trait. Each combination of several pre-determined factors was replicated 1000 times. Pre-determined factors were number of locations, number of animals per treatment per location, desired detectable difference due to treatment, alpha-probability level and ratio …