Social and Behavioral Sciences Commons^™

A Strategy For Using Bias And Rmse As Outcomes In Monte Carlo Studies In Statistics, Michael Harwell

Journal of Modern Applied Statistical Methods

To help ensure important patterns of bias and accuracy are detected in Monte Carlo studies in statistics this paper proposes conditioning bias and root mean square error (RMSE) measures on estimated Type I and Type II error rates. A small Monte Carlo study is used to illustrate this argument.

Performance Evaluation Of Confidence Intervals For Ordinal Coefficient Alpha, Heather J. Turner, Prathiba Natesan, Robin K. Henson

Journal of Modern Applied Statistical Methods

The aim of this study was to investigate the performance of the Fisher, Feldt, Bonner, and Hakstian and Whalen (HW) confidence intervals methods for the non-parametric reliability estimate, ordinal alpha. All methods yielded unacceptably low coverage rates and potentially increased Type-I error rates.

Jmasm35: A Percentile-Based Power Method: Simulating Multivariate Non-Normal Continuous Distributions (Sas), Jennifer Koran, Todd C. Headrick

Journal of Modern Applied Statistical Methods

The conventional power method transformation is a moment-matching technique that simulates non-normal distributions with controlled measures of skew and kurtosis. The percentile-based power method is an alternative that uses the percentiles of a distribution in lieu of moments. This article presents a SAS/IML macro that implements the percentile-based power method.

Robust Regression Methods For Massively Decayed Intelligence Data, Akiva Joachim Lorenz

Wayne State University Dissertations

Homeland Security, sponsored by governmental initiatives, has become a vibrant academic research field. However, most efforts were placed with the recognition of threats (e.g. theory) and response options. Less effort was placed in the analysis of the collected data through statistical modeling. In a field that collects more than 20 terabyte of information per minute though diverse overt and covert means and indexes it for future research, understanding how different statistical models behave when it comes to massively decayed data is of vital importance.

Using Monte Carlo methods, three regression techniques (ordinary least squares, least-trimmed, and maximum likelihood) were tested …

A Monte Carlo Simulation Of The Robust Rank-Order Test Under Various Population Symmetry Conditions, William T. Mickelson

Journal of Modern Applied Statistical Methods

The Type I Error Rate of the Robust Rank Order test under various population symmetry conditions is explored through Monte Carlo simulation. Findings indicate the test has difficulty controlling Type I error under generalized Behrens-Fisher conditions for moderately sized samples.

Height-Diameter Relationship In Tree Modeling Using Simultaneous Equation Techniques In Correlated Normal Deviates, S. O. Oyamakin

Journal of Modern Applied Statistical Methods

In other to study the complex simultaneous relationships existing in forest/tree growth modeling, six estimation methods of a simultaneous equation model are examined to determine how they cope with varying degrees of correlation between pairs of random deviates using average parameter estimates. A two-equation simultaneous system assumed covariance matrix was considered. The model was structured to have a mutual correlation between pairs of random deviates: a violation of the assumption of mutual independence between pairs of such random deviates. The correlation between the pairs of normal deviates were generated using three scenarios r = 0.0, 0.3 and 0.5. The performances …

Bias In Monte Carlo Simulations Due To Pseudo-Random Number Generator Initial Seed Selection, Jack C. Hill, Shlomo S. Sawilowsky

Journal of Modern Applied Statistical Methods

Pseudo-random number generators can bias Monte Carlo simulations of the standard normal probability distribution function with initial seeds selection. Five generator designs were initial-seeded with values from 10000_HEX to 1FFFF_HEX, estimates of the mean were calculated for each seed, the distribution of mean estimates was determined for each generator and simulation histories were graphed for selected seeds.

Assessing Trends: Monte Carlo Trials With Four Different Regression Methods, Daniel R. Thompson

Journal of Modern Applied Statistical Methods

Ordinary Least Squares (OLS), Poisson, Negative Binomial, and Quasi-Poisson Regression methods were assessed for testing the statistical significance of a trend by performing 10,000 simulations. The Poisson method should be used when data follow a Poisson distribution. The other methods should be used when data follow a normal distribution.

Impact Of Rank-Based Normalizing Transformations On The Accuracy Of Test Scores, Shira R. Soloman, Shlomo S. Sawilowsky

Journal of Modern Applied Statistical Methods

The purpose of this article is to provide an empirical comparison of rank-based normalization methods for standardized test scores. A series of Monte Carlo simulations were performed to compare the Blom, Tukey, Van der Waerden and Rankit approximations in terms of achieving the T score’s specified mean and standard deviation and unit normal skewness and kurtosis. All four normalization methods were accurate on the mean but were variably inaccurate on the standard deviation. Overall, deviation from the target moments was pronounced for the even moments but slight for the odd moments. Rankit emerged as the most accurate method among all …

A Monte Carlo Power Analysis Of Traditional Repeated Measures And Hierarchical Multivariate Linear Models In Longitudinal Data Analysis, Hua Fang, Gordon P. Brooks, Maria L. Rizzo, Kimberly A. Espy, Robert S. Barcikowski

Journal of Modern Applied Statistical Methods

The power properties of traditional repeated measures and hierarchical linear models have not been clearly determined in the balanced design for longitudinal studies in the current literature. A Monte Carlo power analysis of traditional repeated measures and hierarchical multivariate linear models are presented under three variance-covariance structures. Results suggest that traditional repeated measures have higher power than hierarchical linear models for main effects, but lower power for interaction effects. Significant power differences are also exhibited when power is compared across different covariance structures. Results also supplement more comprehensive empirical indexes for estimating model precision via bootstrap estimates and the approximate …

The Effect On Type I Error And Power Of Various Methods Of Resolving Ties For Six Distribution-Free Tests Of Location, Bruce R. Fay

Journal of Modern Applied Statistical Methods

The impact on Type I error robustness and power for nine different methods of resolving ties was assessed for six distribution-free statistics with four empirical data sets using Monte Carlo techniques. These statistics share an underlying assumption of population continuity such that samples are assumed to have no equal data values (no zero difference–scores, no tied ranks). The best results across all tests and combinations of simulation parameters were obtained by randomly resolving ties, although there were exceptions. The method of dropping ties and reducing the sample size performed poorly.

Jmasm23: Cluster Analysis In Epidemiological Data (Matlab), Andrés M. Alonso

Journal of Modern Applied Statistical Methods

Matlab functions for testing the existence of time, space and time-space clusters of disease occurrences are presented. The classical scan test, the Ederer, Myers and Mantel’s test, the Ohno, Aoki and Aoki’s test, and the Knox’s test are considered.

The Influence Of Reliability On Four Rules For Determining The Number Of Components To Retain, Gibbs Y. Kanyongo

Journal of Modern Applied Statistical Methods

Imperfectly reliable scores impact the performance of factor analytic procedures. A series of Monte Carlo studies was conducted to generate scores with known component structure from population matrices with varying levels of reliability. The scores were submitted to four procedures: Kaiser rule, scree plot, parallel analysis, and modified Horn’s parallel analysis to find if each procedure accurately determines the number of components at the different reliability levels. The performance of each procedure was judged by the percentage of the number of times that the procedure was correct and the mean components that each procedure extracted in each cell. Generally, the …

Determining The Correct Number Of Components To Extract From A Principal Components Analysis: A Monte Carlo Study Of The Accuracy Of The Scree Plot, Gibbs Y. Kanyongo

Journal of Modern Applied Statistical Methods

This article pertains to the accuracy of the of the scree plot in determining the correct number of components to retain under different conditions of sample size, component loading and variable-tocomponent ratio. The study employs use of Monte Carlo simulations in which the population parameters were manipulated, and data were generated, and then the scree plot applied to the generated scores.

Depth Based Permutation Test For General Differences In Two Multivariate Populations, Yonghong Gao

Journal of Modern Applied Statistical Methods

For two p-dimensional data sets, interest exists in testing if they come from the common population distribution. Proposed is a practical, effective and easy to implement procedure for the testing problem. The proposed procedure is a permutation test based on the concept of the depth of one observation relative to some population distribution. The proposed test is demonstrated to be consistent. A small Monte Carlo simulation was conducted to evaluate the power of the proposed test. The proposed test is applied to some numerical examples.

Teaching Random Assignment: Do You Believe It Works?, Shlomo S. Sawilowsky

Journal of Modern Applied Statistical Methods

Textbook authors admonish students to check on the comparability of two randomly assigned groups by conducting statistical tests on pretest means to determine if randomization worked. A Monte Carlo study was conducted on a sample of n = 2 per group, where each participant’s personality profile was represented by 7,500 randomly selected and assigned scores. Independent samples t tests were conducted and the results demonstrated that random assignment was successful in equating the two groups on 7,467 variables. The students’ focus is redirected from the ability of random assignment to create comparable groups to the testing of the claims of …

You Think You’Ve Got Trivials?, Shlomo S. Sawilowsky

Journal of Modern Applied Statistical Methods

Effect sizes are important for power analysis and meta-analysis. This has led to a debate on reporting effect sizes for studies that are not statistically significant. Contrary and supportive evidence has been offered on the basis of Monte Carlo methods. In this article, clarifications are given regarding what should be simulated to determine the possible effects of piecemeal publishing trivial effect sizes.

Fast Permutation Tests That Maximize Power Under Conventional Monte Carlo Sampling For Pairwise And Multiple Comparisons, J. D. Opdyke

Journal of Modern Applied Statistical Methods

While the distribution-free nature of permutation tests makes them the most appropriate method for hypothesis testing under a wide range of conditions, their computational demands can be runtime prohibitive, especially if samples are not very small and/or many tests must be conducted (e.g. all pairwise comparisons). This paper presents statistical code that performs continuous-data permutation tests under such conditions very quickly – often more than an order of magnitude faster than widely available commercial alternatives when many tests must be performed and some of the sample pairs contain a large sample. Also presented is an efficient method for obtaining a …