Physical Sciences and Mathematics Commons^™

The Impact Of Prior Exposure To Physics On Students' Transition To Flipped Classrooms With Active Learning, Johnny Gonzalez

Graduate College Dissertations and Theses

The pedagogical approach of the flipped classroom with an active learning model shows great benefit for all students in many disciplines. This study explored the possibility that certain subsets of students may have varying benefits from this model. In this study, we used survey questions to categorize students and analysis of the Forced Concept Inventory (FCI) scores, including normalized gain (NG), to measure learning improvement. In particular, we used a questionnaire with two questions that were used to partition each sub-group. The first group consisted of students that have varying levels of physics knowledge and the second group consisted of …

Effect Sizes And Intra-Cluster Correlation Coefficients Measured From The Green Dot High School Study For Guiding Sample Size Calculations When Designing Future Violence Prevention Cluster Randomized Trials In School Settings, Md. Tofial Azam, Heather M. Bush, Ann L. Coker, Philip M. Westgate

Biostatistics Faculty Publications

Purpose: Cluster randomized controlled trials (cRCTs) are popular in school-based research designs where schools are randomized to different trial arms. To help guide future study planning, we provide information on anticipated effect sizes and intra-cluster correlation coefficients (ICCs), as well as school sizes, for dating violence (DV) and interpersonal violence outcomes based on data from a cRCT which evaluated the bystander-based violence intervention ‘Green Dot’.

Methods: We utilized data from 25 schools from the Green Dot High School study. Effect size and ICC values corresponding to dating and interpersonal violence outcomes are obtained from linear mixed effect models. We …

Generalizability Of Effect Sizes Within Aviation Research: More Samples Are Needed, Rian Mehta, Stephen Rice, Scott Winter, Tyler Spence, Maarten Edwards, Karla Candelaria-Oquendo

International Journal of Aviation, Aeronautics, and Aerospace

It is often the case that researchers attempt to generalize findings from a single convenience sample to the population. They may also wish to make the claim that the sample effect sizes they discover are reasonably similar to the population parameters. The current study attempts to show that they can be mistaken in this assumption, and that different samples can vary dramatically in effect sizes due to myriad discrepancies, such as demographics, sample size, and random error, among other aspects of the samples. Seven hundred and eighty-one participants were recruited from Amazon’s Mechanical Turk, Florida Institute of Technology, Embry-Riddle Aeronautical …

Bi-Dimensional Null Model Analysis Of Presence-Absence Binary Matrices, Giovanni Strona, Werner Ulrich, Nicholas J. Gotelli

College of Arts and Sciences Faculty Publications

Ecology, published by Wiley Periodicals, Inc., on behalf of the Ecological Society of America. Comparing the structure of presence/absence (i.e., binary) matrices with those of randomized counterparts is a common practice in ecology. However, differences in the randomization procedures (null models) can affect the results of the comparisons, leading matrix structural patterns to appear either “random” or not. Subjectivity in the choice of one particular null model over another makes it often advisable to compare the results obtained using several different approaches. Yet, available algorithms to randomize binary matrices differ substantially in respect to the constraints they impose on the …

Errors In A Program For Approximating Confidence Intervals, Andrew V. Frane

Journal of Modern Applied Statistical Methods

An SPSS script previously presented in this journal contained nontrivial flaws. The script should not be used as written. A call is renewed for validation of new software.

Bias And Precision Of The Squared Canonical Correlation Coefficient Under Nonnormal Data Condition, Lesley F. Leach, Robin K. Henson

Journal of Modern Applied Statistical Methods

Monte Carlo methods were employed to investigate the effect of nonnormality on the bias associated with the squared canonical correlation coefficient (R_c²). The majority of R_c² estimates were found to be extremely biased, but the magnitude of bias was impacted little by the degree of nonnormality.

Constructing Confidence Intervals For Effect Sizes In Anova Designs, Li-Ting Chen, Chao-Ying Joanne Peng

Journal of Modern Applied Statistical Methods

A confidence interval for effect sizes provides a range of plausible population effect sizes (ES) that are consistent with data. This article defines an ES as a standardized linear contrast of means. The noncentral method, Bonett’s method, and the bias-corrected and accelerated bootstrap method are illustrated for constructing the confidence interval for such an effect size. Results obtained from the three methods are discussed and interpretations of results are offered.

A Robust Root Mean Square Standardized Effect Size In One-Way Fixed-Effects Anova, Guili Zhang, James Algina

Journal of Modern Applied Statistical Methods

A robust Root Mean Square Standardized Effect Size (RMSSER) was developed to address the unsatisfactory performance of the Root Mean Square Standardized Effect Size. The coverage performances of the confidence intervals (CI) for RMSSE_R were investigated. The coverage probabilities of the non-central F distribution-based CI for RMSSER were adequate.

New Effect Size Rules Of Thumb, Shlomo S. Sawilowsky

Theoretical and Behavioral Foundations of Education Faculty Publications

Recommendations to expand Cohen’s (1988) rules of thumb for interpreting effect sizes are given to include very small, very large, and huge effect sizes. The reasons for the expansion, and implications for designing Monte Carlo studies, are discussed.

New Effect Size Rules Of Thumb, Shlomo S. Sawilowsky

Journal of Modern Applied Statistical Methods

Recommendations to expand Cohen’s (1988) rules of thumb for interpreting effect sizes are given to include very small, very large, and huge effect sizes. The reasons for the expansion, and implications for designing Monte Carlo studies, are discussed.

Estimation Of The Standardized Mean Difference For Repeated Measures Designs, Lindsey J. Wolff Smith, S. Natasha Beretvas

Journal of Modern Applied Statistical Methods

This simulation study modified the repeated measures mean difference effect size, d⁼_RM , for scenarios with unequal pre- and post-test score variances. Relative parameter and SE bias were calculated for dRM ≠ versus dRM = . Results consistently favored d^≠_RM over d⁼_RM with worse positive parameter and negative SE bias identified for d⁼_RM for increasingly heterogeneous variance conditions.

Estimating Explanatory Power In A Simple Regression Model Via Smoothers, Rand R. Wilcox

Journal of Modern Applied Statistical Methods

Consider the regression model Y = γ(X) + ε , where γ(X) is some conditional measure of location associated with Y , given X. Let Υ̂ be some estimate of Y, given X, and let τ² (Y) be some measure of variation. Explanatory power is η² = τ² (Υ̂) /τ²(Y) . When γ(X) = β₀ + β₁X and τ²(Y) is the variance of Y , η² = ρ² , …

Confidence Intervals For The Squared Multiple Semipartial Correlation Coefficient, James Algina, H. J. Keselman, Randall D. Penfield

Journal of Modern Applied Statistical Methods

The squared multiple semipartial correlation coefficient is the increase in the squared multiple correlation coefficient that occurs when two or more predictors are added to a multiple regression model. Coverage probability was investigated for two variations of each of three methods for setting confidence intervals for the population squared multiple semipartial correlation coefficient. Results indicated that the procedure that provides coverage probability in the [.925, .975] interval for a 95% confidence interval depends primarily on the number of added predictors. Guidelines for selecting a procedure are presented.

Coverage Performance Of The Non-Central F-Based And Percentile Bootstrap Confidence Intervals For Root Mean Square Standardized Effect Size In One-Way Fixed-Effects Anova, Guili Zhang, James Algina

Journal of Modern Applied Statistical Methods

The coverage performance of the confidence intervals (CIs) for the Root Mean Square Standardized Effect Size (RMSSE) was investigated in a balanced, one-way, fixed-effects, between-subjects ANOVA design. The noncentral F distribution-based and the percentile bootstrap CI construction methods were compared. The results indicated that the coverage probabilities of the CIs for RMSSE were not adequate.

Significance Tests Harm Progress In Forecasting, J. Scott Armstrong

J. Scott Armstrong

Based on a summary of prior literature, I conclude that tests of statistical significance harm scientific progress. Efforts to find exceptions to this conclusion have, to date, turned up none. Even when done correctly, significance tests are dangerous. I show that summaries of scientific research do not require tests of statistical significance. I illustrate the dangers of significance tests by examining an application to the M3-Competition. Although the authors of that reanalysis conducted a proper series of statistical tests, they suggest that the original M3 was not justified in concluding that combined forecasts reduce errors and that the selection of …

Reliability And Statistical Power: How Measurement Fallibility Affects Power And Required Sample Sizes For Several Parametric And Nonparametric Statistics, Gibbs Y. Kanyongo, Gordon P. Brook, Lydia Kyei-Blankson, Gulsah Gocmen

Journal of Modern Applied Statistical Methods

The relationship between reliability and statistical power is considered, and tables that account for reduced reliability are presented. A series of Monte Carlo experiments were conducted to determine the effect of changes in reliability on parametric and nonparametric statistical methods, including the paired samples dependent t test, pooled-variance independent t test, one-way analysis of variance with three levels, Wilcoxon signed-rank test for paired samples, and Mann-Whitney-Wilcoxon test for independent groups. Power tables were created that illustrate the reduction in statistical power from decreased reliability for given sample sizes. Sample size tables were created to provide the approximate sample sizes required …

Confidence Intervals For An Effect Size When Variances Are Not Equal, James Algina, H. J. Keselman, Randall D. Penfield

Journal of Modern Applied Statistical Methods

Confidence intervals must be robust in having nominal and actual probability coverage in close agreement. This article examined two ways of computing an effect size in a two-group problem: (a) the classic approach which divides the mean difference by a single standard deviation and (b) a variant of a method which replaces least squares values with robust trimmed means and a Winsorized variance. Confidence intervals were determined with theoretical and bootstrap critical values. Only the method that used robust estimators and a bootstrap critical value provided generally accurate probability coverage under conditions of nonnormality and variance heterogeneity in balanced as …

Reliability, Effect Size, And Responsiveness And Intraclass Correlation Of Health Status Measures Used In Randomized And Cluster-Randomized Trials, Paula Diehr, Lu Chen, Donald L. Patrick, Ziding Feng, Yutaka Yasui

UW Biostatistics Working Paper Series

Background: New health status instruments are described by psychometric properties, such as Reliability, Effect Size, and Responsiveness. For cluster-randomized trials, another important statistic is the Intraclass Correlation for the instrument within clusters. Studies using better instruments can be performed with smaller sample sizes, but better instruments may be more expensive in terms of dollars, lost opportunities, or poorer data quality due to the response burden of longer instruments. Investigators often need to estimate the psychometric properties of a new instrument, or of an established instrument in a new setting. Optimal sample sizes for estimating these properties have not been studied …

Jmasm19: A Spss Matrix For Determining Effect Sizes From Three Categories: R And Functions Of R, Differences Between Proportions, And Standardized Differences Between Means, David A. Walker

Journal of Modern Applied Statistical Methods

The program is intended to provide editors, manuscript reviewers, students, and researchers with an SPSS matrix to determine an array of effect sizes not reported or the correctness of those reported, such as rrelated indices, r-related squared indices, and measures of association, when the only data provided in the manuscript or article are the n, M, and SD (and sometimes proportions and t and F (1) values) for twogroup designs. This program can create an internal matrix table to assist researchers in determining the size of an effect for commonly utilized r-related, mean difference, and difference in proportions indices when …

Bias Affiliated With Two Variants Of Cohen’S D When Determining U1 As A Measure Of The Percent Of Non-Overlap, David A. Walker

Journal of Modern Applied Statistical Methods

Variants of Cohen’s d, in this instance dt and d_{adj, has the largest influence on U1 measures used with smaller sample sizes, specifically when n1 and n2 = 10. This study indicated that bias for variants of d, which influence U1 measures, tends to subside and become more manageable, in terms of precision of estimation, around 1% to 2% when n1 and n2 = 20. Thus, depending on the direction of the influence, both dt and dadj are likely to manage bias in the U1 measure quite well for smaller to …}

Deconstructing Arguments From The Case Against Hypothesis Testing, Shlomo S. Sawilowsky

Theoretical and Behavioral Foundations of Education Faculty Publications

The main purpose of this article is to contest the propositions that (1) hypothesis tests should be abandoned in favor of confidence intervals, and (2) science has not benefited from hypothesis testing. The minor purpose is to propose (1) descriptive statistics, graphics, and effect sizes do not obviate the need for hypothesis testing, (2) significance testing (reporting p values and leaving it to the reader to determine significance) is subjective and outside the realm of the scientific method, and (3) Bayesian and qualitative methods should be used for Bayesian and qualitative research studies, respectively.

A Comparison Of Equivalence Testing In Combination With Hypothesis Testing And Effect Sizes, Christopher J. Mecklin

Journal of Modern Applied Statistical Methods

Equivalence testing, an alternative to testing for statistical significance, is little used in educational research. Equivalence testing is useful in situations where the researcher wishes to show that two means are not significantly different. A simulation study assessed the relationships between effect size, sample size, statistical significance, and statistical equivalence.

Deconstructing Arguments From The Case Against Hypothesis Testing, Shlomo S. Sawilowsky

Journal of Modern Applied Statistical Methods

The main purpose of this article is to contest the propositions that (1) hypothesis tests should be abandoned in favor of confidence intervals, and (2) science has not benefited from hypothesis testing. The minor purpose is to propose (1) descriptive statistics, graphics, and effect sizes do not obviate the need for hypothesis testing, (2) significance testing (reporting p values and leaving it to the reader to determine significance) is subjective and outside the realm of the scientific method, and (3) Bayesian and qualitative methods should be used for Bayesian and qualitative research studies, respectively.

Jmasm9: Converting Kendall’S Tau For Correlational Or Meta-Analytic Analyses, David A. Walker

Journal of Modern Applied Statistical Methods

Expanding on past research, this study provides researchers with a detailed table for use in meta-analytic applications when engaged in assorted examinations of various r-related statistics, such as Kendall’s tau (τ) and Cohen’s d, that estimate the magnitude of experimental or observational effect. A program to convert from the lesser-used tau coefficient to other effect size indices when conducting correlational or meta-analytic analyses is presented.

Trivials: The Birth, Sale, And Final Production Of Meta-Analysis, Shlomo S. Sawilowsky

Theoretical and Behavioral Foundations of Education Faculty Publications

The structure of the first invited debate in JMASM is to present a target article (Sawilowsky, 2003), provide an opportunity for a response (Roberts & Henson, 2003), and to follow with independent comments from noted scholars in the field (Knapp, 2003; Levin & Robinson, 2003). In this rejoinder, I provide a correction and a clarification in an effort to bring some closure to the debate. The intension, however, is not to rehash previously made points, even where I disagree with the response of Roberts & Henson (2003).

Not All Effects Are Created Equal: A Rejoinder To Sawilowsky, J. Kyle Roberts, Robin K. Henson

Journal of Modern Applied Statistical Methods

In the continuing debate over the use and utility of effect sizes, more discussion often helps to both clarify and syncretize methodological views. Here, further defense is given of Roberts & Henson (2002) in terms of measuring bias in Cohen’s d, and a rejoinder to Sawilowsky (2003) is presented.

Trivials: The Birth, Sale, And Final Production Of Meta-Analysis, Shlomo S. Sawilowsky

Journal of Modern Applied Statistical Methods

The structure of the first invited debate in JMASM is to present a target article (Sawilowsky, 2003), provide an opportunity for a response (Roberts & Henson, 2003), and to follow with independent comments from noted scholars in the field (Knapp, 2003; Levin & Robinson, 2003). In this rejoinder, I provide a correction and a clarification in an effort to bring some closure to the debate. The intension, however, is not to rehash previously made points, even where I disagree with the response of Roberts & Henson (2003).

The Trouble With Trivials (P > .05), Shlomo S. Sawilowsky, Jina S. Yoon

Journal of Modern Applied Statistical Methods

Trivials are effect sizes associated with statistically non-significant results. Trivials are like Tribbles in the Star Trek television show. They are cute and loveable. They proliferate without limit. They probably growl at Bayesians. But they are troublesome. This brief report discusses the trouble with trivials.