Education Commons^™

Problems With Null Hypothesis Significance Testing, Jeffrey A. Gliner, George A. Morgan, Nancy L. Leech, Robert J. Harmon

Nancy Leech

In our last column, we discussed inferential statistics and hypothesis testing, both instrumental in the process of null hypothesis significance testing (NHST). Although NHST has been the hallmark of biological, behavioral, and social sciences for most of the past century, objections and concerns have recently increased for two reasons. First, the logic underlying NHST is difficult to understand and appears to be backward. Second, and perhaps more important, NHST is often improperly used in medical and behavioral research.

Teaching Statistics Courses: Some Important Considerations, Anthony J. Onwuegbuzie, Nancy L. Leech

Nancy Leech

Most college students from the behavioral and social sciences are required to enroll in at least one statistics course. Unfortunately, many of these students often attain lower levels of achievement in these courses than in their other classes. Consequently, statistics instructors are faced with the challenge of deciding how to maximize student learning and minimize anxiety and disaffection. Thus, this paper provides a discussion of considerations upon which instructors must reflect in order to address students' needs: context (e.g., undergraduate vs. master's vs. doctoral), content (e.g., measurement vs. evaluation vs. research design), and pedagogical style (e.g., web-based vs. traditional; theory …

The Meaning Of Validity In The New Standards For Educational And Psychological Testing: Implications For Measurement Courses, Laura D. Goodwin, Nancy L. Leech

Nancy Leech

• The treatment of validity in the newest edition of Standards for Educational and Psychological Testing (Standards; American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 1999) is quite different from coverage in earlier editions of the Standards and in most measurement textbooks. The view of validity in the 1999 Standards is discussed, and suggestions for instructors of measurement courses are offered.

The Effects Of Full And Alternative Day Block Scheduling On Language Arts And Science Achievement In A Junior High School, Chance W. Lewis, R. Brian Cobb, Marc Winokur, Nancy L. Leech, Michael Viney, Wendy White

Nancy Leech

The effects of a full (4 X 4) block scheduling program and an alternate day (AB) block scheduling program in a junior high school were under investigation in this study through the use of an ex post facto, matched sampling design. Measures investigated were standardized achievement tests in science and language arts. Both forms of block scheduling had been in place for several years, and one teacher in science and one teacher in language arts had taught students under both forms of scheduling. Because the sampling designs and analyses were different for the science and the language arts areas, two …

Post-Hoc Power: A Concept Whose Time Has Come, Anthony J. Onwuegbuzie, Nancy L. Leech

Nancy Leech

This article advocates the use of post hoc power analyses. First, reasons for the nonuse of a priori power analyses are presented. Next, post hoc power is defined and its utility delineated. Third, a step-by-step guide is provided for conducting post hoc power analyses. Fourth, a heuristic example is provided to illustrate how post hoc power can help to rule in/out rival explanations in the presence of statistically nonsignificant findings. Finally, several methods are outlined that describe how post hoc power analyses can be used to improve the design of independent replications.

The Role Of Sampling In Qualitative Research, Anthony J. Onwuegbuzie, Nancy L. Leech

Nancy Leech

Many qualitative researchers state that sample size and sampling are not issues in qualitative research. However, we argue that making sampling and sample size considerations is central to qualitative research. First, we refute arguments made by qualitative researchers who claim that sampling and sample size considerations are not relevant. Second, we contend that sampling represents a multidimensional construct. Third, we posit that most qualitative studies involve some type of analytical generalization. Thus, choosing a sample size and sampling scheme represent an active process of reflection.

Taking The “Q” Out Of Research: Teaching Research Methodology Courses Without The Divide Between Quantitative And Qualitative Paradigms, Anthony J. Onwuegbuzie, Nancy L. Leech

Nancy Leech

The purpose of this paper is to provide evidence that the debate between quantitative and qualitative is divisive and, hence, counterproductive for advancing the social and behavioral science field. We advocate that all graduate students learn to utilize and to appreciate both quantitative and qualitative research methodologies. As such, students will develop into pragmatist researchers who are able to utilize both quantitative and qualitative techniques when conducting research. We contend that the best way to accomplish this is by eliminating quantitative research methodology and qualitative research methodology courses from curricula and replacing these with research methodology courses at different levels …

On Becoming A Pragmatic Researcher: The Importance Of Combining Quantitative And Qualitative Research Methodologies, Anthony J. Onwuegbuzie, Nancy L. Leech

Nancy Leech

The last 100 years have witnessed a fervent debate in the USA about quantitative and qualitative research paradigms. Unfortunately, this has led to a great divide between quantitative and qualitative researchers, who often view themselves as in competition with each other. Clearly, this polarization has promoted purists, namely, researchers who restrict themselves exclusively either to quantitative or to qualitative research methods. Mono‐method research is the biggest threat to the advancement of the social sciences. Indeed, as long as we stay polarized in research, how can we expect stakeholders who rely on our research findings to take our work seriously? Thus, …

Researching Women's Groups: Findings, Limitations, And Recommendations, Nancy L. Leech, Nathalie L. Kees

Nancy Leech

Although most counselors believe that women's groups are helpful and important, there is little empirical research in this area. The purpose of this article is to report the current state of published research on women's groups. An overview of previous reviews of the literature is discussed, and 26 articles that were located through PsycINFO and ERIC and are analyzed and discussed. Recommendations for future research are included.

Understanding Correlation: Factors That Affect The Size Of R, Laura D. Goodwin, Nancy L. Leech

Nancy Leech

The authors describe and illustrate 6 factors that affect the size of a Pearson correlation: (a) the amount of variability in the data, (b) differences in the shapes of the 2 distributions, (c) lack of linearity, (d) the presence of 1 or more "outliers," (e) characteristics of the sample, and (f) measurement error. Also discussed are ways to determine whether these factors are likely affecting the correlation, as well as ways to estimate the size of the influence or reduce the influence of each.

Sexual Minority Youth Perspectives On The School Environment And Suicide Risk Interventions: A Qualitative Study, Philip A. Rutter, Nancy L. Leech

Nancy Leech

This qualitative study explored the experiences of five gay, lesbian, and bisexual adolescents' perspectives on their schools' acceptance of their sexual orientation, and perceptions of these schools' approach to suicide risk intervention. Focus groups were tape recorded, transcribed, and analyzed through constant comparative analysis. Themes that emerged from the data included participants feeling judged by school counselors and teachers, feeling unsafe at school, and concerned about school staff's response to suicide risk and interventions. Implications for school counselors, teachers and administrators are discussed.

Sampling Designs In Qualitative Research: Making The Sampling Process More Public, Anthony J. Onwuegbuzie, Nancy L. Leech

Nancy Leech

The purpose of this paper is to provide a typology of sampling designs for qualitative researchers. We introduce the following sampling strategies: (a) parallel sampling designs, which represent a body of sampling strategies that facilitate credible comparisons of two or more different subgroups that are extracted from the same levels of study; (b) nested sampling designs, which are sampling strategies that facilitate credible comparisons of two or more members of the same subgroup, wherein one or more members of the subgroup represent a sub-sample of the full sample; and (c) multilevel sampling designs, which represent sampling strategies that facilitate credible …

Toward A Unified Validation Framework In Mixed Methods Research, Amy B. Dellinger, Nancy L. Leech

Nancy Leech

The primary purpose of this article is to further discussions of validity in mixed methods research by introducing a validation framework to guide thinking about validity in this area. To justify the use of this framework, the authors discuss traditional terminology and validity criteria for quantitative and qualitative research, as well as present recently published validity terminology for mixed methods research. The authors discuss the rationale for their framework and how it unifies thinking about validity in mixed methods research. Finally, they discuss how the framework can be used.

Understanding Urban High School Students Of Color Motivation To Teach: Validating The Fit-Choice Scale, Nancy L. Leech, Carolyn A. Haug, Margarita Bianco

Nancy Leech

The Factors Influencing Teaching Choice (FIT-Choice) scale was completed by 86 high school students of color prior to beginning a class focusing on motivating students to become interested in teaching. Findings based on confirmatory factor analysis support the underlying FIT-Choice scale factor structure reported in previous studies. Students’ perceptions about teaching and motivations to teach were significantly lower compared with predominantly White female pre-service and in-service teachers for the majority of factors. The high school students of color were most drawn to teaching by their respect for the proficiency required of a good teacher.

Conclusion: A Four-Phase Model For Teaching And Learning Mixed Research, Anthony J. Onwuegbuzie, Rebecca K. Frels, Kathleen M.T. Collins, Nancy L. Leech

Nancy Leech

In this editorial, we outline a model for teaching a doctoral-level mixed research course either face-to-face or on-line. This model is designed to provide doctoral students with varied opportunities to acquire the knowledge and skills to formulate, to plan, and to implement rigorous and successful mixed research studies and programs of research. We discuss the four overlapping phases of the model: conceptual/theoretical, technical, applied, and emergent scholar phases. Finally, we provide the challenges for instructors using this model via a qualitative inquiry, and challenges and learning benefits of students via a mixed research analysis of reflections of the course. In …

A Qualitative Study Examining Mexican Heritage Students’ Motivation To Succeed In Higher Education, Nate Easley Jr., Margarita Bianco, Nancy L. Leech

Nancy Leech

The disparity between the educational attainment of Mexican heritage and White individuals illustrate a need for research on factors associated with the high educational attainment of some immigrant and first-generation students of Mexican descent. Using autobiographies, student interviews, and family interviews as data sources, this article explores one prominent theme that emerged: Students’ desire to honor parental struggle and sacrifice with academic achievement—Ganas. The implications and recommendations for further research are discussed.

Do Effect-Size Measures Measure Up?: A Brief Assessment, Anthony J. Onwuegbuzie, Joel R. Levin, Nancy L. Leech

Nancy Leech

Because of criticisms leveled at statistical hypothesis testing, some researchers have argued that measures of effect size should replace the significance-testing practice. We contend that although effect-size measures have logical appeal, they are also associated with a number of limitations that may result in problematic interpretations of them in research on children and adults with learning disabilities (LD). The purpose of the present paper is to provide a framework for reporting and interpreting empirical research findings in LD research. Specifically, we recommend that: (1) researchers apply criteria of both statistical significance and substantive significance to help consumers of research assess …

Measures Of Clinical Significance, Helena Chmura Kraemer, George A. Morgan, Nancy L. Leech, Jeffrey A. Gliner, Jerry J. Vaske, Robert J. Harmon

Nancy Leech

Behavioral scientists are interested in answering three basic questions when examining the relationships between variables (Kirk, 2001). First, is an observed result real or should it be attributed to chance (i.e., statistical significance)? Second, if the result is real, how large is it (i.e., effect size)? Third, is the result large enough to be meaningful and useful (i.e., clinical or practical significance)? In this last column in the series, we treat clinical significance as equivalent to practical significance.

Critical Friends' Perspectives On Problems Of Practice And Inquiry In An Edd Program [Book Chapter], Deanna Sands, Connie L. Fulmer, Alan Davis, Shelley Zion, Rodney Blunck, Nancy Leech, Ron Tzur, Maria Araceli Ruiz-Primo

Nancy Leech

About this book:

This volume demonstrates that Critical Friendship Theory can help distinguish education doctorate (EdD) programs from research doctorates (education PhDs). Drawing on multiple, detailed case studies of CFT implementation at universities, it covers curriculum and implementation, online and in-person education, challenges, and strategies for success.

Enhancing The Interpretation Of “Significant” Findings: The Role Of Mixed Methods Research, Anthony J. Onwuegbuzie, Nancy L. Leech

Nancy Leech

The present essay outlines how mixed methods research can be used to enhance the interpretation of significant findings. First, we define what we mean by significance in educational evaluation research. With regard to quantitative-based research, we define the four types of significance: statistical significance, practical significance, clinical significance, and economic significance. With respect to qualitative-based research, we define a significant finding as one that has meaning or representation. Second, we describe limitations of each of these types of significance. Finally, we illustrate how conducting mixed methods analyses can be used to enhance the interpretation of significant findings in both quantitative …

Conducting Mixed Analyses: A General Typology, Anthony J. Onwuegbuzie, John R. Slate, Nancy L. Leech, Kathleen Mt Collins

Nancy Leech

In this article, we provide a typology of mixed analysis techniques, namely the Mixed Analysis Matrix, that helps researchers select a data analysis technique given the number of (a) data types collected (i.e. quantitative or qualitative; or quantitative and qualitative) and (b) analysis types used (i.e. quantitative or qualitative; or quantitative and qualitative)—yielding a 2 X 2 representation involving four cells that each contain specific analytical techniques, with two of these cells containing a total of 15 mixed analysis techniques. Furthermore, we describe the fundamental principle of mixed analysis, describe the steps in a mixed analysis, and delineate the rationale …

Use And Interpretation Of Multiple Regression, Nancy L. Leech, Jeffrey A. Gliner, George A. Morgan, Robert J. Harmon

Nancy Leech

Multiple regression is a frequently used statistical method for analyzing data when there are multiple independent variables. While it can be used in place of analysis of variance, it is most commonly used in the associational approach. For example, in an article from this Journal,Logan and King (2002) were interested in parents’ ability to identify signs of depression in their adolescent children. They hypothesized that prior use of mental health services, impact of the adolescent's emotional/behavioral problems on the family, presence of substance abuse, adolescent–parent communication, and parental depression would be related to a parent's ability to identify depression …

Validity And Qualitative Research: An Oxymoron?, Anthony J. Onwuegbuzie, Nancy L. Leech

Nancy Leech

Although the importance of validity has long been accepted among quantitative researchers, this concept has been an issue of contention among qualitative researchers. Thus, the first purpose of the present paper is to introduce the Qualitative Legitimation Model, which attempts to integrate many of the types of validity identified by qualitative researchers. The second purpose of this article is to describe 24 methods for assessing the truth value of qualitative research. Utilizing and documenting such techniques should prevent validity and qualitative research from being seen as an oxymoron.

Using Group Counseling Techniques To Clarify And Deepen The Focus In Supervision Groups, Nathalie L. Kees, Nancy L. Leech

Nancy Leech

Suggestions for using group counseling techniques for clarifying and deepening the focus within a supervision group are described. Examples are provided from a supervision group of advanced group counseling students. Each student facilitated an ongoing group in settings ranging from schools to residential treatment. Clarifying and deepening rounds and exercises were used in this supervision group to enhance learning and model effective use of group counseling techniques. Considerations for establishing a supervisory group in which these techniques can be used are also discussed.

Change In Classification Level And The Effects On Research Productivity And Merit Scores For Faculty In A School Of Education, Nancy L. Leech, Carolyn A. Haug, Deanna Iceman-Sands, Jennifer Moriarty

Nancy Leech

In this article we present results of an empirical study of the relationship between research productivity and research-related merit ratings over a 10-year period for tenured and tenure-track faculty in a school of education. The purpose of the study was to assess change in faculty productivity as a result of the institution's change in the Carnegie classification system. Results show that higher research expectations affected the productivity level of faculty in regard to increasing international conference presentations, national conference presentations, state conference presentations, and large unfunded grants. Furthermore, over time the variables that were important for merit scores changed and …

Challenges To Teaching Mixed Research Courses, Rebecca K. Frels, Anthony J. Onwuegbuzie, Nancy L. Leech, Kathleen M.T. Collins

Nancy Leech

Across the United States, many faculty members are developing new mixed re-search courses. However, before embarking on teaching these courses, it would be helpful for instructors to be aware of the challenges faced by instructors and stu-dents in mixed research courses. Thus, the purpose of this qualitative-dominant mixed research study was to document these challenges. Participants were 11 in-structors of mixed research courses from institutions around the United States who were selected purposively via critical case sampling such that they represent-ed a diverse set of instructors. The following four themes emerged that represent-ed dimensions of challenges: Time, Diversity, Format/Life Situations, …

Assessment In Statistics Courses: More Than A Tool For Evaluation, Anthony J. Onwuegbuzie, Nancy L. Leech

Nancy Leech

The current assessment reform movement in statistics encourages instructors to think more broadly about cognitive measures which assess student learning. In response, statistics instructors have begun incorporating innovative methods of assessment into their courses, the most common of these procedures being authentic assessment, performance assessment, and portfolio assessment. This paper will discuss areas to consider for assessment, problems with typical assessments, and statistical authenticity for understanding student learning.

A Typology Of Mixed Methods Research Designs [Book Chapter], Nancy L. Leech, Anthony J. Onwuegbuzie

Nancy Leech

About the book: Evaluation is an essential characteristic of the human condition, and perhaps the single most important and sophisticated cognitive process in the repertoire of human reasoning and logic. Evaluation serves society by providing affirmations of worth, value and improvement to name just a few, and is a process which permeates all areas of human activity, scholarship and production. This work is split into four volumes: Volume One: Contains articles featuring contemporary issues and emerging trends in evaluation Volume Two: Contains articles highlighting recent theoretical, methodological, and empirical developments in quantitative evaluation designs Volume Three: Contains a summation of …

A Proposed Fourth Measure Of Significance: The Role Of Economic Significance In Educational Research, Nancy L. Leech, Anthony J. Onwuegbuzie

Nancy Leech

The purpose of this paper is to examine economic significance as a fourth measure of significance. In addition to describing and operationalising the concept of economic significance, a typology of economic significance indices is presented, including an example of how to compute these measures, as well as how to utilise them in applied research. We demonstrate how interventions that yield no statistical, practical or clinical effects may be economically significant. Economic significance is not only relevant in the majority of educational research studies, but also is more readily understood by policy makers and stakeholders than are the other three measures …

A Call For Qualitative Power Analyses, Anthony J. Onwuegbuzie, Nancy L. Leech

Nancy Leech

The purpose of this paper is to emphasize the importance of sampling and sample size considerations in all qualitative research. Such considerations would help qualitative researchers to select sample sizes and sampling designs that are most compatible with their research purposes. First, we discuss the importance of sampling in qualitative research. Next, we outline 24 designs for selecting a sample in qualitative research. We then discuss the importance of selecting a sample size that yields data that have a realistic chance of reaching data saturation, theoretical saturation, or informational redundancy. Based on the literature, we then provide sample size guidelines …