Education Commons^™

Validation Of The Child And Adult Social Support Scale (Casss) Which Measures Social Support In The Indonesian Version, Endah Mastuti, Fajrianthi, Fitri Andriani

International Conference on Assessment and Learning (ICAL)

Social support has an important role, so that students in online learning can reduce the various problems they face. Social support here comes from parents, teachers, classmates, close friends, and the school itself as an institution. Based on this, it is necessary to do research related to social support to get an overview of the support from whom students need in online learning. Furthermore, this can be used as input to intervene in the problems faced by students. To conduct this research, it is necessary to have a measuring tool to conduct social support research. One of the comprehensive measuring …

An Investigation Into The Relationship Between Teacher Grit, Technology Self-Efficacy, And Technology Integration, Joshua Jeremiah Shiloh Marsh

Theses and Dissertations--Education Sciences

The use of educational technology applications has grown tremendously in the last decade. Instructors are now equipped with hardware and software applications previously unavailable, such as mobile and interactive technologies. These tools can have tremendous impact on students’ learning and teacher practices. Teachers can improve their assessment capabilities through technology integration, provide better learning opportunities for students with learning disabilities, and promote deeper learning practices. Due to these benefits, budgets at the federal, state, and local levels of the United States now have specific allocations regarding technology-related purchases. Nevertheless, barriers remain regarding the effective integration of technologies in public schools. …

Measurement Invariance Across Immigrant And Non-Immigrant Populations On Pisa Cognitive And Non-Cognitive Scales, Maritza Casas

Doctoral Dissertations

International large-scale educational assessments (ILSAs) have played a relevant role in educational policies targeting immigrant students across countries as their results are used by governments as input for decision-making purposes. Given the potential impact that ILSAs can have, the psychometric features of these assessments must be carefully assessed and empirical evidence about the extent to which the inferences made based on test results are valid must be collected. To do so, the first step is to determine if the test results have the same meaning across countries and groups of examinees that is, if the measures are invariant so that …

Rate To Measure Mathematics Teaching: Using The Many-Facet Rasch Modeling To Reevaluate The Mathematics Classroom Observation Protocol For Practices (Mcop2), Chunling Niu

Theses and Dissertations--Education Sciences

Rater-mediated classroom observation protocols are increasingly being used for teaching performance assessments, which makes identifying and controlling for various rater effects a central issue to ensure the rating quality. A series of validation studies under the classical test theory framework, including content validity, interrater reliability, and structure analysis, have been completed for the 16-item Mathematics Classroom Observation Protocol for Practices (MCOP²).

However, the MCOP² data have never been investigated under the Rasch framework. Due to the methodological limitations of the CTT approach for rater-mediated assessments, it is imperative to examine the MCOP² validity and reliability using …

An Evaluation Of The Factor Structure And Internal Consistency Of The ‘Conceptions Of Learning’ And ‘Preferences For Teaching’ Measures In American Occupational Therapy Students, Tore Bonsaksen, Adele Breen-Franklin

Journal of Occupational Therapy Education

When planning to use measurement scales in new samples and contexts, examining the scales’ psychometric properties is an important initial step. This study examined the factor structure and internal consistency of two measures that are part of the Approaches and Study Skills Inventory for Students (ASSIST) – the Conceptions of learning and Preferences for teaching and courses – in a sample of American occupational therapy students. The students (n = 115) completed the measures and provided basic sociodemographic information. Scale structure was examined with Principal Components Analysis (PCA), while consistency between scale items was assessed with mean inter-item correlations. …

Validating Professional Standards For Teachers: A Practical Guide For Research Design: Snapshot Literature Review, Jen Jackson, Yung Nietschke

Dr Jen Jackson

This report aims to establish a strong evidence base for planning a validation study of professional standards for teachers. It presents findings from a rapid "snapshot" review of relevant research literature, to identify previous examples of validation studies, and extract lessons from these about worthwhile methods and considerations in research design. This review was originally conducted to inform the design of a validation study of the draft Myanmar Teacher Competency Standards Framework, but may also have wider relevance to other education systems pursuing similar standards-based reforms. This report presents findings from the literature review. It is divided into four sections: …

Validating Professional Standards For Teachers: A Practical Guide For Research Design: Snapshot Literature Review, Jen Jackson, Yung Nietschke

Teaching standards and teacher evaluation

This report aims to establish a strong evidence base for planning a validation study of professional standards for teachers. It presents findings from a rapid "snapshot" review of relevant research literature, to identify previous examples of validation studies, and extract lessons from these about worthwhile methods and considerations in research design. This review was originally conducted to inform the design of a validation study of the draft Myanmar Teacher Competency Standards Framework, but may also have wider relevance to other education systems pursuing similar standards-based reforms. This report presents findings from the literature review. It is divided into four sections: …

Effects Of Structural Flaws On The Psychometric Properties Of Multiple-Choice Questions, Sarah B. Mcbrien

Department of Teaching, Learning, and Teacher Education: Dissertations, Theses, and Student Research

The sentiment that there is more work to be done than there is time is pervasive among faculty members at most academic institutions. At health science centers, faculty members often balancing teaching responsibilities, clinical loads, and research endeavors. Creative use of educational support staff may provide institutions an avenue for accomplishing goals related to quality improvement, curriculum revision, and accreditation tasks. One such task is the maintenance of a bank of multiple-choice examination items that are free of structural flaws. This study measured the effects of a systematic approach to revising structural flaws in multiple-choice questions on the psychometric properties …

Introduction: History And Conceptual Basis Of Assessment In Higher Education, Peter Ewell, Tammie Cumming

Publications and Research

Assessment and accountability are now inescapable features of the landscape of higher education, and ensuring that these assessments are psychometrically sound has become a high priority for accrediting agencies and therefore also for higher education institutions. Bringing together the higher education assessment literature with the psychometric literature, this book focuses on how to practice sound assessment.

This volume provides comprehensive and detailed descriptions of tools for and approaches to assessing student learning outcomes in higher education. The book is guided by the core purpose of assessment, which is to enable faculty, administrators, and student affairs professionals with the information they …

The Trouble With Test Banks, Harvey Richman, Molly Hrezo

Perspectives In Learning

We compared the psychometrics of quiz questions randomly selected from a test bank with the psychometrics of quiz questions the instructor had selected from the bank for quality and modified (if necessary). On multiple psychometric indices, the instructor selected/modified questions were superior to questions randomly selected from the test bank. Most notably, when compared with instructor written/modified questions, randomly selected bank questions were nearly 6.5 times more likely to contain a distractor that drew more responses than the correct answer. Details and implications are discussed.

Reliability And Validity Of Michigan School Libraries For The 21st Century Measurement Benchmarks, Natosha Nicole Floyd

Wayne State University Dissertations

The purpose of this study was to examine the psychometric properties of the Michigan School Libraries for the 21st Century Measurement Benchmarks (SL21). The instrument consists of 19 items with three subscales: Building the 21st Century Learning Environment Subscale, Teaching for 21st Century Learning Subscale, and Leading the Way to 21st Century Learning Subscale. The sample consisted of 54 respondents who were administered the instrument in 2014 and 2015. Cronbach’s alpha for the total instrument was 0.807 (n = 19 items). Exploratory factor analysis (EFA) was used to measure construct validity. The findings derived from the EFA did not tend …

Identifying Examinees Who Possess Distinct And Reliable Subscores When Added Value Is Lacking For The Total Sample, Joseph A. Rios

Doctoral Dissertations

Research has demonstrated that although subdomain information may provide no added value beyond the total score, in some contexts such information is of utility to particular demographic subgroups (Sinharay & Haberman, 2014). However, it is argued that the utility of reporting subscores for an individual should not be based on one’s manifest characteristics (e.g., gender or ethnicity), but rather on individual needs for diagnostic information, which is driven by multidimensionality in subdomain scores. To improve the validity of diagnostic information, this study proposed the use of Mahalanobis Distance and H^T indices to assess whether an individual’s data significantly departs …

Evaluating The Validity Of Technology-Enhanced Educational Assessment Items And Tasks: An Empirical Approach To Studying Item Features And Scoring Rubrics., Ally Thomas

Dissertations, Theses, and Capstone Projects

With the advent of the newly developed Common Core State Standards and the Next Generation Science Standards, innovative assessments, including technology-enhanced items and tasks, will be needed to meet the challenges of developing valid and reliable assessments in a world of computer-based testing. In a recent critique of the next generation assessments in math (i.e., Smarter Balanced), Rasmussen (2015) observed that many aspects of the technology “enhancements” can be expected to do more harm than good as the computer interfaces may introduce construct irrelevant variance. This paper focused on issues surrounding the design of TEIs and how cognitive load …

Factor Analysis Of The Preschool Behavioral And Emotional Rating Scale For Children In Head Start Programs, Cynthia J. Cress, Matthew C. Lambert, Michael Epstein

Department of Special Education and Communication Disorders: Faculty Publications

Strength-based assessment of behaviors in preschool children provides evidence of emotional and behavioral skills in children, rather than focusing primarily on weaknesses identified by deficit-based assessments. The Preschool Behavioral and Emotional Rating Scales (PreBERS) is a normative assessment of emotional and behavioral strengths in preschool children. The PreBERS has well-established reliability and validity for typically developing children as well as children with identified special education needs, but this has not yet been established for children in Head Start programs, who tend to be at high risk for development of emotional and behavioral concerns. This study explores the factorial validity of …

The Estimation Of Polytomous Item Response Models With Many Dimensions, Nikolai Volodin, Ray J. Adams

Professor Ray Adams

Identification conditions and an improved estimation method for a D-dimensional mixed coefficients multinomial logit model are discussed. This model is a generalisation of the Adams and Wilson (1997) random coefficients multinomial logit and it can be used to fit multdimensional forms of a wide range of Rasch measurement models. The computational demands of the numerical integration required in fitting such models have limited previous implementations to three and perhaps four-dimensional problems (Glas, 1992; Adams, Wilson and Wang, 1997). This paper illustrates a Monte Carlo integration method that permits the estimation of models with much higher dimensionality. The example in this …

The Confounding Effects Of Ability, Item Difficulty, And Content Balance Within Multiple Dimensions On The Estimation Of Unidimensional Thetas, Ki Lynn Matlock

Graduate Theses and Dissertations

When test forms that have equal total test difficulty and number of items vary in difficulty and length within sub-content areas, an examinee's estimated score may vary across equivalent forms, depending on how well his or her true ability in each sub-content area aligns with the difficulty of items and number of items within these areas. Estimating ability using unidimensional methods for multidimensional data has been studied for decades, focusing primarily on subgroups of the population based on the estimated ability for a single set of data (Ackerman, 1987a, 1989; Ansley & Forsyth, 1985; Kroopnick, 2010; Reckase, Ackerman, & Spray, …

A Preliminary Investigation Of The Validity Of Time-Based Measures Of Sustained Attention For Children, Michael R. Kulfan

Antioch University Dissertations & Theses

This study is a preliminary investigation of the validity of using time-based measures to quantify sustained attention in children ages 6-12. Problems with sustained attention negatively affect childhood learning and development. The prevalence of disorders known to impact sustained attention performance continue to rise in the United States. Currently, commercially available, objective measures of sustained attention use normative comparisons that provide limited information about the effect such problems have on child performance in natural settings. We reviewed test data from 290 charts of children ages 6-12 referred for neuropsychological evaluation. The Test of Everyday Attention for Children (TEA-Ch) is an …

Ineffective Psychometric Testing: Gre Test Administration, Brittney Dawhn Perry

Masters Theses & Specialist Projects

The effectiveness of the GRE was measured through a mixed-methods study. Quantitative data was studied to determine a relationship between GRE scores and the completion of higher education. Students and employers were surveyed to clarify a link between the content the GRE measures and the skills that are needed in graduate school and the workforce. In addition, students were asked if test administration, time-constrained questions, and question bias had any effect of their GRE score. Together, these findings were inconclusive and do not suggest that the GRE is effective or ineffective in its measurement of potential graduate students in relation …

An Overview Of Psychometric Properties Of The Ausse Student Engagement Questionnaire (Seq), Hamish Coates

Australasian Survey of Student Engagement (AUSSE)

The quality of education is a product of what students do, and how teachers, support professionals and institutions support good educational practice. This means that measuring students’ participation in good educational practices and measuring how institutions support such participation goes to the heart of educational quality. An important link in this line of reasoning is that the instruments used for measurement provide valid, reliable and efficient measurement. This is essential, for otherwise insights into how students engage in education will be biased or diffuse and wrong decisions may be made that have serious implications for policy and practice. To that …

Development And Validation Of The Counterfactual Thinking For Negative Events Scale, Tarika Daftary Kapur, Mark S. Rye, Melissa B. Cahoon, Rahan S. Ali

Department of Justice Studies Faculty Scholarship and Creative Works

We examined the psychometric properties of the newly created Counterfactual Thinking for Negative Events Scale (CTNES) in two studies involving university undergraduates. In Study 1 (N = 634), factor analysis revealed four subscales that correspond with various types of counterfactual thinking: Nonreferent Downward, Other-Referent Upward, Self-Referent Upward, and Nonreferent Upward. The subscales were largely orthogonal and had adequate internal consistency and test–retest reliability. The CTNES subscales were positively correlated with a traditional method of assessing counterfactual thinking and were related as expected to contextual aspects of the negative event, negative affect, and cognitive style. In Study 2 (N …

The Pond You Fish In Determines The Fish You Catch: Exploring Strategies For Qualitative Data Collection, Muninder Kaur Ahluwalia, Lisa A. Suzuki, Agnes Kwong Arora, Jacqueline S. Mattis

Department of Counseling Scholarship and Creative Works

Qualitative research has increased in popularity among social scientists. While substantial attention has been given to various methods of qualitative analysis, there is a need to focus on strategies for collecting diverse forms of qualitative data. In this article, the authors discuss four sources of qualitative data: participant observation, interviews, physical data, and electronic data. Although counseling psychology researchers often use interviewing, participant observation and physical and electronic data are also beneficial ways of collecting qualitative data that have been underutilized.

The Estimation Of Polytomous Item Response Models With Many Dimensions, Nikolai Volodin, Ray J. Adams

Assessment and Reporting

Identification conditions and an improved estimation method for a D-dimensional mixed coefficients multinomial logit model are discussed. This model is a generalisation of the Adams and Wilson (1997) random coefficients multinomial logit and it can be used to fit multdimensional forms of a wide range of Rasch measurement models. The computational demands of the numerical integration required in fitting such models have limited previous implementations to three and perhaps four-dimensional problems (Glas, 1992; Adams, Wilson and Wang, 1997). This paper illustrates a Monte Carlo integration method that permits the estimation of models with much higher dimensionality. The example in …