Educational Assessment, Evaluation, and Research Commons^™

Evaluation Of Modern Missing Data Handling Methods For Coefficient Alpha, Katerina Matysova

College of Education and Human Sciences: Dissertations, Theses, and Student Research

When assessing a certain characteristic or trait using a multiple item measure, quality of that measure can be assessed by examining the reliability. To avoid multiple time points, reliability can be represented by internal consistency, which is most commonly calculated using Cronbach’s coefficient alpha. Almost every time human participants are involved in research, there is missing data involved. Missing data means that even though complete data were expected to be collected, some data are missing. Missing data can follow different patterns as well as be the result of different mechanisms. One traditional way to deal with missing data is listwise …

Examining The Effects Of Specifying Bayesian Priors On The Wald's Test For Dif, Paulius Satkus, Christine E. Demars

Department of Graduate Psychology - Faculty Scholarship

No abstract provided.

An Applied Example Of A Two-Tier Multiple-Group Testlet Model, Paulius Satkus, Christine E. Demars

Department of Graduate Psychology - Faculty Scholarship

No abstract provided.

Learning From Incidents To Reduce The Risk Of Drowning In Swimming Pools: Implementation Of Experience-Based Feedback Regarding Near-Misses In Four Public Facilities In France, Élie Vignac, Pascal Lebihain, Bastien Soulé

International Journal of Aquatic Research and Education

The prevention of sports accidents must rely on a detailed knowledge of accident circumstances and risk factors. Today, very few studies have investigated in depth non-fatal drowning incidents that have occurred in public swimming pools (PSP). Learning from incidents seems likely to advance the knowledge of accident scenarios. This research study aimed to capture minor incidents that might identify safety lessons and preventive measures. Incidents of minor and major aquatic events were collected from four PSPs that had hosted 700,000 bathers per year. About 800 incidents and 300 aquatic rescues performed by lifeguards were recorded within a time frame of …

What Can Early Childhood Education And Care Settings Teach Us About Skills For The 21st Century?, Dan Cloney, Kellie Picker

2009 - 2019 ACER Research Conferences

Early childhood education and care (ECEC) settings are naturally oriented towards promoting 21st century skills. This can be seen in Australia, where learning is defined as the development of identity, social and emotional skills, problem-solving, and communication skills. A 21st century orientation is also seen in the playbased pedagogies implemented in ECEC settings. A gap, however, exists in the ability of the ECEC sector to communicate its successes. This gap relates to the lack of measurement tools to quantify the quality of the adult–child interactions in ECEC settings, and children’s growth in these 21 century skills and abilities. This paper …

Assessment In The Interpersonal Domain: Experiences From Empathy Assessment In Medical Education, Neville Chiavaroli

2009 - 2019 ACER Research Conferences

Frameworks for the teaching and assessment of 21st-century skills commonly recognise the importance of learning and skill development in the interpersonal domain. They also usually acknowledge the challenge of reliably and validly assessing students in this domain. In the field of medical education and in selecting students for medical courses, the concept of empathy has become central to representing the particular interpersonal understandings and skills expected of students and practising doctors. Attempts to assess these attributes during medical training are just as challenging as in school contexts. This presentation draws on several years’ experience of working with medical educators to …

Assessing And Understanding Social And Emotional Skills: The Oecd Study On Social And Emotional Skills, Sue Thomson

2009 - 2019 ACER Research Conferences

In an increasingly fast-changing and diverse world, the importance of developing social and emotional skills is becoming more evident. The large body of accumulated evidence shows that these skills have strong relationships with life outcomes and they have been referred to as a key component of 21st century skills. The Organisation for Economic Co-operation and Development (OECD) Study on Social and Emotional Skills is a new international assessment of these skills in students at primary and secondary schools. This study also gathers information on students’ families, schools and community learning contexts, aiming to provide information about the conditions or practices …

Theoretical Considerations For Extracting Meaning From Personal Profile System Data: The Need For Independent Construct Validity Studies, Thomas G. Henkel, James Noel Wilmoth

Tom G. Henkel

The Personal Profile System (PPS) is a psychological testing instrument that has been widely used. The construct validity of the PPS was studied through a review of the literature. This paper organizes the literature review into three broad categories: the background of the PPS; the reliability of the PPS; and the validity of the PPS. The PPS is a self-scoring instrument measuring the behavioral responses of people along four dimensions: (1) dominance; (2) influencing; (3) steadiness; and (4) compliance. The instrument is designed to provide a systematic and comprehensive perception of an individual's behavioral tendencies and the behavioral tendencies of …

Researching Mbti Personality Types: Project Management Master’S Degree Students, Thomas G. Henkel, James W. Marion Jr, Debra T. Bourdeau

Tom G. Henkel

The purpose of this research study was twofold: 1) to explore if a university’s Master of Science in Project Management students’ MBTI® personalities differ significantly; 2) to gain a better understanding if the MBTI® personality traits of university students enrolled in a project management degree differ significantly from those MBTI® personalities of the general population. The goodness of fit test was used in order to test the hypotheses that the 177 graduate project management students (observed data) have the same MBTI® distribution as in the general population (expected data). Overall, the present study showed that the student population has 27.18% …

Factor Analysis Of The Personal Profile System, Thomas G. Henkel, James Noel Wilmoth

Tom G. Henkel

Principal components extraction with orthogonal and oblique rotations tested construct validity for the Personal Profile System. MOST-LIKE endorsements of 96 behavioral descriptors were coded with 4, LEAST-LIKE with 1, and unendorsed with 2.5. Descriptor data from 1,045 senior noncommissioned Air Force officers were normalized. Four factors accounted for 85% of total variance, with 19 descriptors loading significantly on two factors and the remaining 77 on just one factor. The measure of sampling adequacy for every descriptor exceeded .94. One factor for the varimax-rotated (best) analysis was bi-scalar, loading on Steadiness and Compliance descriptors; a second resembled Influencing, a third loaded …

What Can Early Childhood Education And Care Settings Teach Us About Skills For The 21st Century?, Dan Cloney, Kellie Picker

Dr Dan Cloney

Early childhood education and care (ECEC) settings are naturally oriented towards promoting 21st century skills. This can be seen in Australia, where learning is defined as the development of identity, social and emotional skills, problem-solving, and communication skills. A 21st century orientation is also seen in the playbased pedagogies implemented in ECEC settings. A gap, however, exists in the ability of the ECEC sector to communicate its successes. This gap relates to the lack of measurement tools to quantify the quality of the adult–child interactions in ECEC settings, and children’s growth in these 21 century skills and abilities. This paper …

Assessment In The Interpersonal Domain: Experiences From Empathy Assessment In Medical Education, Neville Chiavaroli

Neville Chiavaroli

Frameworks for the teaching and assessment of 21st-century skills commonly recognise the importance of learning and skill development in the interpersonal domain. They also usually acknowledge the challenge of reliably and validly assessing students in this domain. In the field of medical education and in selecting students for medical courses, the concept of empathy has become central to representing the particular interpersonal understandings and skills expected of students and practising doctors. Attempts to assess these attributes during medical training are just as challenging as in school contexts. This presentation draws on several years’ experience of working with medical educators to …

Assessing And Understanding Social And Emotional Skills: The Oecd Study On Social And Emotional Skills, Sue Thomson

Dr Sue Thomson

In an increasingly fast-changing and diverse world, the importance of developing social and emotional skills is becoming more evident. The large body of accumulated evidence shows that these skills have strong relationships with life outcomes and they have been referred to as a key component of 21st century skills. The Organisation for Economic Co-operation and Development (OECD) Study on Social and Emotional Skills is a new international assessment of these skills in students at primary and secondary schools. This study also gathers information on students’ families, schools and community learning contexts, aiming to provide information about the conditions or practices …

Context And Regulation Of Homeschooling: Issues, Evidence, And Assessment Practices, Janet F. Carlson

Buros Center: Professional Staff Publications

The article discusses salient factors that influence the current context within which homeschooling occurs. Individual states have applied various approaches to establish regulations that both preserve the rights of homeschooling parents and fulfill the state’s obligation to ensure that its residents receive the education to which they are constitutionally entitled. Case and ethnographic studies or research involving small and selected samples often appear in outlets associated with homeschool advocacy groups or in outlets that are not mainstream. The paucity of empirical evidence derived from methodologically strong research paradigms has led to little certainty about many aspects of homeschooling including its …

An Analytical Study About The Dialectical Use Of Statistical Significance In Psychological And Educational Research, عايش صباح Dr

International Journal for Research in Education

The statistical significance has been adopted more than 300 years ago, It has served an important purpose in promoting research in social sciences, however, there has been much controversy over the misuse and interpretation of the statistical significance test.Compelling criticisms of statistical significance testing can be found in virtually all areas of the social and life sciences, economics, sociology, education and psychology. Because it is the overwhelmingly dominant statistical method in these sciences, criticisms need to be taken seriously.

Yet, after half a century of cogent arguments against statistical significance and calls to adopt alternative practices some disciplines, such as …

Empowered Learning Systems In Student Success, Craig C. Laupheimer

Education | Master's Theses

Abstract/Description

Although various support systems are attempted by public schools to cause changes in student motivation and academic performance, students continue to consistently under perform and doubt their academic potential. A literature review revealed a growing body of research outlining a direct relationship between internal beliefs and performance outcomes. Yet little has been done to understand student thought patterns from their perspective, or to what extent they are able to recognize and address the internal systems of language, belief, and emotion that interact with learning. By analyzing a wide range of student perspectives, this study examines what tools and strategies …

Opportunities For Self-Evaluation Increase Student Calibration In An Introductory Biology Course, Jennifer L. Osterhage, Ellen Usher, Trisha A. Douin, William M. Bailey

Biology Faculty Publications

Accurate self-evaluation is critical for learning. Calibration describes the relationship between learners’ perception of their performance and their actual performance on a task. Here, we describe two studies aimed at assessing and improving student calibration in a first-semester introductory biology course at a 4-year public institution. Study 1 investigated students’ (n = 310) calibration (the difference between estimated and actual exam performance) across one semester. Students were significantly miscalibrated for the first exam: their predicted scores were, on average, significantly higher than their actual scores. The lowest-performing students had the most inaccurate estimates. Calibration improved with each exam. By …

Test-Retest Reliability And Internal Consistency Of Childhood Measures Of Psychosocial Development In Adolescents With Developmental Disorders, Lauren Zellmer

Dissertations

Introduction: Neurodevelopmental disabilities have been increasing over time, therefore there is an amplified need for psychometrically sound measures of psychosocial well-being to assist in proper diagnosis and treatment planning. This study aimed to evaluate the test-retest reliability and internal consistency of six measures of psychosocial well-being in a group of adolescents diagnosed with developmental disorders. Methods: Thirty-five adolescent students’ ages 14 to 22 with developmental disabilities, 35 parents, and 3 teachers anwered surveys measuring psychosocial well-being. Test-retest reliability was measured using the intraclass correlation coefficient, and internal consistency was measured using Cronbach’s α. Results: The total scores of the Asher …

Developing And Evaluating Staff Training In Responsible Gambling In Search Of A Benchmark, Michelle Beckett, Alex Blaszczynski

International Conference on Gambling & Risk Taking

No abstract provided.

The Influence Of Gambling On The Immigrant And Refugee Community, Tamika Harvey

International Conference on Gambling & Risk Taking

The state of Pennsylvania approved new gambling activities, such as internet gambling and “Play at the Pump” to increase revenue in effort to support special populations. Forming partnerships with community leaders the City of Philadelphia learned how former and new gambling activities has impacted the special population group involving the immigrant and refugee community. The discussion will consist of gambling related activities commonly practiced in their culture and the challenges faced when attempting to access services for problem gambling.

How Internalized And Externalized Behaviors In Adolescents Impact Academic Achievement In Faith-Based Institutions, Subira Brown, Chioma Tait, Jade Callahan, Deyana Cox

Adventist Human-Subject Researchers Association

The purpose of this literature review is to address how internalized and externalized behaviors in adolescents impact academic achievement in faith-based institutions. This research intends to bring awareness to the prevalence of mental health concerns faced by adolescents, as well as the need for mental health services in the education system. This research will also identify the relationship between internalizing and externalizing behaviors, and academic achievement. Based on the findings, the next steps will be to evaluate the deficits within the Adventist school system and begin developing programs and resources necessary to support the mental health needs of children and …

Analyzing Two-Year College Student Success Using Structural Equation Modeling, Jessica Taylor

Graduate Theses, Dissertations, and Capstones

The goal of this study is to more fully understand the scope of community college student success using the principles of mindset, engagement, and college readiness. Using structural equation modeling ensures this study is able to measure the combined effects these concepts have on student success, group differences, and the combined model of student success. Findings suggest student success can be significantly impacted by self-belief and mindset behaviors that can outweigh the initial effect of academically under-prepared students. Groups included in this study are non-traditional students, minority populations, first generation students, and Pell eligible students.

Test Emotions, Value, And Self-Efficacy: A Longitudinal Model Predicting Examinee Effort And Performance On A Low-Stakes Test, Paulius Satkus

Masters Theses, 2010-2019

The validity of scores from low-stakes tests may be compromised by examinee motivation. Expectancy-Value theory (EV) has been used to frame the antecedents of examinee motivation in low-stakes testing contexts. According to EV theory, the perceived value of the test and the expectancy to succeed on the test directly affect examinee effort, which then affects test performance. Cross-sectional research studies in low-stakes testing contexts offer some support of EV theory. Control-Value theory (CV) serves as another theory to understand motivation toward a task. CV theory encompasses the constructs of expectancy and value from EV theory, but incorporates test emotions as …

An Initial Development Of A Hardiness Scale For Elementary School Students, Stephen Ferrara

Educational Specialist, 2009-2019

There are limited studies that have investigated levels of hardiness in children. There is even less information on finding hardiness scales that have been normed on children in elementary school. The purpose of this study was to test the validity and reliability of the Hardiness Scale for Children (HSC), which assesses the three subscales of hardiness: Challenge, Control, and Commitment. 121 elementary school students (2^nd-5^th grade) were selected to complete the HSC. Their parents were also asked to complete a three-item scale to measure their child’s hardiness. The results indicated that older children tended to give themselves …

Effects Of Critical Reflection On University Students’ Cognitive Achievement In Agricultural Mechanization, Patterson P. Hilaire

Graduate Theses and Dissertations

Experiential learning continues to play a vital role in agricultural education and mechanization laboratories (Shoulders & Myers, 2013). An essential component of experiential learning is critical reflection. Critical reflection helps students process and construct meaning from laboratory experiences. A study conducted by Shoulders and Myers (2013) revealed that only 15.4% of the agricultural educators the researchers interviewed incorporated reflective practices laboratory experiences. The purpose of this study was to evaluate the effect of critical reflection on students’ achievement following five agricultural mechanization laboratory activities. The results from this study revealed that critical reflection had a consistently positive effect on immediate …

Beyond Right Or Wrong: The Influences Of Thinking Disposition And Item Difficulty On Student Behavior During High-Stakes Testing, Kristina Lindquist

UNLV Theses, Dissertations, Professional Papers, and Capstones

A hallmark of clinician decision making is the ability to know when to make quick decisions and when decision making should be slowed to account for complicating factors. Throughout the physician training process, multiple choice test items are used to assess student knowledge however, these items do not assess the process used by a student to arrive at the answer choice. If an important characteristic of decisions in clinical practice is timing, then decision timing could be an important consideration for medical school assessments. The purpose of this study, therefore, is to investigate factors that may affect the amount of …

Evaluating The Effectiveness Of Motor Planning With Core Vocabulary: A Behavior Analytic Account, Alison Judith Karnes

Graduate Theses and Dissertations

As the prevalence of autism spectrum disorders (ASD) increases, it is important for practitioners to continue to improve evidence-based practices (EBP) for the treatment of ASD symptoms (i.e., impairments in social communication and repetitive behaviors and restricted interests; American Psychiatric Association [APA], 2013). It is estimated that 30-50% of individuals with autism do not acquire functional speech (Wodka, Mathy, & Kalb, 2013). These individuals would make appropriate candidates for Augmentative and Alternative Communication (AAC; Mirenda, 2003). One form of AAC is the speech-generating device (SGD). Over the last ten years, tablet-based technologies including iPad minisⓇ have been emphasized in the …

Mathematics Attitudes And Mathematics Performance: Novel Approaches Towards Noncognitive Educational Measurement, Applications To Large-Scale Assessment Data, And Examinations Of Multigroup Invariance, Kalina Gjicali

Dissertations, Theses, and Capstone Projects

Academic performance is predicted by a multitude of demographic, contextual, cognitive, and noncognitive constructs. The noncognitive factors of achievement in mathematics that have previously been explored in depth are study skills, collaborative problem-solving, confidence, self-efficacy, and personality traits (Kyllonen, 2012). Limited applied research has explored the predictive value of noncognitive factors such as attitudes and beliefs in mathematics achievement – even though attitudes towards mathematics are a promising avenue for understanding the variability in mathematics achievement. The current research uses the theory of planned behavior (TPB) to explain high school students’ performance in mathematics in a series of three studies. …

Considerations In S-Χ2: Rest Score Or Summed Score, Priors, And Violations Of Normality, Christine E. Demars, Derek Sauder

Department of Graduate Psychology - Faculty Scholarship

The S-χ2 item fit index is one of the few item fit indices that appears to maintain accurate Type I error rates. This study explored grouping examinees by the rest score or summed score, prior distributions for the item parameters, and the shape of the ability distribution. Type I error was slightly closer to the nominal level for the total-score S-χ2 for the longest tests, but power was higher for the rest-score S-χ2 in every condition where power was < 1. Prior distributions reduced the proportion of estimates with extreme standard errors but slightly inflated the Type I error rates in some conditions. When the ability distribution was not normally distributed, integrating over an empirically-estimated distribution yielded Type I error rates closer to the nominal value than integrating over a normal distribution.

Categorization Of Exam Questions To Improve Metacognition, Tiffany Barrett

UNLV Best Teaching Practices Expo

This practice addresses UNLV students’ need for greater metacognitive awareness. Students often employ memorization as a study strategy when preparing for an exam. This can create an inability to apply the knowledge in a complex or different scenario. We know that “practice retrieving” study activities produce greater gains in meaningful learning (Karpicke 2011) but often students don’t identify a need for a change in study habits and a deeper level of understanding, especially if their exam score is considered passing.