Educational Assessment, Evaluation, and Research Commons^™

The Effect Of An Overall Rating Item On Halo Error In Performance Evaluations, S. Elizabeth Hogue

Masters Theses & Specialist Projects

This study focuses on how the presence or absence of an overall rating item on a performance evaluation form affects levels of halo error and satisfaction with the form. Participants included undergraduate college students who were randomly assigned to groups receiving a form with or without an overall rating item at the beginning of the form. A satisfaction item was included on both forms. The analyses included a z-test for correlations from independent samples to determine the difference between the two evaluation forms and a t-test to determine the difference between the satisfaction scores of the two forms. The analyses …

A Situational Assessment Of Student Leadership: An Evaluation Of Alternate Forms Reliability And Convergent Validity, Patricia Slack

Masters Theses & Specialist Projects

The Situational Assessment of Leadership: Student Assessment (SALSA©) was developed in the spring of 2009 to be used as a measure of student leadership. Study 1 assessed alternate forms reliability of the SALSA using scores from 178 students. The overall scores on SALSA Form A and SALSA Form B showed a significant correlation (rAB = .906, p < .01). Dimension scores on the two forms ranged from rAB = .475 to rAB = .804. Study 2 evaluated the convergent validity between the SALSA and the Western Kentucky University Center for Leadership Excellence assessment center. SALSA scores as well as assessment scores from 53 students were analyzed. The overall scores on the SALSA and CLE assessment center had a significant yet moderate correlation (r = .513). Dimension correlations were significant but low, ranging from r = .310 to r = .392. The strong correlations in Study 1 indicate the two forms of the SALSA may be used as alternate measures such as in a pre and post-test of leadership. The convergent validities in Study 2 demonstrate that both the SALSA and assessment center may be used to assess leadership. However, the low convergent validities across dimensions indicate overall scores likely should be used rather than dimension scores.

Population Cross-Validity Estimation And Adjustment For Direct Range Restriction: A Monte Carlo Investigation Of Procedural Sequences To Achieve Optimal Cross-Validity, David Matthew Goins

Masters Theses & Specialist Projects

The current study employs Monte Carlo analyses to evaluate the effectiveness of various statistical procedures for determining specific values of interest within a population of 1,000,000 cases. Specifically, the proper procedures for addressing the opposing effects of direct range restriction and validity overestimation were assessed through a comparison of multiple correlation coefficients derived using various sequences of procedures in randomly drawn samples. A comparison of the average bias associated with these methods indicated that correction for range restriction prior to the application of a validity overestimation adjustment formula yielded the best estimate of population parameters over a number of conditions. …

Freshman Transition And Its Effectiveness On Student Success As Measured By Improved Attendance, Improved Grades, Decreased Discipline Referrals, And Decreased Dropout Rate, Jana Lea Thornsberry

Dissertations

Research on freshman transition programs suggests the need for a transition program that assists students as they enter high school. Many freshman transition programs consist of a visit from the high school counselors or a tour of the school. These programs do not help meet the needs of many students who will need constant support throughout the school year. The study school implemented a freshman transition program to help meet the needs of students academically and emotionally. The purpose of this study was to determine if the implementation of a freshman transition program would make a difference in student success …

A Summer Literacy Intervention For Struggling Readers At The Middle School Level Using Strategies In Comprehension, Fluency, Vocabulary, And Positive School Climate To Improve Reading Comprehension: An Evaluation Study, Jeffrey Joseph Haug

Dissertations

A middle school created an intervention called the Summer Literacy Program to increase reading comprehension levels. The middle school believed that by exposing students to fluency, vocabulary, and comprehension strategies in a positive school culture, reading comprehension levels would improve. The program was created due to a high number of students reading below grade level. The purpose of this study is to evaluate the impact of the four-week Summer Literacy Program designed to provide extra interventions for struggling readers. The study is significant because it demonstrated that a four-week summer program can improve reading comprehension levels. The statistical analysis for …

A Comparison Of Limited-Information And Full-Information Methods In Mplus For Estimating Irt Parameters For Non-Normal Populations, Christine E. Demars

Department of Graduate Psychology - Faculty Scholarship

In structural equation modeling software, either limited-information (bivariate proportions) or full-information item parameter estimation routines could be used for the 2PL IRT model. Limited-information methods assume the continuous variable underlying an item response is normally distributed. For skewed and platykurtic latent variable distributions, three methods were compared in Mplus: limited-information, full-information integrating over a normal distribution, and full-information integrating over the known underlying distribution. For the most discriminating easy or difficult items, limited-information estimates of both parameters were considerably biased. Full-information estimates obtained by integrating over a normal distribution were somewhat biased. Full-information estimates obtained by integrating over the true …

Diagnostic Decision-Making: How Much Do Behavior Rating Scales Influence School Psychologists?, Lesley Ann Higgins

Masters Theses & Specialist Projects

Behavior rating scales are commonly used as part of the evaluation process throughout the field of psychology. Behavior rating scales help assess social, emotional, and/or behavioral problems in children, adolescents, and teens. Behavior rating scales indicate the severity of problem behaviors compared to a normative sample. Four scenarios were developed that varied scores on a behavior rating scale and the amount of other information that supported a specific diagnosis. A rating of the likelihood of a diagnosis was requested to see how much influence behavior rating scale scores have on diagnostic decision-making. Each of the four scenarios was sent to …

An Evaluation Of The Convergent Validity Of Multi-Source Feedback With Situational Assessment Of Leadership - Student Assessment (Salsa©), Heather Stroupe

Masters Theses & Specialist Projects

The current study assessed the convergent validity of the Situational Assessment of Leadership – Student Assessment (SALSA©), a situational judgment test (SJT), with multi-source ratings. The SALSA© was administered to ROTC cadets via Blackboard; multi-source ratings, which paralleled the leadership dimensions of the SALSA©, were administered via paper. Each cadet completed the SALSA© and was rated by 10 peers, his/herself, and at least one cadre (superior). SALSA© scores were not correlated with any of the corresponding dimensions on multi-source ratings, with one exception. Cadre ratings of Consideration/Team Skills were positively correlated with SALSA© scores on the same dimension. This finding …

Assessment Matters #10, May 2010, Assessment In Action Committee

Assessment Matters @ WMU

No abstract provided.

Electronic Portfolios Across The Disciplines: Using Technology As An Authentic Assessment Tool, Linda Anstendig, Beth Klingner

Cornerstone 3 Reports : Interdisciplinary Informatics

No abstract provided.

An Adult Accelerated Degree Program: Student And Instructor Perspectives And Factors That Affect Retention, Cindy K. Manjounes

Dissertations

This was a blended qualitative and quantitative study, which investigated adult student and instructor perspectives, motivators, factors and life events that have affected adult students as they pursue their degrees. This study also examined the environment, format, and course content of accelerated degree programs. This study focused solely on undergraduate degree programs. The null hypothesis of this study was that accelerated degree programs for adult students will not result in higher retention rates than traditional day programs. The null hypothesis was rejected. This study is significant because, as pointed out in Stephanie Armour’s June 12, 2003, USA Today article, adults …

Investigating Efforts To Change Educator Attitudes And Teaching Strategies Through Professional Development Focused On The Use Of Backward Design Curriculum And The Principles Of Efficacy: Student Behavior, Feedback And Assessments, Gwendolyn Grooms

Dissertations

School districts continually face problems associated with student achievement. The 2001–2002 federal mandate contained in the No Child Left Behind (NCLB) Act held schools accountable for success of all students in reading and mathematics. Standardized tests were employed to measure the success. The purpose of this study was to investigate efforts to change educator attitudes and teaching strategies through professional development focused on the use of backward design curriculum and the principles of efficacy. Participants completed a pre and post Likert survey. Three 45-minute professional development sessions were conducted incorporating discussions on the principles of efficacy and backward design curriculum. …

Interaction, Internet Self-Efficacy, And Self-Regulated Learning As Predictors Of Student Satisfaction In Distance Education Courses, Yu-Chun Kuo

All Graduate Theses and Dissertations, Spring 1920 to Summer 2023

Online learning research is largely devoted to comparisons of the learning gains between face-to-face and distance students. While student learning is important, comparatively little is known about student satisfaction when engaged in online learning and what contributes to or promotes student satisfaction. Emerging research suggests there are a few strong predictors of student satisfaction, and other predictors that may or may not predict student satisfaction. None of the existing research examines predictors together, or statistically controls for course differences. This study examines the influence of various factors on student satisfaction including three types of interaction, Internet self-efficacy, and self-regulated learning. …

Effects Of At-Home Reading Activities And Parental Involvement On Classroom Reading Scores: Focus On The Middle School Level, Linetta Carter

Dissertations

This dissertation was written collaboratively by Cynthia Warren, Linetta Carter, and George Edwards with the exception of chapter 4 which is the individual effort of the aforementioned researchers. The purpose of this study was to explore the effects of athome reading activities and parental involvement on classroom reading scores with the focus on the middle school level. A total of 150 students from two inner city schools and one suburban school were chosen for the multi-method research. Students’ reading scores were evaluated and their parents were interviewed. A collaborative study involving three investigators was conducted to determine if there was …

Contextual Factors That Influence The Achievement Of Australia’S Indigenous Students: Results From Pisa 2000–2006, Lisa De Bortoli, Sue Thomson

OECD Programme for International Student Assessment (PISA) Australia

Results from international programs that assess the skills and knowledge of young people have indicated that Australia’s Indigenous students perform at a significantly lower level than non- Indigenous students. An in-depth comparison of Indigenous and non-Indigenous students’ performance on the Programme of International Student Assessment (PISA) across three cycles is provided in an accompanying volume to this report, while the current report provides an understanding of how various aspects of students’ background and psychological constructs relate to each other and to student performance. Chapters 2 through 5 each focus on a different group of potential influences on the performance of …

Superintendents' Use Of Research-Based Responsibilities/Practices In High-Performing School Districts, Wayne Russell Stitt

Dissertations

The purpose of this research study was to identify "how" effective superintendents use the research-based leadership responsibilities/practices outlined in Marzano and Waters' (2006) meta-analysis that significantly and positively impact student achievement. A second purpose of this research was to find challenges that superintendents face in their attempts to be successful within their school districts.

The study employed a qualitative multi-site case study using elite purposeful sampling. Five superintendents in southwest Michigan were targeted and agreed to participate in this study. They were the only superintendents in this region who met the criteria, which included school districts who have a total …

Virtual Teacher Talk: Blogging With And By Pre-Service Teachers, Melanie Bishop

Dissertations

This action research study investigated the use of a blog in a pre-service teacher orientation foundational course in a mid-western college. The purpose of using the blog was to integrate face-to-face instruction and virtual interactive technology throughout the course, in particular while the students were away from class for five weeks to participate in a practicum. The face-to-face and virtual interactions were designed to provide mutual support. This study was conducted in a course entitled Orientation to Education, a foundational course required of all undergraduate pre-service education students. Of the 34 students, 24 were freshmen or sophomores. The median age …

The Effects Of Teacher Mentoring On Teacher Retention, Camesha Nichole Hill-Carter

Dissertations

This study was conducted to determine if mentoring plays an important role on new teachers staying in the field of education at the school to which the original position was given. The targeted school is an inner-city charter school in St. Louis, Missouri, where teacher attrition is typically high. Since charter schools do not offer the same three year contracts as traditional public schools, teachers are challenged with the fire at-will clause that is in many charter school contracts. Many new teachers are not secure in the position in which they have attained and often leave the charter school for …

Effects Of At-Home Reading Activities And Parental Involvement On Classroom Reading Scores: Focus On The Elementary School Level, Cynthia Warren

Dissertations

This dissertation was written collaboratively by Cynthia Warren, Linetta Carter, and George Edwards with the exception of chapter 4 which is the individual effort of the aforementioned researchers. The purpose of this study was to explore the effects of athome reading activities and parental involvement on classroom reading scores with a focus on the elementary level. A total of 150 students from two inner city schools and one suburban school were chosen for the multi-method research. Students’ reading scores were evaluated, and their parents were interviewed. A collaborative study involving three investigators was conducted to determine if there was a …

Transformative Pedagogy In Conversation: The Role Of Instructor Interventions In Peer Feedback For Speech Outline Development, Michael J. Stasio

Communication & Leadership Dissertations and Theses

This study investigated the efficacy of written and oral peer feedback used as an instructional strategy to support student learning in speech outline development. One hundred seven students from a California Community College participated in a mixed-methods evaluative quasi-experimental study conducted in a two-fold process with both quantitative and qualitative measures. Existing literature on peer-centered learning has various gaps. These include four areas of uncertainty: implementation interventions, trust, self-perceived benefits, and contexts. Overall perceived learning benefits and four specific instructor intervention variables were measured in this study: (1) creation of a safe space for collaboration, (2) student training on the …