Open Access. Powered by Scholars. Published by Universities.®

Education Commons

Open Access. Powered by Scholars. Published by Universities.®

Educational Assessment, Evaluation, and Research

University of Massachusetts Amherst

Keyword
Publication Year
Publication
Publication Type
File Type

Articles 1 - 30 of 129

Full-Text Articles in Education

A Top-Down Approach For Optimally Designing Multistage-Adaptive Tests, Hwanggyu Lim Jan 2019

A Top-Down Approach For Optimally Designing Multistage-Adaptive Tests, Hwanggyu Lim

Doctoral Dissertations

In multistage-adaptive testing (MST), there are many interrelated design variables that impact the nature and quality of ability estimation. Previous research has identified general principles for the effective design of MSTs in terms of measurement performance. However, those principles are unlikely to apply uniformly to every testing context.

The purpose of this dissertation is to propose a process of finding an MST design that has optimal measurement properties, given a specific set of test circumstances. To achieve this goal, an efficient strategy was introduced at each of three phases to discover the optimal design of the MST; constructing MSTs, systematically ...


Urban Biodiversity Experience And Exposure: Intervention And Inequality At The Local And Global Scale, Evan Kuras Jan 2019

Urban Biodiversity Experience And Exposure: Intervention And Inequality At The Local And Global Scale, Evan Kuras

Masters Theses

As cities expand globally, researchers must clarify how human activities and institutions shape biodiversity and conversely, how ecological processes shape human outcomes. Two features of contemporary cities motivate this thesis. First, urban residents, and especially children, are spending less time in nature and consequently, miss out on healthy and formative experiences with biodiversity. Second, residents with the least access to biodiversity tend to be those with the lowest socioeconomic status (SES). Together, these patterns convey a multi-layered environmental injustice: not only might urbanites become increasingly estranged from biodiversity, disinterested from its conservation, and disconnected from its benefits, but these outcomes ...


Examining The Effects Of Changes In Automated Rater Bias And Variability On Test Equating Solutions, Michelle Boyer Jan 2018

Examining The Effects Of Changes In Automated Rater Bias And Variability On Test Equating Solutions, Michelle Boyer

Doctoral Dissertations

Many studies have examined the quality of automated raters, but none have focused on the potential effects of systematic rater error on the psychometric properties of test scores. This simulation study examines the comparability of test scores under multiple rater bias and variability conditions, and addresses questions of their effects on test equating solutions. Effects are characterized by a comparison of equated and observed raw scores and estimates of examinee ability across the bias and variability scenarios. Findings suggest that the presence of, and changes in, rater bias and variability affect the equivalence of total raw scores, particularly at higher ...


Investigation Repeater Effects On Small-Sample Equating: Include Or Exclude?, Hongyu Diao Jan 2018

Investigation Repeater Effects On Small-Sample Equating: Include Or Exclude?, Hongyu Diao

Doctoral Dissertations

In licensure testing programs, some examinees might attempt the test multiple times till they are satisfied with their final score, those who take the same test repeatedly are referred to as repeaters. Previous studies suggested that repeaters should be removed from the total sample before implementing equating procedures for two reasons: 1) repeater group is distinguishable from the non-repeater group and the total group, 2) repeaters may memorize anchor items and cause an item drift in common items in the non-equivalent anchor test (NEAT) design. However, removing repeaters might not be the best solution if the testing program only has ...


Using Strategic Discourse For Building Understanding In Elementary Mathematics: What Do Teachers And Students Think?, Mary Coakley Jan 2018

Using Strategic Discourse For Building Understanding In Elementary Mathematics: What Do Teachers And Students Think?, Mary Coakley

Doctoral Dissertations

The mathematics reform movement has not had a significant or lasting impact on the practice of teachers and learning of students throughout the country (Boylan, 2010, Kazemi & Stipek, 2001). Students are not developing the types of skills critical thinking skills needed to solve problems in mathematics. Research suggests a need for structural changes that include providing opportunities for students to develop more autonomy and authority in the mathematics classroom (Cuban, 2013). To meet these challenges, teachers and students must make significant changes to implement instruction that fulfills this demand. This expectation has left teachers struggling to determine essential changes and ...


Teacher Education And Professional Development On Classroom Assessment In Bangladesh: Exploring Policy And Practice Through A Vertical Case Study, Sumera Ahsan Jan 2018

Teacher Education And Professional Development On Classroom Assessment In Bangladesh: Exploring Policy And Practice Through A Vertical Case Study, Sumera Ahsan

Doctoral Dissertations

Formative Classroom Assessment (FCA) can be the single most powerful activity to promote students’ learning (Hattie, 2009). In Bangladesh FCA is not in practice (Ahsan, 2009). Policies and teacher preparation on FCA are the two factors that influence the practice of formative assessment in classrooms (Stiggins, 1999; Plake, 1993). In my research I aimed to learn how different actors, discourses, and materials come together to produce policies on FCA in policy network and translate the policies in a Teachers’ Training College (TTC), and in classrooms in an urban school.

I used ‘critical socio-cultural approach to policy as practice’ (Levinson, Sutton ...


Using Exemplar Items To Define Performance Categories: A Comparison Of Item Mapping Methods, Ana Karantonis Jan 2017

Using Exemplar Items To Define Performance Categories: A Comparison Of Item Mapping Methods, Ana Karantonis

Doctoral Dissertations

Score reporting is an extremely important and yet often neglected component of large-scale assessment programs. One element of score reporting that frequently leads to misunderstanding is the interpretation of performance levels. One way to help define performance levels is through the use of "exemplars." Exemplars are test items that are supposed to best characterize each performance level. In this study, a Monte Carlo simulation was conducted to examine the performance of two item-mapping methods and different criteria for identifying exemplars under several simulated conditions.

The results of the study were neither clear nor systematic across all conditions and performance levels ...


Social-Emotional Development Assessment: Scale Development For Kindergarten Through Second Grade Youth Universal Screening, James F.M. Brenchley Jan 2017

Social-Emotional Development Assessment: Scale Development For Kindergarten Through Second Grade Youth Universal Screening, James F.M. Brenchley

Doctoral Dissertations

The goal of this project was to address a significant gap in the research literature with regard to available screening tools that allow young children to self-identify needs related to their social-emotional development. A review of current evidence-based social-emotional tools led to the identification of seven domains most frequently utilized: self-regulation, emotional regulation, social skills, self-concept, school connectedness, social responsibility, and optimism/positivity. To accomplish this endeavor, two studies were conducted to develop a screening measure that demonstrated adequate psychometric properties, but also minimized cost related to time for implementation. The first study was a review of 105 pilot scale ...


Investigating The Impact Of Student Opt Out On Value-Added Measures Of Teacher Quality, Joshua Marland Jan 2017

Investigating The Impact Of Student Opt Out On Value-Added Measures Of Teacher Quality, Joshua Marland

Doctoral Dissertations

Student assessment nonparticipation (or opt out) has increased substantially in K-12 schools in states across the country. This increase in opt out has the potential to impact achievement and growth measures used for educator and institutional accountability. This simulation study investigates the extent to which value-added measures of teacher quality are impacted as a result of varying degrees of opt out, as well as various types of nonrandom opt out. Results show that the magnitude of opt out has a greater impact on stability of value-added estimates than the type of nonrandom opt out patterns simulated in this study, with ...


Examining The Capacity Of Instructional Support Networks For The Diffusion Of Computer Science For All (Csforall) In An Urban District, Rebecca Mazur Jan 2017

Examining The Capacity Of Instructional Support Networks For The Diffusion Of Computer Science For All (Csforall) In An Urban District, Rebecca Mazur

Doctoral Dissertations

The purpose of this study was to understand the capacity for diffusion of computer science instruction in an urban school district. Two types of networks, general instructional support and computer science-related support, were described and investigated. Social Network Analysis was used as the primary method to examine the structure of and relationships between the networks. Results suggest that even in schools with dense and distributed instructional support networks, sparse and centralized systems of ties are characteristic of DLCS support networks. Further, an analysis of networks with and without team-supported ties indicates that formal structures for collaborative teaming are critical sources ...


Evaluating A Translingual Administration Of The Early Grades Math Assessment (Egma) In The Democratic Republic Of The Congo, Fernanda Gandara Jan 2017

Evaluating A Translingual Administration Of The Early Grades Math Assessment (Egma) In The Democratic Republic Of The Congo, Fernanda Gandara

Doctoral Dissertations

Translanguaging is a view around languages that normalizes diglossia without separation: the linguistic resources of the bilinguals are considered one integrated system. Translanguaging is also a language practice of bilinguals, who select features from their entire linguistic repertoire to make sense of the world around them. Translanguaging is widely used by students and teachers in the bilingual classroom, as it allows students to build upon their entire set of resources, enhance learning outcomes, perform identities, and develop their languages even further. However, translanguaging is rarely used in assessments of bilinguals. Assessments of bilinguals, especially large-scale tests, are typically monolingual in ...


Promoting Self-Determination Skills Of Individuals With Intellectual Disabilities Participating In The Inclusive Concurrent Enrollment Initiative Programs Throughout Massachusetts, Lyndsey Nunes Jan 2017

Promoting Self-Determination Skills Of Individuals With Intellectual Disabilities Participating In The Inclusive Concurrent Enrollment Initiative Programs Throughout Massachusetts, Lyndsey Nunes

Doctoral Dissertations

There are currently 15 Inclusive Concurrent Enrollment Initiative (ICEI) Programs throughout Massachusetts. Each program is different in its day-to-day planning but all adhere to the mission, full inclusion of students with intellectual and developmental disabilities. The purpose of this evaluative research study was to examine and understand the functional components of postsecondary education programs that promote self-determination for students with intellectual and developmental disabilities and to differentiate programs that include or do not include these components. To answer the five research questions data were collected from Students, Educational Coaches, ICEI Program Coordinators, Transition Liaisons, Statewide Coordinator, observations, and record reviews ...


Who Is Like Whom? Reclassification And Performance Patterns For Different Groupings Of English Learners, Molly M. Faulkner-Bond Jan 2016

Who Is Like Whom? Reclassification And Performance Patterns For Different Groupings Of English Learners, Molly M. Faulkner-Bond

Doctoral Dissertations

Approximately 10 percent of the US K-12 population consists of English learners (ELs), or students who are learning English in addition to academic content in areas like English language arts (ELA) and mathematics. In addition to meeting the same academic content and performance standards set for all students, it is also a goal for ELs to be reclassified – i.e., to master English so that they can shed the EL label and participate in academic settings where English is used without needing special support. Working with a longitudinal cohort of ~28,000 ELs in grades 3 through 8 from one ...


Identifying Examinees Who Possess Distinct And Reliable Subscores When Added Value Is Lacking For The Total Sample, Joseph A. Rios Jan 2016

Identifying Examinees Who Possess Distinct And Reliable Subscores When Added Value Is Lacking For The Total Sample, Joseph A. Rios

Doctoral Dissertations

Research has demonstrated that although subdomain information may provide no added value beyond the total score, in some contexts such information is of utility to particular demographic subgroups (Sinharay & Haberman, 2014). However, it is argued that the utility of reporting subscores for an individual should not be based on one’s manifest characteristics (e.g., gender or ethnicity), but rather on individual needs for diagnostic information, which is driven by multidimensionality in subdomain scores. To improve the validity of diagnostic information, this study proposed the use of Mahalanobis Distance and HT indices to assess whether an individual’s data ...


Subscore Reliability And Classification Consistency: A Comparison Of Five Methods, Fen Fan Jan 2016

Subscore Reliability And Classification Consistency: A Comparison Of Five Methods, Fen Fan

Doctoral Dissertations

Previous studies focused on reporting subscores themselves; however, most proficiency tests are criterion-referenced, reporting classification consistency and accuracy of student placement into performance categories within each subdomain are thus more suitable. Therefore, the primary purpose of this study was to investigate the decision consistency (DC) and decision accuracy (DA) of student placement into performance categories within each of the subdomains measured by the test of interest. A second purpose of this study was to compare the performance of five subscoring methods under some realistic conditions in terms of subscore reliability and classification. To do so, a simulation study was designed ...


Turning Around The Culture Of Teaching And Learning: A Turnaround School Success Story, Rachael B. Lawrence Jan 2016

Turning Around The Culture Of Teaching And Learning: A Turnaround School Success Story, Rachael B. Lawrence

Doctoral Dissertations

In 2009-2010, Riverton Elementary School in Portland, ME was awarded a School Improvement Grant (SIG), supported by the Federal Department of Education (Federal DOE) through the Maine Department of Education. With this funding, Riverton undertook the challenge of the turnaround school model between 2010-2013, which resulted in positive change in academic achievement for their students and an improved teaching and learning climate and culture. This dissertation examines why the turnaround model may work, what is currently known about SIG funded turnaround schools, and what the specific actions and changes that led to the successful turnaround of this school were. While ...


Differential Prediction And Validity Of Advanced Placement (Ap®) For Student Subgroups, Minji K. Lee Jan 2016

Differential Prediction And Validity Of Advanced Placement (Ap®) For Student Subgroups, Minji K. Lee

Doctoral Dissertations

Concerns over fairness permeates every aspect of the testing enterprise, and one characterization of fairness in testing defined by the Standards (AERA, APA, & NCME, 1999) is a fairness as lack of bias. One important way to study bias in college admission context concerns the degree to which prediction equations are equivalent for different groups. To the extent that the AP variables are used together with admission test scores and previous academic records to predict future academic achievement,

it is important to know if members of one group are systematically predicted to obtain lower or higher grades than they actually achieve ...


Detecting Candidate Preknowledge Of Items Using A Predictive Checking Method, Xi Wang Jan 2016

Detecting Candidate Preknowledge Of Items Using A Predictive Checking Method, Xi Wang

Doctoral Dissertations

In on-demand high-stakes testing programs such as GRE and TOEFL, some items are repeatedly used across test administrations to reduce the cost of developing new items constantly. Item exposure provides an opportunity for examinees to have knowledge of particular test items in advance of their administration. It poses a threat to test security and ultimately will result in invalid test scores. Therefore, many testing programs conduct quality control to monitor test compromise at individual and/or group level. A predictive checking method is proposed in this study to detect examinee preknowledge on exposed items. We consider a scenario where a ...


The Development And Validation Of The Emotion Knowledge And Awareness Test, Catherine A. Rossi Jan 2016

The Development And Validation Of The Emotion Knowledge And Awareness Test, Catherine A. Rossi

Doctoral Dissertations

The purpose of this study was to develop, test, and pilot a general outcome measurement tool that will allow educators to test young children’s knowledge of factors of emotional development: emotional identification and fluency, understanding situations where multiple emotions are present, understanding that others may feel differently in situations, and emotional regulation (CASEL, 2014). There are few assessments that reliably measure emotion knowledge in early elementary grades. The Emotion Knowledge and Awareness Test (EKAT) has been developed for kindergarten through second grade students to measure emotion awareness across two domains: knowledge and management. It was developed as a pre ...


Using Social Network Analysis To Investigate The Relationship Between School-Based Team Communication Networks And Implementation Of Positive Behavior Support Systems, Shannon K. Barry Jan 2015

Using Social Network Analysis To Investigate The Relationship Between School-Based Team Communication Networks And Implementation Of Positive Behavior Support Systems, Shannon K. Barry

Doctoral Dissertations

The purpose of this study was to examine the relationship between school-based team communication networks and implementation of school-wide reform efforts and initiatives, namely Positive Behavioral Interventions and Supports (PBIS). The study employed social network analysis (SNA) to determine if a relationship was present between the structure and properties of the team communication network and the level of implementation of PBIS, the position and properties of the PBIS leadership team and the level of implementation of PBIS implementation, and the quality of internal process for collaboration of the PBIS leadership team and PBIS implementation. It was predicted that schools in ...


An Investigation Of Subtest Score Equating Methods Under Classical Test Theory And Item Response Theory Frameworks, Minjeong Shin Jan 2015

An Investigation Of Subtest Score Equating Methods Under Classical Test Theory And Item Response Theory Frameworks, Minjeong Shin

Doctoral Dissertations

Test scores are usually equated only at the total score level. If a test mainly measures a single trait, indicating that the test is essentially unidimensional, equating at the total score level could be the best choice. However, when a test is composed of subtests having negligible relationships among them, separate equating for each subtest offers the best choice. Given a moderate amount of correlations among the subtests, performing individual equating for each subtest may be misleading in that it ignores the relationship of the subtests.

This study applied and compared several possible subtest score equating methods based on classical ...


Evaluating The Impact Of Construct Shift On Item Parameter Invariance, Test Equating And Proficiency Estimates, Xueming Li Jan 2015

Evaluating The Impact Of Construct Shift On Item Parameter Invariance, Test Equating And Proficiency Estimates, Xueming Li

Doctoral Dissertations

Common Core State Standards in English Language Arts and Mathematics at grades K to 12 were introduced in 2009 and at one time had been accepted by 45 of the states in the U.S. The new standards have created national curricula in these two subject areas. Along with this reform, new assessment systems have been developed too. Many of these new tests are showing signs of being more multidimensional than the tests they were replacing because of the use of new item formats, and the assessment of higher level thinking skills and various performance skills. In the short term ...


Translanguaging And Identity In A Kindergarten Classroom: Validating Student's Home Culture And Language In An English-Only Era, Maria Eugenia Lozano Lenis Jan 2015

Translanguaging And Identity In A Kindergarten Classroom: Validating Student's Home Culture And Language In An English-Only Era, Maria Eugenia Lozano Lenis

Doctoral Dissertations

This ethnographic multi-year study examines the effects of federal and state education policies in language-minority school children’s in Western Massachusetts. Specifically, it explores, how, in an increasingly English Only era, a Latina kindergarten teacher resists Massachusetts' restrictive bilingual education law at the same time that she builds on her students’ multi-ethnic identity.

Methodologically, this study combines ethnographic and discourse analysis methods and techniques analyzing the curricular effects that the NCLB and the state of Massachusetts language policy have on an underperforming school serving a predominantly Latino/a population. The focus of the study is the literacy practices enacted by ...


Implementing Universal Social And Emotional Learning Programs: The Development, Validation, And Inferential Findings From The Schoolwide Sel Capacity Assessment, Cheyne A. Levesseur Jan 2015

Implementing Universal Social And Emotional Learning Programs: The Development, Validation, And Inferential Findings From The Schoolwide Sel Capacity Assessment, Cheyne A. Levesseur

Doctoral Dissertations

In order to effectively transport universal social and emotional learning (SEL) programs into natural settings, it is important to understand implementation barriers that may hinder the likelihood of successful outcomes (Fixsen, Naoom, Blasé, Friedman, & Wallace, 2005). The current study is primarily based on the notion that within the planning phase of implementation, few technically adequate assessment measures targeting both organizational capacity (OC) and provider characteristics (PC) for SEL programming actually exist. The purpose is to extend the SEL implementation assessment literature by developing a new rating scale designed to measure SEL implementation barriers (School SEL Capacity Assessment [SSCA]) and make ...


Phallacies: Constructing A Critical Space And Pedagogy For College Men To Engage Across Non-Hegemonic Masculinities, Taj Smith Jan 2015

Phallacies: Constructing A Critical Space And Pedagogy For College Men To Engage Across Non-Hegemonic Masculinities, Taj Smith

Doctoral Dissertations

Peer groups are one of those critical social organizations within our cycles of socialization that assist in regulating culturally acceptable practices of masculinity (Connell, 2005; Gilbert & Gilbert, 1998; Mac an Ghaill, 1994). Men's peer groups are viewed as primary social commitments that should not be broken and maintained through performances of masculinity that are influenced by gendered expectations (Migliaccio, 2009). Unfortunately, the regulation that comes with joining such groups often requires collective and individual oppressive practices that result in some men policing other men's attitudes and behaviors (Flood, 2007).

As a response to Harris and Barone's (2011) call for ...


Impacts Of The Game-Centered Approach On Cognitive Learning Of Game Play And Game Performance During 5-Week Of Spring Season With Intercollegiate Female Soccer Players, Kanae Haneishi May 2014

Impacts Of The Game-Centered Approach On Cognitive Learning Of Game Play And Game Performance During 5-Week Of Spring Season With Intercollegiate Female Soccer Players, Kanae Haneishi

Doctoral Dissertations

Game-centered approaches have been increasingly recognized for their features and the impacts in coaching profession. Research with the game-centered approach is still underdeveloped in coaching sports and physical activities. Therefore, the purpose of this study was to describe the impacts of the game-centered approach on cognitive learning of game play and game performance during 5-week of spring season with intercollegiate female soccer players.

Game performances at beginning, mid, and end of the season were examined through Game Performance Assessment Inventory (GPAI) with seventeen participants. Cognitive learning of game play was also assessed with instant recalls and practice journals with all ...


Ingos In The Mirror: Critical Reflections Of Practitioners Implementing Psychosocial Support Programs, Stephen M. Richardson Jan 2014

Ingos In The Mirror: Critical Reflections Of Practitioners Implementing Psychosocial Support Programs, Stephen M. Richardson

Master's Capstone Projects

The purpose of this qualitative research is to learn from the professional wisdom of practitioners involved in implementing school-based psychosocial support in conflict-affected contexts. Practitioners from four different International Non-governmental Organizations (INGO) working in three different contexts—the Congo Basin region, the South Asia region, and the Sudan region—reflect on the concepts and realities of the psychosocial support models that their organizations use. A common theme emerging from these interviews is that the approach to psychosocial support has the potential to do harm. The practitioners provide real examples of the ways in which harm may occur and their possible ...


Perceptions Of Health Educators And Supervisors About Their Preparation In Alexandria, Egypt (How Well They Believe Their Training And Preparation Prepared Them To Work As Health Educators), Elshaymaa Ahmed Jan 2014

Perceptions Of Health Educators And Supervisors About Their Preparation In Alexandria, Egypt (How Well They Believe Their Training And Preparation Prepared Them To Work As Health Educators), Elshaymaa Ahmed

Master's Capstone Projects

Health educators have many responsibilities, including community education, assessment program development, evaluation, research, health policy and grant writing. Health educators in Egypt do not participate in all these activities, but they mostly do participate in essential activities such as community education. The health educators in Egypt get training and preparation on topics such as addiction, women’s health, chronic diseases, and the skills needed for teaching.

This study investigated the perceptions of health educators and their supervisors about how well they believe their training and preparation has prepared them to work with health clients in Alexandria, Egypt. The study includes ...


Through A Critical Sociocultural Lens: Parents’ Perspectives Of An Early Childhood Program In Guatemala, Yaëlle Stempfelet Jan 2014

Through A Critical Sociocultural Lens: Parents’ Perspectives Of An Early Childhood Program In Guatemala, Yaëlle Stempfelet

Master's Capstone Projects

The present case study is on an Early Childhood program in Guatemala based on participant parents’ feedback. The Early Childhood program is non-formal, focuses on emergent literacy and nutrition, and takes place in a community-run library in a poor, semi-rural town in the mountainous regions of Quiche, Guatemala. The library was set up by a Non-Governmental Organization (NGO) that works in Guatemala as well as another neighboring country.

Using a critical sociocultural lens, this study assumes that the parents’ perceptions reflect the state of the program and that involving their feedback through this research will ultimately help to bolster the ...


Teachers’ Experiences Of Professional Development In (Post)Crisis Katanga Province, Southeastern Democratic Republic Of Congo: A Case Study Of Teacher Learning Circles, Paul St J Frisoli Dec 2013

Teachers’ Experiences Of Professional Development In (Post)Crisis Katanga Province, Southeastern Democratic Republic Of Congo: A Case Study Of Teacher Learning Circles, Paul St J Frisoli

Doctoral Dissertations

Teachers in (post)crisis contexts face an array of de-motivating factors such as insecurity, lack of pay, difficult working conditions, minimal leadership, and feeling undervalued (Johnson, 2006; OECD, 2009; Shriberg, 2007). To bolster their motivation and support their teaching, teachers in these settings need a forum where they feel valued as professionals (Asimeng-Boahene, 2003; Bennel & Akyeampong; Kirk & Winthrop, 2007). One model of teacher professional development (TPD) known as "Teacher Learning Circles"(TLCs) is currently being implemented in Katanga province in southeastern Democratic Republic of Congo (DRC) for early grade reading teachers. TLCs strive to help teachers adopt innovative teaching techniques and create supportive environments. This study examines teachers’ perspectives, using a "crystallized qualitative case study" approach, how these TLCs operate in varied (post)crisis contexts in the DRC where teachers describe their experiences with content, instructional practices and teacher-community support structures of the TLCs (Ellingson, 2008; Rappleye, 2006).

Three major themes arose from the data; Motivating factors for teachers to remain in the profession; the school environment factors that impacted teachers day-to-day; and TLC elements related to technical, social, and emotional collegial assistance. By examining these themes across five schools ...