Educational Assessment, Evaluation, and Research Commons^™

Charter School Closing Inequities: Do Automatic Closure Laws Target Black Charter Entrepreneurs And Black Students?, Ian Kingbury, Martha Bradley-Dorsey, Robert Maranto

Education Reform Faculty and Graduate Students Publications

Charter schools can have their charters rescinded if they fail to meet performance metrics, which are often specified in the charter. In some states, however, charters must meet inflexible, standardized performance standards to survive. Through the lens of public choice theory, we hypothesize that charters that were established by African Americans and those which serve more African American students are more likely to close, and that state-imposed standardized closure rules exacerbate these inequities. Analyses using charter petitions (n=925) and National Center for Education Statistics data since 2010 (n=5,548), tend to confirm hypotheses: The percentage of African American students and having …

Understanding Performance In Test Taking: The Role Of Question Difficulty Order, Lina Anaya, Nagore Iriberri, Pedro Rey-Biel, Gema Zamarro

Education Reform Faculty and Graduate Students Publications

Standardized assessments are widely used to determine access to educational resources with important consequences for later economic outcomes in life. However, many design features of the tests themselves may lead to psychological reactions influencing performance. In particular, the level of difficulty of the earlier questions in a test may affect performance in later questions. How should we order test questions according to their level of difficulty such that test performance offers an accurate assessment of the test taker's aptitudes and knowledge? We conduct a field experiment with about 19,000 participants in collaboration with an online teaching platform where we randomly …

Did Spending Cuts During The Great Recession Really Cause Student Outcomes To Decline?, Jessica Goldstein, Josh B. Mcgee

Education Reform Faculty and Graduate Students Publications

Jackson, Wigger, and Xiong (2020a, JWX) provide evidence that education spending reductions following the Great Recession had widespread negative impacts on student achievement and attainment. This paper describes our process of duplicating JWX and highlights a variety of tests we employ to investigate the nature and robustness of the relationship between school spending reductions and student outcomes. Though per-pupil expenditures undoubtedly shifted downward due to the Great Recession, contrary to JWX, our findings indicate there is not a clear and compelling story about the impact of those reductions on student achievement. Moreover, we find that the relationship between K-12 spending …

The Effect Of School District Consolidation On Student Achievement: Evidence From Arkansas, Josh B. Mcgee, Jonathan N. Mills, Jessica Goldstein

Arkansas Education Reports

School district consolidation is one of the most widespread education reforms of the last century, but surprisingly little research has directly investigated its effectiveness. To examine the impact of consolidation on student achievement, this study takes advantage of a policy that requires the consolidation of all Arkansas school districts with enrollment of fewer than 350 students for two consecutive school years. Using a regression discontinuity model, we find that consolidation has either null or small positive impacts on student achievement in math and English Language Arts (ELA). We do not find evidence that consolidation in Arkansas results in positive economies …

Education And Anti-Semitism, Jay. P. Greene, Albert Cheng, Ian Kingsbury

Education Reform Faculty and Graduate Students Publications

Many people believe that intolerance, in general, and anti-Semitism, in particular, are a function of ignorance, and the solution is education. We see evidence of this whenever concerns about intolerance or anti-Semitism become more salient. Proposed solutions frequently feature improved Holocaust education or expanded diversity, equity, and inclusion training. As two religious leaders recently urged, “The only thing that will truly halt the rise of anti-Semitism in America is education” (Stanton & Marcus, 2019). Profiles of anti-Semites tend to feature rural whites or urban minorities, but they are almost always from low-educational backgrounds. Well-educated people tend to feel secure in …

The Effect Of School District Consolidation On Student Achievement: Evidence From Arkansas, Josh B. Mcgee, Jonathan N. Mills, Jessica S. Goldstein

Education Reform Faculty and Graduate Students Publications

School district consolidation is one of the most widespread education reforms of the last century, but surprisingly little research has directly investigated its effectiveness. To examine the impact of consolidation on student achievement, this study takes advantage of a policy that requires the consolidation of all Arkansas school districts with enrollment of fewer than 350 students for two consecutive school years. Using a regression discontinuity model, we find that consolidation has either null or small positive impacts on student achievement in math and English Language Arts (ELA). We do not find evidence that consolidation in Arkansas results in positive economies …

A Snapshot Semester: Students’ Reflections Of Performance And Mental State Throughout A Graduate Course, Bradley D. Borges

Graduate Theses and Dissertations

Graduate students are anecdotally reputed to be among the most stressed groups of young adults. Common sources of stress for graduate students have included coursework, assistantships, finances, career preparation, and family (Wyatt & Oswalt, 2013; Mazzola, Walker, Shockley & Spector, 2011; Oswalt & Riddock, 2007). Guided reflection can be used by educators to assist students in both learning (Boud, Keogh & Walker, 2013; Kolb, 1984) and managing stress (Shoulders, Whitehead, & Morrison, 2018). By understanding how students’ progress through a semester in terms of both their learning and their management of stress, instructors and academic advisors can better anticipate times …

Lowering Language Learner Anxiety: The Impact Of Collaborative-Dynamic Assessment In The Intermediate University Spanish Classroom, Lynda Mcclellan

Graduate Theses and Dissertations

This dissertation presents a new framework for the application of Vygotsky’s Sociocultural Theory (SCT) in the second language (L2) classroom called Collaborative-Dynamic Assessment (C-DA). C-DA is a melding of Collaborative Learning (CL), Dynamic Assessment (DA), and Group Evaluation (GE) in all aspects of L2 instruction and assessment. A study was conducted with 64 university students in an intermediate Spanish classroom to determine if the application of C-DA would lower Foreign Language Classroom Anxiety (FLCA). Horwitz’ (1986) Foreign Language Classroom Anxiety Scale (FLCAS) was administered to the participants at the beginning of the course and then again at the end of …

A Program Evaluation Of The Implementation Process Of A Phonics First® Initiative, Amanda Picard

Graduate Theses and Dissertations

The purpose of this study was to evaluate the implementation of the Phonics First® program within one district to inform district level curriculum team decisions and program planning moving forward. As a level 4 Directed State Support district, the studied district was looking for ways to close the achievement gap and raise overall academic achievement in reading. To do so, the implementation of core reading program initiatives such as Phonics First® was assessed to determine how best to utilize and monitor the Phonics First® program to achieve the district’s desired outcomes. This study utilized a decision-oriented program evaluation using a …

Implementing Change: A Case Study Of Adaptive Leadership, Tina Smith

Graduate Theses and Dissertations

Persistent low levels of student achievement is a concern that has been the focus of many school improvement initiatives. Research has demonstrated that Professional Learning Communities are an initiative that leads to improved academic outcomes. Additionally, research has demonstrated the importance of effective leadership for a change initiative to be successful.

Using the conceptual framework of adaptive leadership as applied to implement a school improvement initiative, this study was conducted to understand how instructional leaders perceive their leadership changed as a result of implementing professional learning communities in their school. The participants were instructional leaders who led schools participating in …

Data And Assessment Management In Collegiate Recreation, Jeana Carow

Graduate Theses and Dissertations

Collegiate recreation programs and centers typically provide traditional programming space in addition to a range of physical activity spaces and resources, as a valuable part of the student experience. The external pressures of identifying and communicating departmental value and impact on the campus community has resulted in collegiate recreation departments’ use of data to communicate the effectiveness and impact of their work. The purpose of the study was to identify the data collection and assessment management practices of collegiate recreation departments, particularly focusing on the organization of data and assessment strategies as well as data collection, storage, reporting, analyzing, and …

Newsroom Convergence In A College Journalism School: Are Students Prepared For A Job?, Travis Feltner

Graduate Theses and Dissertations

This study aims to examine the current state of broadcast journalism education at the college level. It will also ask if college broadcast journalism students are receiving sufficient education and experience in their university newsrooms to get a job after graduation. The study surveyed professional television news directors from the 210 Designated Market Areas (DMA) in the United States. Most respondents agreed to some extent that converged college newsrooms better prepare students for a job as prior research suggested. However, it does not appear that those same applicants would be preferred over ones with traditional broadcast journalism education. The survey …

Baby Boomers In Technology-Rich Environments: Using Piaac To Study The Association Of Workplace Learning With Technology Competency, Julie M. Galliart

Graduate Theses and Dissertations

A skill gap in Problem Solving in Technology-Rich Environments (PS-TRE) between U.S. Baby Boomers and younger generations has been documented in previous studies using the Programme for the International Assessment of Adult Competencies (PIAAC) Survey of Adult Skills (Rampey et al., 2016). Bringing this generation of workers up to speed in this competency area is important because older workers are a growing segment of the U.S. workforce with 13 million employees expected to be age 65 or older by 2024 (Toossi & Torpey, 2017). Workplace learning may be a solution, but few studies in adult learning document outcomes of training …

Effects Of Gatekeeper Course Modality, Age, Gender And High School Gpa On Kaplan Subject Area Exam Scores, Vernon Hoffman

Graduate Theses and Dissertations

The question of equivalency regarding course modality has plagued and intrigued educators for as long as multiple modalities have existed. In the modern world of academia, the two prevailing modalities are face-to-face or traditional courses and online courses. A multitude of factors have contributed to the increase in online course offerings, including increasingly dependable technology and fiscal pressures on institutions of higher learning.

A great deal of scholarly research has compared modalities using within-course measures such as course grades and comprehensive final exams. Most of these studies have found the two modalities equivalent. However, a dearth of research exists which …

Culturally Responsive Science Education And Teaching: Current Literature, Preservice Elementary Science Teachers And Lesson Planning, Lundon Pinneo

Graduate Theses and Dissertations

First, a meta-analysis of Culturally Responsive (CR) science teaching characteristics and practices was conducted. Findings revealed that there are a limited number of CR teaching studies at the elementary science level. When studies conducted at the middle, secondary and postsecondary level were included, six themes which encompassed the characteristics and pedagogical practices of effective CR science teachers and teacher development emerged. These themes were academics, cultural competency, social inequities, CR learning environment, rejection of the deficit lens and pedagogical strategies.

Next, the impact of curriculum and practices designed to reflect these six themes on preservice elementary science teachers was explored. …

Evaluating School Counseling Site Supervisors’ Level Of Preparedness To Supervise, Amanda G. Stuckey

Graduate Theses and Dissertations

The purpose of this study was to investigate school counseling site supervisors’ level of preparedness to provide adequate supervision to school counselors in training and determine if any variables could predict the level of preparedness. Professional School Counselors in the United States (N=86) were asked to complete a survey about how prepared they believed themselves to assist a supervisee in developing each ASCA (2019c) School Counselor Professional Standards and Competencies, as well as their training and experience with the supervisory process and supervision models.Results from this study indicate a clear need for continued training, as well as specialized training relevant …

Elementary Teachers’ Experiences With Remote Learning And Its Impact On Science Instruction: Multiple Cases From The Early Response To The Covid-19 Pandemic, Beth Pesnell

Graduate Theses and Dissertations

Teachers across the United States and worldwide found themselves in unprecedented educational situations during the early response period to the COVID-19 pandemic in spring of 2020. Efforts to quarantine and provide social distancing to prevent the spread of the disease brought mandated school closures throughout the United States, and teachers were required to pivot from their traditional, in-person instructional methods and deliver instruction remotely. This phenomenological multiple-case study explored the experiences of 10 elementary teachers with remote learning during the early response period and sought to understand the delivery of science instruction in the remote learning experience. Weekly interviews were …

Development Of An Effect Size To Classify The Magnitude Of Dif In Dichotomous And Polytomous Items, James D. Weese

Graduate Theses and Dissertations

A standardized effect size for the SIBTEST/POLYSIBTEST procedure is proposed, allowing for Differential Item Functioning (DIF) to be classified with a single set of DIF heuristics regardless of whether data are dichotomous or polytomous. This proposed standardized effect size accounts for both variability in responses and whether participants are included in the SIBTEST/POLYSIBTEST calculations. First, a new set of unstandardized effect size heuristics are established for dichotomous data that are more aligned with Educational Testing Service (ETS) standards using two and three parameter logistic (2PL and 3PL) models. Second, a standardized effect size is proposed and compared to other DIF …

Legislating Social Inclusion: Social Interaction And Perceptions Of Middle Level Students Toward Those With Disabilities, Deidre Necol Whitehead

Graduate Theses and Dissertations

Recognition of the need for students with disabilities to receive free and appropriate education was brought to the forefront with the passage of Public Law 93 – 112, Rehabilitation Act in 1973. In 1975 Public Law 94-142, the Education for All Handicapped Children Act was enacted requiring all schools to develop and provide a free, appropriate public education for all children and youth with disabilities in the regular classroom to ensure an appropriate education. However, mandating inclusion in classrooms does not ensure social interaction, which is an integral part of learning, especially for early adolescents.

This dissertation uses the Chedoke-McMaster …

Culturally Responsive Pedagogy And Writing Achievement For Native High School Students, Angelia Christine Greiner

Graduate Theses and Dissertations

This study focuses on the effects of culturally responsive pedagogy as it is implemented in a high school classroom with Native American students. This mixed methods case study collects, analyzes and synthesizes both quantitative and qualitative data within an overall formative and experimental design approach to measure the effects of culturally responsive pedagogy on students’ writing and their attitudes about writing. The study identified specific approaches to literacy instruction that hold promise for engaging Native students. Furthermore, findings from the study reveal the use of visual art as a particularly powerful tool that extends students’ meaning-making skills, leading to more …

2019 Pulaski County Education Report Card, Sarah Mckenzie

Education Report Card

Our goal is to help parents, school personnel, community members, and policy makers understand how effectively the students in their community are being served.

Pulaski students are demonstrating similar growth in achievement on the ACT Aspire as students in the state overall, but in achievement, graduation rates, and School Quality and Student Success, Pulaski County schools are performing below the state average.

This report card presents information about these key performance indicators for the districts in Pulaski counties. The Growth, Achievement, School Quality, and Overall scores of the four traditional districts are compared to the average for that group, and …

Teachers' Willingness To Pay For Retirement Benefits: A National Stated Preferences Experiment, Dillon Fuchsman, Josh B. Mcgee, Gema Zamarro

Education Reform Faculty and Graduate Students Publications

Many states have recently made or are considering changes to their teacher retirement systems. However, little is known about how teachers value various elements of their retirement benefits versus other aspects of their jobs and compensation. To help alleviate this gap, we use a discrete choice stated preferences experiment embedded in a nationally representative survey of teachers to estimate their willingness-to-pay for various retirement plan characteristics and other non-salary job components. We find that teachers would be indifferent between a traditional pension and alternative retirement plan designs if the alternatives were paired with 2 to 3 percent salary increases. Our …

Normalizing Struggle, Catherine Martin Christopher

Arkansas Law Review

A person who is effective in law school, on the bar exam, and in practice utilizes the same set of skills in each of those scenarios: close, critical reading; synthesis of multiple sources of law into a coherent rule or schema; appreciation of both the big picture and the fine details of a set of rules of law; analysis of a factual scenario for facts that meet or fail a legal test; assessment of the validity and strength of counterarguments; and, of course, clear, concise, thorough, organized communication. Because all these skills are useful from the first day of law …

An Experimental Evaluation Of Arts Field Trips, Heidi H. Erickson, Angela R. Watson, Jay P. Greene

Education Reform Faculty and Graduate Students Publications

This paper presents results of a multi-visit, longitudinal experiment on the academic and social-emotional effects of arts-based field trips. We randomly assign fourth and fifth grade students to receive arts-based field trips throughout the school year or to serve as a control. Treatment students express greater tolerance for people with different opinions and a desire to consume arts. Additionally, treatment students have fewer behavioral infractions, attend school more frequently, score higher on their end-of-grade exams, and receive higher course grades. Effects are strongest when students enter middle school. We find no effect on students’ desire to participate in the arts, …

Computer Science Teacher Survey, Josh B. Mcgee, Sarah C. Mckenzie

Arkansas Education Reports

In April/May of 2020, the University of Arkansas’ Office for Education Policy (OEP), in partnership with Arkansas Governor Asa Hutchinson's Computer Science and Cyber Security Task Force, fielded a survey with the 400+ Arkansas educators who at that time held a computer science endorsement (528), computer science approval code (5016), or computer science technical permit (5014) on their educator’s license. The survey received 153 responses, a nearly 40 percent response rate.

The Role Of Student Effort On Performance In Pisa: Revisiting The Gender Gap In Achievement, Lina Anaya, Gema Zamarro

Education Reform Faculty and Graduate Students Publications

International assessments are important to benchmark the quality of education across countries. However, on low-stakes tests, students’ incentives to invest their maximum effort may be minimal. Research stresses that ignoring students’ effort when interpreting results from low-stakes assessments can lead to biased interpretations of test performance across groups of examinees. We use data from the Programme for International Student Assessment (PISA), a low-stakes test, to analyze the extent to which student effort helps to explain test scores heterogeneity across countries and by gender groups. Our results highlight the importance of accounting for differences in student effort to understand cross-country heterogeneity …

2019 Northwest Arkansas Education Report Card, Sarah C. Mckenzie

Education Report Card

Northwest Arkansas students are demonstrating greater growth in achievement and earning higher scores on the ACT Aspire than are the students in the state overall. Schools in NWA also have higher School Quality and Student Success scores, higher graduation rates, and are more likely to receive an “A” or “B” rating than are other schools across the state. . . This report card presents information about these key performance indicators for the districts in Benton and Washington counties. The Growth, Achievement, School Quality, and Overall scores of each 15 traditional districts are compared to the average scores of the regional …

Evaluating The Impacts Of A Multi-Year Arts Integration Program On Student Outcomes, Molly I. Beck

Graduate Theses and Dissertations

My dissertation evaluates the first year of a three-year intervention, the Windgate School Partnership Program (WSPP), where participating schools participate in three week-long arts integration units taught by resident artists and two educational tours at Crystal Bridges Museum of American Art led by museum educators. The WSPP looks to affect students’ academic and non-academic outcomes through an arts-integration program that includes two key features, artist residencies and field trips. A recent meta-analysis of arts integration studies suggests a moderately positive effect on many student outcomes (Ludwig et al., 2017). I use a mixed-methods research design to measure and describe the …

Increasing Female Enrollment In High School Computer Science Education, Zenovia Brown Frazier

Graduate Theses and Dissertations

“Women have played a vital role in the field of computer science and information technology (IT), developing some of the most essential components of modern IT” (Purdue University Global, 2018). Despite their prominence and achievements in these career fields, computer science has experienced a noticeable decline in the representation of females in industry and in programs of study. This is not a phenomenon reserved for the college classroom and/or the world of work, but is a situation present across schools in the P-12 arena, to include ZBF High School. As such, the purpose of this work was to examine the …

Student-Run Agencies: A Guideline For Agency Survival, Brooke Borgognoni

Graduate Theses and Dissertations

The purpose of this paper is to study 110 student-run agencies (including advertising, public relations, integrated/digital marketing, communications, and graphics and/or design) in the U.S. by analyzing how they currently operate, what practices exist in the agencies for learning, growth, and continuation and how leadership roles of students and advisors influence the agency. This project built upon past studies of student-run agencies and expanded on the topic using the evolutionary theory of a firm by analyzing factors that determine whether it appears that a student-run agency might dissolve or last. This was determined by measuring the faculty advisor involvement level, …