Educational Assessment, Evaluation, and Research Commons^™

Comparing And Validating Measures Of Character Skills: Findings From A Nationally Representative Sample, Gema Zamarro, Albert Cheng, M. Danish Shakeel, Collin Hitt

Education Reform Faculty and Graduate Students Publications

Though researchers now are aware of the potential importance of character skills, such as conscientiousness, grit, self-control, and a growth mindset, researchers struggle to find reliable measures of these skills. In this paper, we use data collected from the Understanding America Study, a nationally representative internet panel to study the validity of innovative measures of character skills based on measures of survey effort. We believe surveys themselves can be seen as a behavioral tasks and that respondents provide meaningful information about their character skills by way of the effort they put forward on surveys. In particular, we compare measures of …

Development And Analysis Of A Statics And Kinematics Demonstration As A Learning Tool In The Biomechanics Classroom, Bethany D. Knight

Biomedical Engineering Undergraduate Honors Theses

This study was done with the objective of determining if students in the biomedical engineering department at the University of Arkansas would gain better understanding of basic biomechanics principles through the implementation of in-class demonstrations. Biomechanics was chosen because it is the first class taken in sequence after the “Introduction to Biomedical Engineering” class. A pre-demonstration survey was administered to gauge how comfortable the students were with the topics on the syllabus. Two demonstrations were done in class and related homework was assigned. The post-semester survey was administered and collected to determine how effective the students felt the presentations to …

Measuring Teacher Effectiveness: A Comparison Across Va Models Utilizing Arkansas Data, Kelli Lane Blackford

Graduate Theses and Dissertations

A shift in accountability systems from No Child Left Behind to a reauthorization of the Elementary and Secondary Education Act redefined the mandate of classroom teachers from being highly qualified to highly effective. Whereas previously, a teacher was deemed highly qualified for having a bachelor’s degree, full state certification, and demonstrated knowledge of subject matter in the field they teach, a highly effective teacher had to demonstrate their abilities to move a student at an acceptable rate of student growth (one grade level in an academic year). To provide this evidence, student assessment data was now going to be a …

Perceived Relevance Of Special Education Performance Indicators: Teacher Excellence And Support System, Tiffany Kristin Mrla

Graduate Theses and Dissertations

A study was conducted to investigate the perceived relevance of using a teacher evaluation rubric with performance indicators specific to special education services in place of the standard rubric for teachers used in the State of Arkansas Teacher Excellence Support System (TESS). Through a multi-method approach, the perceptions of special education teachers and administrators regarding implementation of the current model, the significance of perceived differences in pedagogical factors, and potential barriers to effective implementation were identified. This was achieved through survey, interviews, and observations.

Special education teachers and administrators in a given school district were sent a brief survey to …

Character Assessment: Three Essays, Collin E. Hitt

Graduate Theses and Dissertations

I propose a new approach to measuring character skills. In the following three essays, my co-‎authors and I measure the effort that adolescent students appear to put forward on surveys ‎and tests. First, I examine the extent to which students simply skip questions or plead ‎ignorance on surveys. Second, I develop new methods for detecting careless answers, those ‎instances in which students appear to be "just filling in the bubbles." I show, using ‎longitudinal datasets, that both measures are predictive of educational degree attainment, ‎independent of measured cognitive ability and other demographic factors. Finally, I ‎demonstrate that international differences in …

Investigating The Factor Structure Of The Teachers’ Sense Of Efficacy Scale With Pakistani In-Service And Pre-Service Teachers, Sajid Ali Yousuf Zai

Graduate Theses and Dissertations

Teachers with a high sense of efficacy are motivated to achieve and are generally optimistic about future learning. There is an extensive body of research that indicates a teacher’s self-efficacy beliefs can be a performance indicator for school outcomes. Research on characteristics related to teachers in Pakistan has been increasing over the last decade, however there are a number of instruments being used with this population without any documented validation studies. The purpose of this study was to investigate the validity of the Teacher Sense of Efficacy Scale (TSES; Tschannen-Moran & Woolfolk Hoy, 2001) to determine the latent structure of …

Quality Assurance In Online Graduate Education: Program Review Processes And Assessment Techniques Used In Higher Education, Lindsay N. Turner

Graduate Theses and Dissertations

The purpose for conducting the study was to form an understanding of the online program assessment and assessment processes in higher education, and how program review processes were adopted and implemented through institutional policies in regard to industry quality assurance standards. The study looked at institutions classified as very high, high, or research universities by the Carnegie Classifications, and which offered online masters programs focused on preparing students for positions in educational administration or leadership roles. The study used an electronic survey and content analysis through institutional policy and quality assurance documents to develop a recommendation for institutional level quality …

Advanced Placement In Arkansas: Increasing Equity, Sarah C. Mckenzie, Gary W. Ritter

Policy Briefs

Each spring, secondary students across Arkansas sit for Advanced Placement (AP) Exams. Arkansas has one of the most aggressive policies promoting AP in the nation. Despite this, Arkansas students are still struggling with inequitable and inadequate access to AP courses. This brief examines the effects of Arkansas’ AP legislation on equity, adequacy, and productivity in AP policy, particularly for the state’s low-income and minority students.

Measuring Teacher Conscientiousness And Its Impact On Students: Insight From The Measures Of Effective Teaching Longitudinal Database, Albert Cheng, Gema Zamarro

Education Reform Faculty and Graduate Students Publications

Although research has been unable to find strong links between observable teacher characteristics and a teacher’s ability to improve student achievement, it has generally not considered the role that teacher non-cognitive skills play in affecting student outcomes. In this article, we validate several novel performance-task measures of teacher conscientiousness based upon the effort that teachers exert completing a survey and use these measures to examine the role that teacher conscientiousness plays in affecting both student test scores and student non-cognitive skills. We conduct our analysis using the Measure of Effective Teaching Longitudinal Database where teachers were randomly assigned to their …

The Fiscal Effects Of Eliminating The Louisiana Scholarship Program On State Education Expenditures, Julie R. Trivitt, Corey A. Deangelis

Education Reform Faculty and Graduate Students Publications

Eliminating the Louisiana Scholarship Program has been proposed as a way to reduce state education expenditures for the upcoming fiscal year. Drawing upon Louisiana’s education funding formulas, we determine that the overall effect of removing the program will increase state education expenditures. It is true that the state would avoid $41.6 million of spending if the voucher program is eliminated. However, each current voucher student who returns to a public school increases the local district’s necessary education expenditures without increasing the local tax revenue for schools, obligating the state to provide increased funding to the district. While our results depend …

2015 Arkansas Report Card, Sarah Mckenzie, Gary Ritter

Education Report Card

The Office for Education Policy is pleased to bring you its 2015 Arkansas Report Card for highlighting our students’ characteristics and educational performance through the 2014-2015 academic year. his year marked the first significant change in student assessment in more than a decade, the implementation of a letter grade system for grading schools, and new national data from the NAEP. As we head into a new assessment system this spring, we hope the data presented within continue the conversation that surrounds our education system. The following data are intended to provide a snapshot of K-12 education in our state. Although …

The 80 Percent Pension Funding Target, High Assumed Returns, And Generational Inequity, Robert M. Costrell

Education Reform Faculty and Graduate Students Publications

Generational inequity in pension funding is highly sensitive to the lax policies of 80- percent funding targets and high assumed returns to investment. I develop a simple, powerful relationship between steady-state (SS) inequity in contributions – the percent of extra contributions to fund prior cohorts – and the SS unfunded ratio. I then show how the SS unfunded ratio is governed by x-percent funding targets and the gap between assumed and true returns. The SS degree of inequity is over 60 percent under an 80 percent funding target and over 50 percent with a one point gap between assumed and …

Pursuing Innovation How Can Educational Choice Transform K–12 Education In The U.S.?, Patrick Wolf, Anna J. Egalite

Education Reform Faculty and Graduate Students Publications

This report summarizes the state of competition in American K–12 education. It pays particular attention to the prevalence and market penetration of charter schools, private school vouchers, and tax-credit scholarships as market reforms. The effect of added institutional competition from charters, vouchers, and tax-credit scholarships on the performance of district schools and education funding is examined using a survey of the high-quality research on that topic. These summaries and analyses suggest that growing educational competition from charter schools, vouchers, and tax-credit scholarship programs holds the promise of improving the productivity of district schools, subject to the effective design of school …

The School Choice Voucher: A 'Get Out Of Jail' Card?, Corey A. Deangelis, Patrick J. Wolf

Education Reform Faculty and Graduate Students Publications

In this report we examine crime rates for young adults who experienced Milwaukee's citywide voucher program as high school students and a comparable group of their peers who had been public school students. Using unique data collected as part of a longitudinal evaluation of the program, we consider criminal activity by youth initially exposed to voucher schools and those in public schools at the same time. We also consider subsequent criminal activity by the students that stayed in the voucher program through 12th grade compared to those who were in public schools for the same period. We show that the …

The Effects Of The Louisiana Scholarship Program On Student Achievement After Two Years, Jonathan N. Mills

Education Reform Faculty and Graduate Students Publications

The Louisiana Scholarship Program (LSP) is a statewide initiative offering publicly-funded vouchers to enroll in local private schools to students in low-performing schools with family income no greater than 250 percent of the poverty line. Initially established in 2008 as a pilot program in New Orleans, the LSP was expanded statewide in 2012. This paper examines the experimental effects of using an LSP scholarship to enroll in a private school on student achievement in the first two years following the program’s expansion. Our results indicate that the use of an LSP scholarship has negatively impacted both ELA and math achievement, …

The Competitive Effects Of The Louisiana Scholarship Program On Public School Performance, Anna J. Egalite

Education Reform Faculty and Graduate Students Publications

Given the significant growth rate and geographic expansion of private school choice programs over the past two decades, it is important to examine how traditional public schools respond to the sudden injection of competition for students and resources. This article uses 1) a school fixed effects approach, and 2) a regression discontinuity framework to examine the achievement impacts of the Louisiana Scholarship Program (LSP). This targeted school voucher program has provided public funds for low-income students in low-performing public schools to enroll in participating private schools since the 2012-13 school year. The main findings of the competitive effects analysis reveal …

Measures Of Student Non-Cognitive Skills And Political Tolerance After Two Years Of The Louisiana Scholarship Program, Jonathan N. Mills, Albert Cheng, Collin Hitt, Patrick J. Wolf, Jay P. Greene

Education Reform Faculty and Graduate Students Publications

This report examines the short-term effects of the Louisiana Scholarship Program (LSP) on students’ non-cognitive skills and civic values. While a growing number of studies have evaluated K-12 school voucher programs along academic dimensions, few have focused on the development of non-cognitive skills and civic values. This study aims to address that gap by providing the first analysis of differences in self-reported measures of grit, locus of control, self esteem, and political tolerance associated with the LSP. Using results from a phone survey of applicants to the program, we find little evidence of differences between LSP scholarship recipients and non-recipients. …

The Impact Of The Louisiana Scolarship Program On Racial Segregation In Louisiana Schools, Anna J. Egalite, Jonathan N. Mills, Patrick Wolf

Education Reform Faculty and Graduate Students Publications

The question of how school choice programs affect the racial stratification of schools is highly salient in the field of education policy. We use a student-level panel data set to analyze the impacts of the Louisiana Scholarship Program (LSP) on racial segregation in public and private schools. This targeted school voucher program provides funding for low-income, mostly minority students in the lowest-graded public schools to enroll in participating private schools. Our analysis indicates that the vast majority (82%) of LSP transfers have reduced racial segregation in the voucher students’ former public schools. LSP transfers have marginally increased segregation in the …

Quality Counts 2016, Sarah C. Mckenzie, Gary W. Ritter

Policy Briefs

Last week, Education Week released their 20th annual Quality Counts report. This year’s report discusses school accountability and examines state and federal accountability policies. It also grades the nation and each state on academic performance and outcomes. Overall, Arkansas received a C– and was ranked 41st among the 50 states. This policy brief examines Arkansas’ rank in each category of the report as well as the quality of the report itself.

An Exploration Of Performance-Based Funding At Four-Year Public Colleges In The North Central Association Of Colleges And Schools, Samuel Fincher

Graduate Theses and Dissertations

Performance-based funding has been used to help alleviate state and public calls for higher education accountability and more states have adopted this type of funding model (Tandberg & Hillman, 2014; Dougherty, Natow, & Vega, 2012). The purpose of this study was to explore performance-based funding and examine the relationship between types of funding and performance indicators in the North Central Association of Colleges and Schools (NCA). The state funding trends were examined for all of the states in the NCA. The correlations for state funding and four performance outcomes for all four-year higher education institutions were compared for three states …

A Comparison Of Academic Performance And Persistence Of Incoming Freshman Participants In A First-Year Experience Program, David Lee Stevens

Graduate Theses and Dissertations

The purpose of this study was to compare first-time, full-time incoming freshmen from 2008 to 2011 who participated in an extended orientation, first-year experience (FYE) program at the University of Arkansas Fort Smith known as Cub Camp with incoming freshman from the same time period who did not participate. The study was designed to identify possible significant differences in the persistence rates and the academic performance of these two groups.

The study utilized a series of two proportion z-tests and two tailed t-test to compare persistence and academic performance data for the two groups. The two groups were further compared …

Examining Employability As Associated With Ic3, Mos, And Aca Certifications, Kelley Leigh Todd

Graduate Theses and Dissertations

Forrier and Sels (2003) define employability as “an individual’s chance of a job on the internal and/or external labor market” (p. 106) and is important (Wittekind, Raeder, & Grote, 2010). Possessing an industry certification may be considered an example of human capital skill. The human capital theory suggests qualifications, knowledge, skills, and experience of individuals may lead to increased earnings or productivity (Becker, 1993; Rosen, 1987; Schultz, 1971). As such, the human capital theory provides a framework for studying perceived employability (Wittekind, Raeder, & Grote, 2010; Verhaar & Smulders, 1999) as associated with IC3, MOS, and ACA industry certifications.

Randall …

Examining Inequities In Teacher Pension Benefits, James V. Shuls

Education Reform Faculty and Graduate Students Publications

From funding to teacher quality, inequities exist between school districts. This paper adds to the literature on inequities by examining the impact of pension plan formulas on pension benefits. Using data from the salary schedules of 464 Missouri school districts, this paper analyzes how various final average salary calculations would impact the benefits of teachers in different districts. All of the schools in this analysis belong to Missouri’s Public Employee Retirement System, which is a defined-benefit pension plan. A teacher’s benefit in this plan is based on her years of experience and her final average salary. The system uses a …

National Assessment Of Educational Progress (Naep) Results For 2015, Sarah C. Mckenzie, Gary W. Ritter

Policy Briefs

The National Center for Education Statistics has released this year’s NAEP results which measure nationwide student performance in 4th and 8th grade Reading and Math. NAEP is administered nationally to a representative sample of students from all 50 states, so acts as a standard measure of student performance across states and time. This policy brief will examine Arkansas’ 2015 results and consider possible causes and implications.

Education Funding Equity In Arkansas, Sarah C. Mckenzie, Gary W. Ritter

Policy Briefs

Thanks to the landmark Lake View case, Arkansas has doubled-down on its commitment to ensuring an equitable education to all students. This brief examines the equity of current education spending in Arkansas.

Education Funding Adequacy In Arkansas, Sarah C. Mckenzie, Gary W. Ritter

Policy Briefs

Over the past decade and a half, Arkansas has made enormous strides in ensuring that every child in the state is has access to a quality learning experience. This brief examines the adequacy of current education spending in Arkansas

The State Of Education In Arkansas 2015 – How Much Are Arkansas’ Schools Spending?, Elise Swanson, Gary W. Ritter, Sarah C. Mckenzie

Arkansas Education Reports

When considering the quality of an education system, it is important to understand the resources available to students. Without proper resources, schools are unable to provide a quality educational experience, and students leave school unprepared to be fulfilled, productive members of society. School funding is therefore an important, and often controversial, topic in education policy. Arkansas’ public education system first ran into constitutional trouble over school funding in 1983 when the Supreme Court decided in Dupree v. Alma that the school funding formula was unconstitutional because it was based on valuations of the local tax base, not on the needs …

The Effect Of Public And Private Schooling On Anti-Semitism, Jay P. Greene, Cari A. Bogulski

Education Reform Faculty and Graduate Students Publications

Most major American Jewish organizations oppose voucher and other school choice programs based in part on the fear that private, mostly religious, schools do not check the development of anti-Semitism as well as do government-operated public schools. To examine whether private and public schools differ in their effect on the emergence of anti-Semitic attitudes in adults later in life, we conducted a large survey of a nationally representative sample of adults in the United States. Subjects were asked to provide details on the type of school they attended each year between 1st and 12th grade, including whether the school was …

When You Say Nothing At All: The Predictive Power Of Student Effort On Surveys, Collin Hitt

Education Reform Faculty and Graduate Students Publications

Character traits and noncognitive skills are important for human capital development and longrun life outcomes. Research in economics and psychology now shows this clearly. But research into the exact determinants of noncognitive skills have been slowed by a common data limitation: most large-scale datasets do not contain adequate measures of noncognitive skills. This is a particularly acute problem in education policy evaluation. We demonstrate that there are important latent data within any survey dataset that can be used as proxy measures of noncognitive skills. Specifically, we examine the amount of conscientious effort that students exhibit on surveys, as measured by …

Second Year Results From Razor C.O.A.C.H., Sarah Moore, Josh Raney, Gary W. Ritter, Kristin K. Higgins

Arkansas Education Reports

In this paper, we present the results of a random assignment evaluation on the Razor C.O.A.C.H. (Creating Opportunities for Arkansan’s Career Hopes) program. As the program started in the 2012-13 school year, this evaluation analyzes the impact of the program in its second year of operation and includes two cohorts of students. The focus of the comprehensive evaluation is the impacts of the program on students’ academic outcomes, short-term noncognitive outcomes, short-term college and career readiness outcomes, and post-secondary outcomes.