Educational Assessment, Evaluation, and Research Commons^™

The African American And The California Basic Skills Requirement For Teaching, Willie C. Thomas Ii

University of the Pacific Theses and Dissertations

This study examines why the passing rates of African Americans on the CBEST are the lowest in California at 60%. Madkins (2011) identified licensure testing as a significant reason why African Americans cannot enter the teaching progression. According to Darling-Hammond et al. (2016), California has an ongoing credentialed teacher shortage. An even more significant need is for teachers of color. According to the California Department of Education (2021), 60% of the state’s educator workforce is White, while the state student body, multicultural and multilingual, is only slightly more than 22% White. While licensure testing for teachers is required in all …

Identifying Exceptionally Performing Third Grades In Indiana: Who Is To Blame?, Emily Troyer 19

Honor Scholar Theses

Federal and state governments want to know that their constituents' tax dollars are being spent well, and that public education is serving the People well. To assess a school's efficacy, they calculate an accountability score. Although state-level scoring systems vary, neither the federal law nor a single state's law require that demographic characteristics of a school's student population be held constant when it is being evaluated. This complicates the assessment of a school's adequacy because factors outside of a school's control influence student performance; when a school is evaluated without taking this into account, it is being unfairly credited with …

Development Of A Tool To Assess Students’ Conceptual Understanding In Introductory Statistics, Nathan L. Tintle, Jill Vander Stoep

Faculty Work Comprehensive List

Few tools exist to assess students’ conceptual understanding in post-secondary, introductory statistics courses. The CAOS test is widely considered to be the gold standard, but was first published in 2007 and does not necessarily reflect some of the changes in student learning at the secondary level. Furthermore, it may not be sensitive enough to measure student conceptual understanding in modern post-secondary statistics courses (e.g., simulation-based inference). In this paper we will describe the process of developing a new instrument which uses some CAOS items, as well as additional new items to improve validity and reliability. We will share the validity …

Exploring Validity Of Computer-Based Test Scores With Examinees' Response Behaviors And Response Times, Fusun Sahin

Legacy Theses & Dissertations (2009 - 2024)

Examining the testing processes, as well as the scores, is needed for a complete understanding of validity and fairness of computer-based assessments. Examinees’ rapid-guessing and insufficient familiarity with computers have been found to be major issues that

"That's The Test?" Washback Effects Of An Alternative Assessment In A Culturally Heterogeneous Eap University Class, Abigail Bennett Carrigan

Dissertations and Theses

To determine learner attitudes toward an alternative assessment, additional washback effects, and possible relationships between cultures of learning and learner attitudes, a mixed methods study was conducted in an advanced multicultural EAP (English for Academic Purposes) class in the USA, where a poster project was implemented as a summative assessment. Qualitative and quantitative data from a pre- and post-test questionnaire as well as classroom observations and an interview with the instructor were used to investigate learners' attitudes towards the assessment. The twenty-four participants representing five cultures of learning from the Middle East, South America and Southeast Asia expressed largely positive …

Evaluating An Assessment Instrument For The Oregon Environmental Literacy Plan, Susan Mae Duncan

Dissertations and Theses

This mixed methods study evaluated the validity, and reliability of an instrument designed to assess a middle school student's proficiency in systems thinking as described in the 2010 Oregon Environmental Literacy Plan. In Stage 1, a forum of middle school students, formal, and non-formal educators used the Delphi technique to reach consensus regarding which skills were important to include in a scoring guide for systems thinking. In Stage 2, the scoring guide was field tested by formal and non-formal educators using a sample of students' work. The two groups' scores were compared using Cohen's kappa to make inferences regarding inter-rater …

Predictive Validity Of Curriculum-Based Reading Measures For High-Stakes Outcome Assessments With Secondary Students Identified As Struggling Readers, Tierney Gifford

Legacy Theses & Dissertations (2009 - 2024)

Curriculum-based measurement (CBM) tools are used widely to assess students’ progress within different stages of the Response to Intervention (RTI) process. Despite the wide-spread use, little research has identified the efficacy of reading CBMs in predicting secondary student outcomes on high-stakes assessments. High-stakes assessments are being utilized to determine outcomes for not just students, but teachers, administrators, and districts. More research is needed to determine if reading CBMs are useful tools for the populations of struggling secondary readers. The current study was a secondary analysis of existing data, which attempted to gain an understanding of this through examining the predictive …

The Impact Of Criteria-Referenced Formative Assessment On Fifth Grade Students' Theater Arts And English Language Arts Achievement, Fei Chen

Legacy Theses & Dissertations (2009 - 2024)

The purpose of this study was to examine the effects of criteria-referenced formative assessment (CRFA) on elementary students' achievement in theater arts and the transfer effect of the treatment on English language arts (ELA) achievement. The role of gender and task type in moderating the treatment-achievement relationship was also explored. The analytical sample included 520 fifth-grade students from thirteen classes who participated in the Arts Achieve project. Selection bias in treatment assignment was adjusted with propensity score analysis (Rosenbaum & Rubin, 1983) for the purposes of obtaining asymptotically unbiased estimates of treatment effects.

Effects Of Criteria-Referenced Formative Assessment On Achievement In Music, Christopher Valle

Legacy Theses & Dissertations (2009 - 2024)

The purpose of this study was to investigate the effects of criteria-referenced formative assessment on music achievement. Archival data from 1,173 students who participated in the first two years of the Arts Achieve project were used to examine the overall and school level effects of criteria-referenced formative assessment on music achievement, as well as its effect on content knowledge, listening, and performance achievement tasks. The relationships between sources of feedback from formative assessment – self, peer, teacher – and music achievement were also investigated. Unbiased causal treatment effects were derived after employing propensity score analysis to minimize selection bias and …

Comparing Cognitive Models Of Domain Mastery And Task Performance In Algebra : Validity Evidence For A State Assessment, Zachary Borden Warner

Legacy Theses & Dissertations (2009 - 2024)

This study compared an expert-based cognitive model of domain mastery with student-based cognitive models of task performance for Integrated Algebra. Interpretations of student test results are limited by experts' hypotheses of how students interact with the items. In reality, the cognitive processes that students use to solve each item may be very different than those assumed by the test developers. Without knowledge of how students are actually operating on these test items and the cognitive processes they use, interpretations from the results of the examination may be compromised. In this study, I investigated the cognitive processes that students used when …

Relationships Between Initial California High School Exit Exam Performance And Academic Success Of High School Students, Jennifer Ellen Lucht

Theses Digitization Project

The purpose of this study was to examine the relationships between the California High School Exit Exam (CAHSEE) and 11th grade GPA and ACT scores as measures of academic success.

A Few Questions We're No Longer Asking About Assessment : The Assessment Landscape And Possible Futures, Todd Lundberg, Gary R. Brown, Robin Jeffers

Office of Academic Innovation Publications

This presentation addresses the current assessment landscape, possible and quickly emerging futures. Presenters, all experienced assessment designers/practitioners, address the topic from a 4-year/national, a 2-year/Washington state, and an educational policy perspective. To guide the conversation, presenters pose "what if" questions based on the changing needs of students and current research into effective pedagogies.

The Relationship Between Participation In Extracurricular Activities And Utah's Proficiency Assessments Of Students In A Suburban School District, Everett N. Kelepolo

UNLV Theses, Dissertations, Professional Papers, and Capstones

The purpose of this study was to discover whether a relationship exists between participation in extracurricular activities and meeting Utah proficiency assessment standards. This study took place in a suburban school district in the state of Utah.

Throughout the history of public education, economic hardships have wreaked havoc on school systems that depend on public sources of income. Schools today are managing these budget restraints by reducing or eliminating extracurricular programs.

The relationship between academic success and participating in extracurricular activities is found concretely in the research data. However, schools must make budget decisions which place activities and academics in …

Compared To What? Reconsidering Assessment In Higher Education, Gary R. Brown

Office of Academic Innovation Publications

There is, as Peter Ewell has recently observed, a tension between accountability and improvement. Batson elaborates, noting the parallel realization is dawning that tracking student outcomes toward learning goals, while a useful and necessary exercise, does not yield as much value as we had thought. Batson argues that developing an accountability system has provided rewards to faculty and student painfully insufficient to warrant the work such development requires. Batson argues that ePortfolios provide a rich alternative. This presentation will demonstrate with real student ePortfolio case studies nested within the context of organizational ePortfolio to provide an example that suggests Batson's …

Assessment Alchemy: Transforming Data Into Action--Why It Is Mostly Myth And What To Do About It, Gary R. Brown

Office of Academic Innovation Publications

There is evidence that institutions and faculty are doing more to assess their programs and their innovations. But subsequent evidence that assessment and research are useful for guiding productive changes in teaching practice and that they improve the student learning experience remains elusive. This presentation will share the story of several different studies that have produced little impact on practice despite substantial or significant results. In comparison, a few (sometimes small) studies have offered insights and avenues that have demonstrated an impact, though not necessarily in ways we might expect. This session will explore conditions and strategies that can make …

Unfair Way To Evaluate Teachers: Test Scores About More Than Classroom, Aaron W. Hughey

Counseling & Student Affairs Faculty Publications

No abstract provided.

Fourth And Fifth Grade Student Achievement And Possible Relationship To Three Teacher Selection Tools, Jon D. Hall

Dissertations

This study attempts to identify possible relationships between cognitive ability, teacher dispositions, and content knowledge and elementary classroom achievement. Data was gathered on teachers using the Wonderlic Personnel TestTM, Star Teacher Pre-ScreenerTM, and Praxis IITM assessments and compared to classroom growth on the Measures of Academic Progress (MAP) computer-based adaptive assessment. The sample size was limited but included fourth and fifth grade teachers that volunteered from selected Kentucky public school districts. Pearson correlation analysis did not show any significant relationship between the scores on the three teacher assessments and student academic growth using the MAP growth index scores for corresponding …

An Evaluation Of The Convergent Validity Of Multi-Source Feedback With Situational Assessment Of Leadership - Student Assessment (Salsa©), Heather Stroupe

Masters Theses & Specialist Projects

The current study assessed the convergent validity of the Situational Assessment of Leadership – Student Assessment (SALSA©), a situational judgment test (SJT), with multi-source ratings. The SALSA© was administered to ROTC cadets via Blackboard; multi-source ratings, which paralleled the leadership dimensions of the SALSA©, were administered via paper. Each cadet completed the SALSA© and was rated by 10 peers, his/herself, and at least one cadre (superior). SALSA© scores were not correlated with any of the corresponding dimensions on multi-source ratings, with one exception. Cadre ratings of Consideration/Team Skills were positively correlated with SALSA© scores on the same dimension. This finding …

Population Cross-Validity Estimation And Adjustment For Direct Range Restriction: A Monte Carlo Investigation Of Procedural Sequences To Achieve Optimal Cross-Validity, David Matthew Goins

Masters Theses & Specialist Projects

The current study employs Monte Carlo analyses to evaluate the effectiveness of various statistical procedures for determining specific values of interest within a population of 1,000,000 cases. Specifically, the proper procedures for addressing the opposing effects of direct range restriction and validity overestimation were assessed through a comparison of multiple correlation coefficients derived using various sequences of procedures in randomly drawn samples. A comparison of the average bias associated with these methods indicated that correction for range restriction prior to the application of a validity overestimation adjustment formula yielded the best estimate of population parameters over a number of conditions. …

The Development Of Curriculum-Based Measurement Local Norms In The Area Of Written Expression, Elizabeth Anne Youngman

Masters Theses & Specialist Projects

This project used Curriculum-Based Measurement (CBM) in the area of Written Expression to establish district norms for Bowling Green City Schools. CBM uses brief fluency measures as indicators of students' academic performance. With the use of CBM, it is possible to identify students who are considered to be at-risk for educational performance. AIMSweb probes were used to assess 1,565 first through fifth grade students from five elementary schools within the Bowling Green Independent School District. Performance was scored using the three most common scoring indices: Total Words Written (TWW), Words Spelled Correctly (WSC), and Correct Word Sequence (CWS). Data collected …

Poor Are Hurt When Schools Rely On Standardized Tests, Aaron W. Hughey

Counseling & Student Affairs Faculty Publications

No abstract provided.

Creating A Math Wall In An Elementary School, Lindsey Moorhouse

All Graduate Projects

Creating a Math Data Wall in an elementary school is a way to track students' progress in mathematics, guide instruction and provide strategic interventions to students who are at risk. A Math Data Wall is a visual representation of the students' progress across grade levels. Assessments have been used in schools for many years. Assessments are used to provide a variety of information to the teacher, school and district. Using assessments in the classroom is a necessary step to track students' progress and guide teacher instruction. A Math Data Wall will give teachers a visual representation of their students' progress …

Assessment Issues Direct, Indirect, And Assessment Utility, Gary R. Brown

Office of Academic Innovation Publications

A National University Telecommunications Network webinar presented at the Ohio Learning Network, Central Ohio Regional Center Winter Learning Institute on Jan. 26, 2009. Focuses on assessment, measures to consider, and curriculum review. Discusses improving learning competencies and providing learning experiences that foster the development of critical learning skills. Attempts to answer the question "Does the measure guide improvement?"

The Pedagogy Of Oppression: A Brief Look At The ‘No Child Left Behind’ (Chinese Translation), Ramin Farahmandpur, Peter Mclaren

Educational Leadership and Policy Faculty Publications and Presentations

NOTICE: This article is in Chinese

The driving forces behind the recent educational policies of the No Child Left Behind Act passed in 2001 are neoliberal social and economic policies that favour outsourcing and downsizing methods of production in the name of flexibility and efficiency. Under the neoliberal economic model, schools must perform similarly to corporate entities. Just as the Dow Jones stock indices measure the performance of companies and represent the pulse of Wall Street, so too the Adequate Yearly Progress Report (AYP) measures and ranks the performance of public schools. One of the most pernicious results of the …

Virginia Standards Of Learning: An Analysis Of Sociodemographic Effects On Exam Passing Rates, Anne E. Foster

Sociology & Criminal Justice Theses & Dissertations

This study proposed to analyze the effects that sociodemographic variables have on Virginia Standards of Learning (SOL) exam passing rates. By analyzing Census data along with passing rates from the Virginia Department of Education, the researcher determined if the independent variables white, non-white, poverty, female-headed household and foreign born explained variance in the SOL passing rates. By using linear regressions for each independent variable as well as a regression for a combination of the independent variables, the researcher was able to show the amount of variance that was explained. The results confirm that variance was present in each test subject …

Piloting A Program-Level Learning Assessment Plan In Plant And Soil Science, Teresa Cerny-Koenig, Catherine A. Perillo, Caroline Pearson-Mims, Kathleen M. Williams, Gary R. Brown, Andrew Morozov, William L. Pan, William E. Hendrix

Office of Academic Innovation Publications

Faculty of Washington State University's undergraduate degree programs in Crop Science, Soil Science, and Horticulture initiated the development and implementation of an assessment process to gauge the extent to which WSU students in the plant and soil science programs meet university and program learning goals. This process was undertaken primarily to help improve our joint teaching efforts and students' learning; it also was encouraged by the needed documentation for the 2007 university accreditation and a need to better match our program learning goals with the University's newly developed Learning Goals of the Baccalaureate. The new program-level assessment plan focused on …

The Pedagogy Of Oppression: A Brief Look At The ‘No Child Left Behind’, Ramin Farahmandpur, Peter Mclaren

Educational Leadership and Policy Faculty Publications and Presentations

Discusses the educational policies of the No Child Left Behind Act passed in 2001 in the U.S. Improvement in performance of schools required by the law; Factors that led to the implementation of the law; Causes of underachievement among students of color; Effect of the trend towards standardized curriculum and testing on working-class students and students of color; Removal of obstacles to the recruitment of high school students in the military.

Feedback, Accountability, And The Standards-Based System, Dannelle D. Stevens

Education Faculty Publications and Presentations

The standards-based system provides feedback and accountability, and, more importantly, it challenges educators to re-examine basic assumptions about teaching, learning, and schooling. These assumptions are changing as a result of the standards-based system, and this has positively affected practices in the classroom.

Relationships Between The Degree Of Restructuring In Western Washington Elementary Schools And The Results On Criterion Referenced 4th Grade Assessments For Reading, Mathematics, Writing, And Listening, Gary C. Newbill

Theses and Dissertations

The primary purpose of this study was to explore relationships between the degree of school restructuring in Western Washington elementary schools and results on criterion referenced tests for reading, writing, mathematics, and listening, administered to 4th grade students in 1997. The sample of convenience included 47 elementary schools from 4 Puget Sound counties.

The current study extended a larger project on educational reform conducted by 7 researchers under the direction of Professor J. T. Fouts. This research explored relationships between the degree of school restructuring, a construct developed through factor analysis of classroom teacher responses on the School Practices and …

Test-Retest Reliability Of Individual Student Acquisition And Retention Rates As Measured By Instructional Assessment, Matthew K. Burns

Dissertations

Statement of the problem. Research on Instructional Assessment is in its infancy. To date, no studies regarding its psychometric properties have been published, but the need for using technically sound assessment methods is prominent in education. Therefore, demonstrated technical adequacy is crucial. The lack of psychometric data for Instructional Assessment is especially noteworthy since it is one of the few educational assessment techniques that includes assessment of performance and cognitive learning theory, both of which have been linked to improved student achievement.

Method. Ninety-six students from Grades 1, 3, and 5 were individually taught a series ofwords from …