Educational Assessment, Evaluation, and Research Commons^™

Onward In Higher Education: Business Faculty Perspectives On Authentic Assessment, Farah L. Kashef, Matt Townsley

Journal of Research Initiatives

This mixed-method study explored business faculty’s perspectives on drawbacks and benefits associated with authentic assessment at 10 R1 Midwestern universities. In search of solutions, faculty were also asked to provide recommendations in implementing authentic assessment. Quantitative and qualitative findings suggest most business faculty are in favor of assessment strategies that promote higher order thinking and real-world practices. However, ongoing faculty professional development opportunities and reconsidering the assessment culture of higher education are needed to make this important shift towards authentic assessment.

A 5-Week Personalized Training Workshop To Assess And Evaluate Faculty Members Teaching Online, Felix O. Quayson, Christopher Zirkle

Journal of Research Initiatives

The authors developed the Skills of Inquiry (SoI) model accompanied by a logic model to assess and evaluate faculty members teaching online. The Skills of Inquiry is based on faculty members’ abilities to understand the online environments, skills development pertaining to online teaching, and acquisition of specific online skills. The Skills of Inquiry model was used to personalize a 5-week workshop training and development module specifically for faculty members teaching online. The training workshop is effective in training faculty members to acquire specific online teaching skills through customized and individualized professional development learning. The training is a self-paced asynchronous online …

Establishing The Validity And Reliability Of The Locus Assessments, Tim Jacobbe, Bob Delmas, Brad Hartlaub, Jeff Haberstroh, Catherine Case, Steven Foti, Douglas Whitaker

Numeracy

The development of assessments as part of the funded LOCUS project is described. The assessments measure students’ conceptual understanding of statistics as outlined in the GAISE PreK–12 Framework. Results are reported from a large-scale administration to 3,430 students in grades 6 through 12 in the United States. Items were designed to assess levels of understanding as well as components of the statistical problem solving process as articulated in the GAISE framework. We discuss details of how the model used to develop the LOCUS assessments guided the gathering of evidence for validity and reliability arguments. Three types of validity evidence are …

Focused On Pedagogy: Qr Grading Rubrics For Written Arguments, Ruby Daniels, Kathryn Appenzeller Knowles, Emily Naasz, Amanda Lindner

Numeracy

Institutional assessments of quantitative literacy/reasoning (QL/QR) have been extensively tested and reported in the literature. While appropriate for measuring student learning at the programmatic or institutional level, such instruments were not designed for classroom grading. After modifying a widely accepted institutional rubric designed to assess QR in written arguments, the current mixed method study tested the reliability of two QR analytic grading rubrics for written arguments and explored students’ reactions to the grading tools. Undergraduate students enrolled in a business course (N = 59) participated. A total of 415 QR artifacts from 40 students were assessed; an additional 19 …

2022-2023 Annual Student Assessment Report, Southwestern Oklahoma State University

Assessment Center Reports

Annual Student Assessment Report produced by the SWOSU Assessment Center.

2023 Nsse, Southwestern Oklahoma State University

Assessment Center Reports

National Survey of Student Engagement (NSSE) reports produced by the SWOSU Assessment Center.

2023 Institutional Effectiveness Survey Report, Southwestern Oklahoma State University

Assessment Center Reports

Institutional Effectiveness Survey reports produced by the SWOSU Assessment Center.

Evaluating The Assessment Of Resident Assistant Training, Diego Ulloa

Masters Theses

This poster provides the findings from a qualitative study of four housing professionals responsible for Resident Assistant (RA) training at four institutions in the Great Lake Association of College and University Housing Officers (GLACUHO) professional association that includes the states of Illinois, Indiana, Michigan, and Ohio. This study looked at how those four institutions assessed, evaluated, and improved the centralized training provided to RAs by the department. Key findings include the importance of the use of learning outcomes for training sessions, developing, and maintaining strong relationships with key campus partners, and the issues around prioritizing session topics and departmental needs …

Score Equating Between Aeps-2 And Aeps-3 For 0-3 Year Olds, Yuyan Xia

Theses and Dissertations--Education Sciences

Over the past two decades, the emphasis on educational equity in early childhood education (ECE) and early childhood special education (ECSE) has highlighted the importance of assessment through policies and regulations. Ensuring accurate assessment scores is a fundamental aspect of this trend. The release of the Assessment, Evaluation, and Programming System for Infants and Children, Third Edition (AEPS-3) in December 2021 led to a shift from the Second Edition (AEPS-2) in child development scoring. In order to harmonize the previous and updated assessment versions for children aged 0-3 across six developmental domains, a common item non-equivalent design, featuring fixed parameter …

Mapping Assessment Tasks And Lecturer Considerations: A Case Study Of A Nz Primary Teacher Education Programme, Wendy Carss, Bronwen Cowie, Sheralyn Cook

Australian Journal of Teacher Education

Initial teacher education programme and paper assessments need to encompass university degree requirements and attest to student mastery of teacher professional competencies. This article reports one aspect of an investigation of the nature and principles employed in the design of assessments for a three-year Bachelor of Teaching (Primary) programme. The assessment tasks included in the 2022 academic papers were mapped and paper leaders interviewed; practicum and placement papers were not included because these were being redeveloped at the time. Findings are consistent with assessment design principles associated with authenticity and equity of access to the demonstration of understanding through tasks …

Experiential Learning Projects As Assessment In Initial Teacher Education, Renee Crawford, Louise E. Jenkins, Lydia Wan

Australian Journal of Teacher Education

In a rapidly changing global environment, Initial Teacher Educators (ITE) have a responsibility to role-model contemporary teaching approaches, which develop graduates who think creatively and flexibly in educational workplaces. An important aspect of this work is supporting pre-service teachers (PSTs) to understand how to design assessments which facilitate a deep understanding of student learning. This learning can be achieved through the implementation of assessments which model contemporary practices and enrich student learning in ITE courses. This paper discusses new ways to consider the purpose of assessment by focusing on Experiential Learning (EL) as a form of assessment in ITE. This …

Social Studies And History Curriculum Assessment In Colleges Of Education In Ghana: Reflective Practices Of Teacher Educators, Bernice Oteng, Ronald Osei Mensah, Pearl Adiza Babah, Enock Swanzy-Impraim

Research outputs 2022 to 2026

Assessment is a crucial and essential component of successful instruction and learning. As a result, teachers must examine their actions in the classroom through a process of reflective practices (self-observation and self-evaluation) and think about why they do it and if it works. This essay examines the reflective practices of Ghanaian teacher educators regarding the evaluation of social studies and history curricula. It made use of the sequential explanatory mixed method as a methodology drawn from questionnaires, interviews and document reviews. The findings disclosed that teacher educators in Ghana’s colleges of education have low efficacy and poor reflective practices when …

Assessment Fairness In Freedom Of Learning In Open And Distance Education, Suci Nurhayati, Puryati Puryati

REID (Research and Evaluation in Education)

This research aims to analyze or obtain a comprehensive overview of fairness in independent campus programs in open and distance education by looking at the conformity of the concept of assessment fairness themes in the practice of assessment on the program. This research was conducted using a qualitative research design, and the data obtained were analyzed descriptively. Data collection was carried out by conducting in-depth interviews with seven sources with competencies related to independent campus programs and their policies at the Universitas Terbuka as organizers of open and distance education systems, using interview guides and documentation studies. Data were transcribed …

The Use Of Single-Subject Research Design For Assessment, Evaluation, And Research In Higher Education, Jordan C. Spencer

Theses and Dissertations

While enrolling in a college for a degree is still considered to be more of the expectation than the rule in households across the United States it is facing public criticism regarding its overall cost versus long-term benefits. This has prompted researchers to reevaluate practices in higher education to determine ways in which post-secondary institutions can improve their overall outcomes. In part, research indicates that this should include revisions to assessment, evaluation, and research (AER) practices. Currently, higher education faces many challenges associated with rigorous AER practices, which include an institutional focus on accountability measures, organizational challenges, and difficulties in …

Review Of The Australian Core Skills Framework And Digital Literacy Skills Framework And Relevant Assessment Tools. Final Report, Louise Wignall, Anita Roberts, Jana Scomazzon

Vocational, adult and professional education research

This is the final report of a project commissioned in 2020 by the then Department of Education, Skills and Employment to review the Australian Core Skills Framework (ACSF) and Digital Literacy Skills Framework (DLSF) and tools available to support assessment using these frameworks. The review explored the development and history of the ACSF and DLSF, their use and application in the Australian context and examined features of selected international adult skills frameworks and curriculum. Through this desktop research and range of consultation activities with stakeholders, the Australian Council for Educational Research (ACER) has investigated current use of the ACSF and …

Developing Rubrics Using The New Oru Outcomes, Kim Boyd, Trevor Ellis, Leighanne Locke, Terry Shannon, Rachael Valentz

Academy Resources

The development and examples of key program assessments (KPAs) are shared. Dr. Boyd opens the presentation. Dr. Shannon (B.S. Sports Management) begins by providing an overview of aligning program and ORU outcomes. Prof. Locke (B.S. Mathematics) walks through how current assignments were chosen to be used as key program assessments and then how the rubrics were revised to improve alignment. Dr. Valentz (B.S. Nursing) discusses how to improve the foundational alignment between program outcomes and the criterion (rubric row headings) used to measure them. She shares how criterion can be contextualized in different assignments and demonstrates in Brightspace, by D2L, …

Examinees’ Affective Preference For Online Speaking Assessment: Synchronous Vs Asynchronous, Yuxiao Du, Fangzheng Zhang

Chinese Language Teaching Methodology and Technology

With technological advancement and the COVID pandemic, online speaking assessment is increasingly used in language teaching. Two modes are developed: online synchronous testing (direct human-to-human interview) and online asynchronous testing (semi-direct human-to-machine interview). Ample literature has explored how each of the two online modes differs from traditional face-to-face speaking assessments. However, few studies have investigated the differences between the two modes, especially in terms of examinees’ affective preferences. This study, therefore, compares the extent to which each mode is accepted and favored by test takers and explores why such an affective preference emerges. The participants are 46 college students enrolled …

Book Review: Going Gradeless, Grades 6–12: Shifting The Focus To Student Learning, Ana De Jesús, Alesia Mickle Moldavan

Excelsior: Leadership in Teaching and Learning

This book review of Going Gradeless, Grades 6–12: Shifting the Focus to Student Learning by E. Burns and D. Frangiosa (2021) provides an alternative pedagogical method to assessment that uses a “gradeless” approach to learning for purposes of removing the stress and negative impacts of traditional grading practices while maintaining accountability with equity in mind. In this review, we describe the foundational underpinnings that frame the book and summarize some of the observed benefits as well as challenges faced by the authors who implemented this approach. We provide an overview of the chapters situated in four major takeaways guiding this …

It’S All About To Change: Implications Of Reforming Grading & Assessment Within A Public School District, Divonna M. Stebick, Megan L. Pilarcik, Daniel W. Hartman

Education Faculty Publications

Calls to reform grading systems and other assessment practices have been growing for several decades. There is consensus among many educators that grading and assessment practices that have been traditionally accepted as good practice are at best ineffective and at worst have a negative impact on raising achievement. Consequently, there is no single solution or methodology for grading that has emerged as the best practice. A variety of contemporary grading approaches have gained widespread popularity in recent years, typically being referred to as standards-based grading, standards-referenced grading, proficiency-based grading, or competency-based learning. A challenge, however, is that different school districts …

High Quality, Equity, And Assessment: An Analysis Of Variables Impacting English Learner Standardized Science Test Performance And Implications For Construct Validity, Maria Del Carmen Salazar, Joanna K. Bruno, Melissa P. Schneider

Curriculum and Instruction: Faculty Scholarship

In the United States, assessment is seen as a lever that can facilitate high-quality education. This study on English learners, students whose native language is not English, was based on data from eighth-grade English learners’ performance on science general and content-specific (physical, life, and earth science) standardized exams and an English language proficiency exam. The researchers utilized regression analysis to examine factors (i.e., socioeconomic status, home language, English language proficiency, and receptive and productive elements of language) that are predictive of English learner performance in general and content-specific science standardized assessments to identify implications for construct validity of high-stakes science …

Living, Learning, Serving: Outcomes Of Combining A Living-Learning Program With Service-Learning Courses, Allen Brizee, Kate Figiel-Miller, Marianna Carlucci

Jesuit Higher Education: A Journal

Abstract

This article discusses a two-year IRB-approved programmatic case study that measured the outcomes of merging a living-learning program (LLP) with service-learning. The study compared student survey data from four different pedagogical models, one of which was the hybrid LLP-service-learning model where service-learning students also participated in the LLP. We also interviewed instructors who used the LLP with their service-learning pedagogy. We used a one-way ANOVA and a non-parametric test to code and analyze the survey data. We used grounded theory to code and analyze interview data. Survey data revealed that the LLP-service-learning hybrid model scored the lowest of the …

Equitable Assessment For Elementary Dual-Language Learners, Joliette Mandel

Dissertations, Theses, and Capstone Projects

This thesis points out the contradictions between the goals of standardized testing and the goals of dual-language elementary education. I argue that dual-language elementary schools in the New York City Department of Education would be better served by a performance assessment model to measure student and school accountability for several reasons. Performance assessment is more equitable for students who are marginalized by their race, language, or class. Many students who attend dual-language schools in the NYCDOE fall into all these categories. I will discuss in depth why standardized testing is failing students, particularly those in dual-language elementary programs. Next, I …

Taking Time: Part-Time Students And Student Learning Outcomes Assessment, Sarah K. H. Macdonald

Dissertations, 2020-current

For decades, higher education institutions have undertaken comprehensive and systematic efforts to explore, document, and improve the assessment of student learning outcomes, as well as improving learning itself. However, many of these assessment practices have been designed for full-time traditional students, even as the number of non-traditional students enrolled in higher education continues to grow. One group of these non-traditional students remains particularly invisible on their campuses, in their classrooms, and in assessment practices: part-time students.

Part-time students, defined by the U.S. Department of Education’s Integrated Postsecondary Education Data System (IPEDS), are students who are not full-time (USDOE, n.d.). This …

Drafting An Assessment Plan For Your Instruction Program: Sustainably Assessing Information Literacy In An Undergraduate Stem Course, Kevin Moore, Clinton K. Baugess

All Musselman Library Staff Works

Assessing student learning across a library instruction program can be infeasible without being strategic, intentional, and realistic. Librarians at a small college will share how they developed a sustainable, 3-year assessment plan for the ACRL Framework and targeted a 100-level biology course-one of the two high-enrollment STEM courses that receive library instruction on their campus each year. The presenters will share their assessment plan, flipped instruction model, workflow-management strategies, and lessons learned for collaborating with STEM faculty to assess information literacy.

Exploring Associations Between Self-Concept, Ideal Self, And Assessment Practices Of High School Science Teachers, Jeffrey Celebre

Seton Hall University Dissertations and Theses (ETDs)

Despite being faced with major barriers to overcome, local school districts and teachers must work to align their instructional practices in science and their assessment practices to the Next Generation Science Standards (NGSS) and the New Jersey Student Learning Standards for Science (NJSLS-Science). As described in the literature, a three-dimensional approach to assessment design is one of the qualities considered the model for a true and authentic measurement of student aptitude in science. As teachers work to understand NGSS alignment and their ability to meet alignment standards as an assessor while designing and implementing NGSS-aligned assessments, they will undoubtedly encounter …

My First Time Ungrading: Approach Used And Reflections, Heather Leslie

Feminist Pedagogy

A few months ago, I began devouring information about ungrading with a fervent appetite. I started with the book Ungrading: Why Rating Students Undermines Learning (and What To Do Instead) edited by Susan Blum and listened to just about every podcast where she was interviewed about this topic. I then read other books she recommended like Wad-Ja-Get: The Grading Game in American Education by Howard Kirschenbaum and Punished By Rewards: The Trouble with Gold Stars, Incentive Plans, A’s, and Praise by Alfie Kohn. Recently, I have become much more dialed into the ungrading movement by reading articles from Teachers Going …

Open-Ended Problems: How Much Time Is Enough Time?, Kaitlyn Solymosi

Honors Projects

The purpose of this action research is to observe and analyze students’ responses to open-ended questions, examine student engagement in assessment, and to gauge a reasonable timetable of how long students are predicted to work through them. A great amount of research has been conducted regarding the benefits and impact of open-ended problems in K-12 classrooms, but very little research has focused on the proper amount of time that should be allotted for them. This action research aims to establish a general rule for determining the proper amount of time allotted for open-ended problems. Implications for mathematics teachers will be …

Diseño De Una Prueba De Clasificación De Español No Estandarizada, David Sánchez-Jiménez

Publications and Research

Resumen

El presente estudio propone una nueva prueba de clasificación en línea específicamente creada para los estudiantes de español de New York City College of Technology (NYCCT), una institución caracterizada por la elevada diversidad de su comunidad educativa. En 2017 el Departamento de Humanidades decidió crear su propia prueba de clasificación de lenguas. En el caso del idioma español, esta se creó y se pilotó en 2018 con una muestra de 144 alumnos distribuidos en 3 niveles: inicial, intermedio y avanzado alto. Los resultados de esta práctica mostraron datos relevantes que permitieron identificar factores inesperados en el análisis, como la …

Building Librarians’ Research Skills Through Experiential Learning, Raeda K. Anderson, Katherine Fisher, Emily Williams, George Usmanov

Georgia Library Quarterly

Experiential learning programs are an effective method to teach data-focused research skills and statistical analysis. We examine the effects of a participatory research training program developed and executed by a data librarian and administered to library employees at a large academic library. The program aimed to improve research skills and increase research productivity. This study employs a survey within a concurrent mixed methods methodological framework to examine the outcomes of the training program. Our findings show that the program served as a low-cost, short-term, effective method of teaching data collection and quantitative analysis that increased participants’ knowledge of the research …

Looking Behind Virtual Lenses: Field Experience, Modeling, Coaching, Partnerships, Supervision, And Feedback, Tamara Lynn, Shantel Farnan, Jessica A. Rueter, Adam Moore

Education Faculty Publications

Small special education programs (SSEPs) are composed of limited faculty tasked with educating interns dispersed across large geographical areas (Reid, 1994). These needs underscore a call for more flexible educational program options. Moreover, Kebritchi et al. (2017) found professors in higher education institutions sought a variety of instructional methods to critically respond to barriers experienced by SEPPs. The purpose of this article is to highlight virtual methods utilized by SSEPs for field experiences, modeling, coaching, feedback, supervision, and partnerships to leverage faculty expertise effectively and efficiently, to expand recruitment in programs, and to support teacher retention efforts. Using the Council …