Educational Assessment, Evaluation, and Research Commons^™

Measuring Civic And Citizenship Education Through Large-Scale Assessments: Insights From The International Civic And Citizenship Education Study (Iccs), Wolfram Schulz, Valeria Damiani

Civics and Citizenship Assessment

This chapter from the open access book ‘Promoting K-12 Civic Learning and Engagement Through Assessment’ edited  by Hamilton, Rikoon and  Kidd, discusses ways of measuring aspects of civic and citizenship education (CCE) cross-nationally with a specific focus on the International Civic and Citizenship Education Study (ICCS) commissioned by the IEA (International Association for the Evaluation of Educational Achievement). The chapter starts with a brief overview of the main characteristics of the study, the legacy with previous IEA surveys of civic education, and the assessment design including a description of the cognitive and affective-behavioral dimensions that are measured in the …

Establishing Cultivating Dialogic Feedback (Cdf): Enhancing Teacher Evaluation Through The Humanization Of Teachers, Adrian Michael Blumenthal

Electronic Theses and Dissertations

Current practices in K-12 teacher evaluation look to mechanize the evaluation model, and effectuate better teaching by enforcing compliance to the evaluation model. This practice treats teachers as disseminators of instructional practice and thereby of information to students which dehumanizes teachers. The following research takes a different approach to understanding teacher evaluation by working away from fixing the evaluation model by replacing it with another model. Instead, the research examines the breadth of compliance culture, called performativity, and examines the relationship between teacher and evaluator. At the root of the research is discovering how the relationship between teacher and evaluator …

Evaluating Learning Outcomes In A Serious Game: A Practical And Model Checking Approach, Matthew D. Douglas

Theses and Dissertations

This research introduces a novel computational framework to evaluate and predict the educational impact of serious games during development. By using finite state machines (FSM) and model-checking techniques, this study evaluates two serious games. Traditional evaluation approaches, often reliant on resource-intensive human trials, lack scalability and fail to provide early insight into the alignment of game mechanics with learning objectives. This study addresses these challenges of traditional evaluation methods.

Remodeling Pedagogical Evaluation Tools To Incorporate Student Self-Efficacy And Sense Of Belonging In Scientific Research, Richelle L. Tanner, Nicholas P. Burnett, Emily E. King, Anne E. Todgham

Biology, Chemistry, and Environmental Sciences Faculty Articles and Research

Curated undergraduate research experiences have been widely used at colleges and universities for decades to build student interest, technical preparation, and confidence in the pursuit of scientific careers. Educators often employ standardized survey instruments to evaluate learning outcomes for research experiences, but many of these assessments consider only technical skill development and career interests and are not rooted in discrete pedagogical theories. As higher education aims to create inclusive and equitable learning experiences for students, we argue that pedagogical assessment tools for undergraduate research experiences need to expand to consider outcomes such as increased science literacy, confidence in relational “soft” …

Evaluation Of Ste(A)M Education: An Analysis Of Research And Practices From 2014 To 2023, Billy T.M. Wong, Kam Cheong Li, Mengjin Liu

Journal of Educational Technology Development and Exchange (JETDE)

STEM (Science, Technology, Engineering, and Mathematics) and STEAM (by including Arts) education (hereafter STE(A)M education) has been widely studied and practiced. Evaluation plays an important role in assessing and improving the effectiveness of relevant initiatives. Although there has been extensive research on evaluating STE(A)M education, little attention has been paid to examining the patterns of said evaluations. To address the research gap, this paper analyses relevant work on the evaluation of STE(A)M education practices with respect to their countries/regions, levels of education, discipline domains, pedagogical approaches, objectives of evaluation, evaluation areas, as well as data collection and analysis methods. It …

Game-Based Learning With Augmented Reality, Kantida Nanon

Dissertations

Augmented Reality (AR) technology provides an effective and controllable learning environment, especially for complicated, potentially dangerous, or critical content. Effective implementation of interaction and platforms for educational tools can maximize learner engagement and optimize the learning experience. This dissertation explores the potential of AR technology to enhance learning experiences across multiple platforms and subject areas, particularly within chemistry and mathematics. The dissertation is grounded in the field of Human-Computer Interaction (HCI) and explores the potential of AR to increase learner engagement, improve learning outcomes, and transform traditional pedagogical practices.

The dissertation is divided into two parts. The first part examines …

The Alchemy Of Assessment And Evaluation, Jessica Kahlow

Mavs Open Press Open Educational Resources - Archive

This comprehensive guide illuminates the art and science of educational assessment, blending ancient alchemical wisdom with modern AI innovations. Part one lays the foundation, exploring essential principles of assessment and evaluation and its pivotal role in shaping learning outcomes. Part two delves into diverse assessment types, from formative assessments driving real-time adjustments to summative evaluations measuring overall achievement. Part three offers expert guidance on crafting effective assessment designs, ensuring authenticity and inclusivity. Part four examines the alchemy of feedback and grading, empowering educators with strategies for constructive engagement and growth. Part five focuses on evaluations, showcasing methods to assess program …

Estimating And Applying Parameters Necessary To Plan Cluster Randomized Trials (Crts) And Multisite Cluster Randomized Trials (Mscrts), Dea Mulolli

Dissertations

Cluster randomized trials (CRTs) are commonly used to study the effectiveness of educational interventions. During the design phase of a study, it is critical for researchers to ensure their studies are adequately powered to detect meaningful treatment effects, including both main and moderator effects. Designing CRTs with adequate power to detect main and moderator effects requires accurate estimates of design parameters. This research aims to advance the literature on design parameters for power analyses, specifically focusing on empirical estimates of intraclass correlations (ICCs). The work consists of three research papers that examine the role of including the teacher level in …

Using An Interrupted Case Study To Engage Undergraduates’ Critical Thinking Style And Enhance Content Knowledge, Kelsey Hall, Katherine Starzec

Journal on Empowering Teaching Excellence

Access the online Pressbooks version of this article here.

The interrupted case study is a structured way to engage students in active learning. Interruptions, or pauses for reflection and discussion scheduled within the case-study presentation, provide students with a chance to collaborate and engage in critical thinking. Critical thinking style, which is a measure of how one tends to think critically, provides insight into how one tackles problem solving. This article describes a pilot project that combined critical-thinking style and an interrupted case study, delivered over a two-class-period time frame, to four college courses. The project’s goals were to assess …

Profess 12: Project Report 5: The Profess 12 Summer School - Evaluation, Una Beagon, Ruchita Jani, Janet Mckennedy, Brian Bowe, Margaret Morgan, Rosalind M Henry

Reports

This report outlines the evaluation of the summer school which was a key element of the PROFESS 12 project. The key steps involved in the 5-day summer school along with the method by which the feedback and the reflections of the participants of the summer school were collected is described here. Thereafter, the reflections received in the form of audio and text files are analysed and themes identified from the experience of the summer school participants are explained in detail.

The survey responses from the feedback on the summer school participation are also shown graphically. This analysis helped us identify …

Defining Success For An Introductory Mathematics Non-Stem Pathway Course, Clark D. Metz

Graduate Theses, Dissertations, and Problem Reports (ETD)

The enduring debate surrounding college algebra, traditionally a gateway for STEM pathways, underscores the necessity for curricular evolution in response to the rising popularity of non-STEM majors. This dissertation explores the success of an applied college algebra course designed for non-STEM students. The study aimed to determine whether this alternative course could meet diverse student needs while promoting academic achievement and fostering positive attitudes towards mathematics. Findings revealed through a novel approach of a difference-in-differences analysis that students enrolled in the applied college algebra course exhibited higher pass rates due to the introduction of the new course. The lower failure …

An Evaluation Of Clemson University’S Cooperative Extension Services Online Learning Programs And Framework For Program Development And Evaluation, Christina Leard

All Dissertations

The COVID-19 pandemic brought online learning to the forefront with the need to define online learning with best practices. Clemson University Extension conducted a retrospective observational study compiling survey data from virtual extension agents and supervisor’s conducting online programs. A literature review analyzed terminology for online learning to obtain a standardized definition to define online learning for Clemson Extension Services programs; evaluate and identify through literature strengths, weaknesses, opportunities, and challenges (SWOC) of on-line learning; determine and identify online learning competencies for instructors, assessment and evaluation; and analyze and evaluate response data from virtual extension agents delivering online programs in …

Evaluation And Improvement Of Processes And Metrics In A Dual Credit Program, Kristian E. Fischer

Education Theses and Dissertations

This dissertation used convergent mixed methods design in a program implementation evaluation of a dual credit program. A binary logistic regression was performed on dual credit enrollment persistence. A stakeholder survey was deployed to parents, administrators, instructors and counselors associated with the dual credit program. A full review of district documents related to dual credit was included as part of the qualitative component. The results of the regression indicated non-significance in the predictor variables for the dependent variable of dual credit persistence. The emergent themes from the survey included concerns about student readiness for dual credit coursework and a need …

Research Conference 2023: Becoming Lifelong Learners. Proceedings And Program, Kylie Burns (Ed.)

2021-2030 ACER Research Conferences

We know learning in the early years sets the foundations for people’s ongoing academic, cognitive and socio-emotional development. Education across those formative first 12 years is currently segmented, with children often starting fresh in new systems. How can each of us make learning in the early childhood and primary years more effective? Research Conference 2023 will examine research around how we can improve the continuity of learning from birth to 12 years. It will bring together leading international and Australian researchers to provide insights into the best ways to identify, conceptualise, develop and assess these new linkages for learning.

A Qualitative Inquiry Of Research Administrator Perceptions Of Job Satisfaction In A Central Research Administration Unit At The University Of Denver, Noelle L. Strom

Electronic Theses and Dissertations

The Sponsored Programs Administration (SPA) team provides critical research administration services for the research community at the University of Denver. As research funding continues to rise at the University of Denver, retention of research administrators is critically important. Job satisfaction is key to employee engagement and retention. Higher job satisfaction contributes to several positive outcomes for institutions, including lower turnover, higher productivity, lower costs, and employee loyalty. The need to retain experienced research administration professionals is highlighted by the fact that few formal educational routes exist that prepare employees for a career in research administration; rather, developing expertise takes a …

Administrative Evaluations Of Teachers In Tested And Non-Tested Subjects., Natasha Lanham

Electronic Theses and Dissertations

The teacher evaluation process seeks to help teachers grow and thrive, and as a result, affect student achievement. Administrators evaluate teachers from different subjects, grade levels, and experience levels. This qualitative study examined the perception of feedback given through the cooperating school district’s evaluation system from both the teacher and the principal viewpoint in order to explore their perceptions of the quality and quantity of feedback given during the evaluation process. Participants were arts teachers, tested subject teachers, and their evaluating administrators from a suburban school district in Kentucky. The analysis revealed that generic evaluations do not differentiate for teachers …

Iea International Civic And Citizenship Education Study 2022: Assessment Framework, Wolfram Schulz, Julian Fraillon, Bruno Losito, Gabriella Agrusti, John Ainley, Valeria Damiani, Tim Friedman

Civics and Citizenship Assessment

The International Civic and Citizenship Education Study (ICCS) investigates the preparation of young people to undertake their roles as citizens. It gathers and analyzes data from representative national samples on students’ conceptual knowledge and understanding of civics and citizenship, as well as their attitudes to, and engagement with, aspects of civics and citizenship. ICCS builds on a succession of IEA studies in this field dating back to 1971, and especially since 2009. The 2022 study has been developed to build on previous perspectives on, and monitor changes in, such enduring issues as: levels of civic knowledge and understanding; patterns of …

The Effects Of Early Identification And Intervention On Reading Scores At The Kindergarten Level, Cara Milby

Murray State Theses and Dissertations

Research indicates that reading fluently is the key to success: academically, economically, socially, as well as to a healthier lifestyle (Forrest, 2018; Wanzek et al., 2018). While research has shown that Response to Intervention (RTI) is a positive instructional program that will increase primary students’ academic abilities at grades 1 and 2 (Richards et al., 2007), there is a need for more research regarding RTI with kindergarten students. This quasi-experimental quantitative research study examined if early identification and intensive intervention through the addition of Response to Intervention (RTI) at the kindergarten level will lead to increased reading scores and better …

An Evaluation Of The Implementation Of Study-From-Home Policy In Senior High School During Covid-19 Pandemic, Junus Simangunsong, Hastuti Mustikaningsih

REID (Research and Evaluation in Education)

The increase in Coronavirus Disease-19 in Indonesia has prompted the central and regional governments to issue a study-from-home (SFH) policy through the Minister of Education and Culture Circular Letter Number 4 of 2020. The policy aims to reduce activities in schools but uses a meaningful curriculum. The purpose of the evaluation is to see the SFH policy implementation related to teaching and learning activities. The collection of information and data, interviews, and group discussion forums were conducted through online meetings, including video conferencing and other social media. This policy implementation was analyzed in the aspects of context, input, process, and …

Introducing The Education Technology Value Evaluation Tool For Schools, Ralph Saubern, Pauline Taylor-Guy, Fabienne Van Der Kleij

International Conference on Assessment and Learning (ICAL)

Despite the enthusiasm for technology in school education, the importance attributed to it by policymakers, educators, students and parents, and the extensive impact of the changes in practices and provision related to the adoption of digital technologies, research has provided little evidence that the adoption of digital technologies has led to improved outcomes for learners. Given the significant, ongoing investment in technology and research evidence that technology has potential to improve learning outcomes, it is timely for education systems and schools to have access to tools and frameworks that can support effective use of technology in education. This paper introduces …

Evaluating Utah's Rural Online Initiative: Empowering Organizational Leaders Through Remote Work, Paul A. Hill, Amanda D. Ali, Lendel K. Narine, Andrea T. Schmutz, Tyson M. Riskas, Debra M. Spielmaker

Journal of Extension

Compared to urban counties, Utah's rural counties experienced high levels of unemployment. Informed by a statewide needs assessment, Utah State University Extension developed a remote work leadership course to equip business leaders with knowledge and skills to create remote jobs as a solution to rural unemployment. This descriptive evaluation study collected data from course participants (N = 62). Findings showed short-term outcomes were achieved; participants experienced increases in knowledge and skills and had more positive intentions toward creating remote jobs and hiring employees from rural counties. Extension professionals can design and evaluate their programs using the framework in this study.

Teacher Evaluation In Saudi Arabia Relative To National And International Teacher Evaluation Standards And Best Practices, Suhayb Kattan

Dissertations

Teacher evaluation systems are essential in tracking the quality and effectiveness of teachers and education systems. This dissertation examines the teacher evaluation system in Saudi Arabia through multiple perspectives, including research-based frameworks for teacher evaluation, norms and best practices from OECD countries, and established Saudi Teaching Standards. An explanatory sequential mixed methods model was used. Data collection involved 643 responses to a teacher survey that covered working conditions, existence of professional learning communities, and teacher evaluation practices, methods, and uses. Following the survey, in-depth interviews were conducted with 11 teachers and 3 key informants.

Findings from this study indicate the …

Exploring The Impact Of An Online Leadership Course On Japanese Undergraduates' Conceptions Of Leadership, Ryosuke Watanabe

Dissertations

Today’s complex problems transcend borders and require a collective and adaptive learning response. Literature suggests that, because the traditional hierarchical or positional style of leadership cannot effectively address problems we face today, leadership should take a more collaborative and systemic form. Rost (1991) characterized this new approach as the postindustrial paradigm of leadership. Unfortunately, changing the existing conception of leadership is not easy. Furthermore, assessing people’s leadership perceptions is challenging because leadership is often a tacit and latent construct. Because formal leadership education is still in its infancy in Japan, little is known about the impact of leadership education in …

Iea International Civic And Citizenship Education Study 2022: Assessment Framework [Early Online Version], Wolfram Schulz, Julian Fraillon, John Ainley, Bruno Losito, Valeria Damiani, Tim Friedman

Civics and Citizenship Assessment

The International Civic and Citizenship Study (ICCS) 2022 continues IEA’s investigation into the ways in which young people understand and are prepared to be citizens in a world where contexts of democracy and civic participation continue to change. This assessment framework provides insight into the study’s conceptual background, cognitive, affective-behavioral and contextual content, and assessment design. It also describes content relevant for the measurement of the United Nations’ Sustainable Development Goals (SDGs) that is related to Global Citizenship Education (GCED) and Education for Sustainable Development (ESD). Countries that participated in ICCS 2009, ICCS 2016, and ICCS 2022 will be able …

Contributing Factors Of Success For Middle School Agriculture Teachers, Sara Hughes

Murray State Theses and Dissertations

The changing dynamics of middle school agriculture education requires research into the causes for some teachers and chapters to succeed while others do not. The reasons for success may be varied and related to many different factors. The aim of this study is to determine how programs perform at high levels, what factors help them achieve success, and the role that evaluation may play in the process. The types of evaluation used for middle school agriculture teachers in Georgia will also be analyzed.

Students’ Perceptions Of Factors Contributing To Enrollment In Arabic Courses At A College Level, Qays Q. Majeed

USF Tampa Graduate Theses and Dissertations

The purpose of this evaluation was to investigate the factors that contribute to students’ recruitment and retention in the Arabic program at my college. The two questions guiding this evaluation were: (1) what factors influence students to join the Arabic program at my college and (2) what factors influence students to continue to enroll in subsequent higher-level Arabic courses? To accomplish this evaluation, I conducted interviews with six former students --four women and two men--in the Arabic program. Eight themes emerged following analysis of data. The findings show that students joined the Arabic program for a variety of reasons. Several …

Success And Sustainability Of Sponsored International Development Programs In Education: A Narrative Inquiry, Sara Beth Leikin

USF Tampa Graduate Theses and Dissertations

In a 2008 working paper published by the U.S. Agency for International Development, the authors lamented that, despite billions of dollars in investment in global education initiatives, access to education had been improved in targeted areas, but the return on investment with regards to learning outcomes was “much less impressive” (Gillies & Jester Quijada, 2008). From 2012-2017, USAID engaged in the Egypt STEM Schools Project (ESSP) along with four partner organizations with the goal of building model science and mathematics public schools that would challenge the educational status quo by providing students with hands-on, constructivist learning and providing an alternative …

Competent Office Staffing For Minority, Women-Owned, And Local Business Enterprise Contractors Via Workforce Development, Ralston C. O'Connor

Walden Dissertations and Doctoral Studies

AbstractConversations about workforce development in the construction industry are centered around improving the construction trade skills of displaced workers or other unskilled or untrained individuals. Rarely will that conversation focus on the administrative skills necessary for contractor success, particularly in public works construction where administrative oversight is a challenge for the minority, women-owned, and local business enterprise (MWLBE) community. Anecdotal data indicated that MWLBE contractors are skilled at their trades and have little or no problem accessing a skilled labor force. Where these contractors tend to fall short is in the administrative requirements of government contracts. The purpose of this …

Pre-Reading Activities And Students’ Achievement In French Oracy: Implication For Evaluation In Library And Information Science, Kenneth O. Eze Phd, Grace C. Offorma Phd, Christian S. Ugwuanyi Phd, Chinedu I.O. Okeke Phd

Library Philosophy and Practice (e-journal)

Library and Information Science is a profession that is full of people who are passionate about making a positive change in the world. In this profession, the workers are known as Librarians whose duty is to bridge the gaps that exist between people, information and technology. Librarians work to create reader’s advisory resources to encourage young students to develop a lifelong love of reading and learning through pre-reading activities. This paper thus, investigated the effect of pre-reading activities on students’ achievement in French oracy. Using quasi-experimental research design, 154 senior secondary one students participated in the study. French Oracy Achievement …

A Correlational Study Of School Report Card Grades And Degrees Of Poverty, Morgan V. Blanton

Journal of Organizational & Educational Leadership

This study explores the ways in which public schools are evaluated and the results are reported in North Carolina. Currently using a formula that calculates a summative score based on academic achievement and academic growth, the impact of poverty and the complexity of school evaluation is investigated using demographic and testing data from schools across the state. Furthermore, recommendations to alter current practices of school evaluation and reporting as well as suggestions for future research are discussed based on this study’s findings.