Educational Assessment, Evaluation, and Research Commons^™

Walking The Talk: Embedding Standards-Based Grading In An Educational Leadership Course, Matt Townsley

Journal of Research Initiatives

The purpose of this paper is to provide a model for educational leadership faculty who aspire to walk the talk of effective feedback by embedding standards-based grading (SBG) in their courses. Rather than focusing on learning, points are the currency of K-12 classrooms across the country. Over 100 years of grading research suggests typical grading practices are subjective at best. Some schools are responding by implementing SBG, yet few articles describe how higher education embeds this philosophy in educator preparation coursework. In this essay, the author documents how to design assessments, align rubrics, and provide feedback to aspiring school leaders …

The Double-Edged Sword Of Standardized Testing, Barbara Meyer, Christine Paxson

Faculty Publications - College of Education

Assessment in Pk-12 schools has always been a challenge. Measurement and comparison of students, schools, school districts and states provides accountability for all stakeholders in education. Standardized testing has become the norm, but it is overused. The Partnership for Assessment of Readiness for College and Careers is one test adopted on a large scale to measure whether students are prepared for college and career. Parents are one of the stakeholders who had had concerns about another standardized test, but they also recognize the need. A survey was conducted of parents whose children took PARCC to learn their expectations of the …

Setting The Foundation For Democratization: Assessing The Quality Of A Political Science Program In Egypt, Yasmin Khodary

Political Science

Purpose – The purpose of this paper is to investigate the quality of a political science program in an Egyptian private university through assessing three particular dimensions: knowledge of political science core facts and theories; reading comprehension skills; and critical thinking (CT).

Design/methodology/approach – A case study research approach was used. The study relies also on a quantitative methodology. Quantitative data were collected from students in the second and fourth years of political science to assess their knowledge of core political science facts and theories, reading comprehension and CT through the online California Critical Thinking Skills Test.

Findings – Unlike …

Surveying The Soul: Shaping Auto-Criticism With Reflective Arts-Based Inquiry To Assess A Female Doctoral Experience, Monica Fain Rezac

Electronic Theses and Dissertations

This dissertation is an arts-based auto-criticism assessing the author's experience in her doctoral program. It is also perhaps the first dissertation elaborating on a method called auto-criticism, as derived from educational criticism and connoisseurship. The process of inquiry is rooted in arts-based methods of representation and theories of educational criticism and critical theory in the humanities. Philosophical approaches to artwork and the use of it in these methodologies was inspired by the independent arts practices of intuitive artists. The purpose of this project is for the author to develop, articulate, and create a sense of what auto-criticism entails in the …

2019 Nsse, Southwestern Oklahoma State University

Assessment Center Reports

National Survey of Student Engagement (NSSE) reports produced by the SWOSU Assessment Center.

An Explorative Case-Study Of The Use Of Peerwise To Foster Student Centred, And Peer Supported, Learning In A First-Year Business Module, Anna Dynan, Barry J. Ryan

Irish Business Journal

Peer based learning is not a new concept in business studies in higher education; however, the integration of technology enabled and asynchronous peer learning has limited reported use and even less evidence based evaluation. In this explorative case study, the online tool PeerWise was used to scaffold and support an asynchronous peer-learning environment for a group of 212 first year business studies students. Students were required to create, answer and rate multiple choice questions on topics aligned to their curriculum within the peer constructed PeerWise question database. While there was no statistically significant correlation between PeerWise engagement and final module …

Report From The Stem 2026 Workshop On Assessment, Evaluation, And Accreditation, Rebecca Bates, Angela Arnold, Cary Komoto, Peggy Brickman, R. Alan Cheville, Elizabeth Longley, Jose Mestre, Mihaela Sabin, James Warnock

Integrated Engineering Department Publications

A gathering of science, technology, engineering, and math (STEM) higher education stakeholders met in November 2018 to consider the relationship between innovation in education and assessment. When we talk about assessment in higher education, it is inextricably linked to both evaluation and accreditation, so all three were considered. The first question we asked was can we build a nation of learners? This starts with considering the student, first and foremost. As educators, this is a foundation of our exploration and makes our values transparent. As educators, how do we know we are having an impact? As members and implementers of …

Report From The Stem 2026 Workshop On Assessment, Evaluation, And Accreditation, Rebecca Bates, Angela Arnold, Cary Komoto, Peggy Brickman, R. Alan Cheville, Elizabeth Longley, Jose Mestre, Mihaela Sabin, James Warnock

Reports

A gathering of science, technology, engineering, and math (STEM) higher education stakeholders met in November 2018 to consider the relationship between innovation in education and assessment. When we talk about assessment in higher education, it is inextricably linked to both evaluation and accreditation, so all three were considered. The first question we asked was can we build a nation of learners? This starts with considering the student, first and foremost. As educators, this is a foundation of our exploration and makes our values transparent. As educators, how do we know we are having an impact? As members and implementers of …

Exploring The Factors Teachers Consider In Determining Students’ Grades, Thomas R. Guskey, Laura J. Link

Educational, School, and Counseling Psychology Faculty Publications

The purpose of this study was to investigate the specific factors teachers consider when assigning students’ report card grades. Data were gathered from 943 K-12 teachers from five school districts in a southeastern state in the United States who completed the Teachers’ Grading Practices Survey. Analyses focused on how teachers weigh different factors in determining report card grades, and if these factors and weights differ among teachers who teach at different grade levels and have different amounts of classroom experience. Results revealed statistically significant differences among teachers at different grade levels but no differences associated with teachers’ years of …

Spring, Summer, Fall 2019 Graduation Survey Responses, Uno Office Of Instituional Effectiveness

Graduation

Spring, Summer, Fall 2019 Graduation survey responses master file without identifiers.

Quantitative Literacy In The Core Curriculum Of Hood College: Chapter Ii, Outcomes And Assessment, Betty Mayfield, Ann Stewart

Numeracy

In a previous article, we described our college’s new core curriculum, which included a Quantitative Literacy (QL) component for the first time. We explained how we defined QL in the college catalog, and how we used that definition to choose courses to satisfy the new requirement. We then discussed our early efforts at assessing the effectiveness of the QL program and described our plans for the future. Here we report on our progress towards those goals, including working with faculty from other departments and with our institutional research office to develop a more sophisticated assessment plan, as well as creating …

Reliability And Validity Of An Evaluation Capacity Assessment Instrument For Principals Of Public Schools In Michigan And The Current Distribution Of Evaluation Capacity By Poverty Concentration, Christine M. Lewis

Wayne State University Dissertations

This study used Confirmatory Factor Analysis (CFA) to investigate the reliability and validity of an Evaluation Capacity Assessment Instrument (ECAI) for principals of K-12 public schools in Michigan. The ECAI was then used to investigate whether scores between principals of high poverty schools and low poverty schools were significantly different. The ECAI was administered to 121 principals across Michigan. Based on the results of the CFA, evidence of fit to the a priori model was not established but internal consistency statistics provide some reliability evidence. A Principle Components Analysis (PCA) was conducted and factor loadings were examined. Some groups of …

Undergraduate Students' Attitudes About The Collection, Use, And Privacy Of Search Data In Academic Libraries, Laura W. Gariepy

Theses and Dissertations

The purpose of this study was to understand undergraduate students’ attitudes about search data privacy in academic libraries and their preferences for how librarians should handle information about what students search for, borrow, and download. This is an important area of study due to the increasingly data-driven nature of evaluation, accountability, and improvement in higher education, along with libraries’ professional commitment to privacy, which has historically limited the amount of data collected about student use. Using a qualitative approach through the lens of interpretive description, I used the constant comparative method of data collection and analysis to conduct semi-structured interviews …

Advising Assessment Practices Of Advising Professionals At An Accredited United Kingdom Urban Career-Focused Higher Education Institution: A Replicated Quantitative, Descriptive Analysis, Catherine E. Mahrt-Washington

Dissertations

The Problem

As the world has flattened, the globalization and quality education movement surrounding higher education worldwide has led to the accountability of all stakeholders regarding student success in and beyond the classroom. Student mobility continues to impact tertiary enrollments as families and students consider the proliferated traditional and non-traditional enrollment alternatives along with prospective lower tertiary debt options. Although assessment, an accountability tool, in co-curricular areas such as advising has been overlooked by leaders, advising is not impervious to accountability consequences. The problem is that assessment of advising, if performed, is oftentimes implemented informally, without a well-defined framework or …

Perceptions Of Assessment In Norwegian Occupational Therapy Students, Tore Bonsaksen, Astrid Gramstad, Gry Mørk, Susanne G. Johnson

Journal of Occupational Therapy Education

In addition to securing minimum standards of learning among students, assessment is increasingly used as a tool to improve students’ learning. Assessment quality is measured as part of the Course Experience Questionnaire; however, the original ‘appropriate assessment’ scale has demonstrated ambiguous psychometric properties. The current study aimed to gain knowledge about occupational therapy students’ perceptions of assessment, and this was achieved by examining the factor structure and internal consistency of the six items on the ‘appropriate assessment’ scale. Students from six Norwegian universities (n = 187, response rate 61.3 %) completed the scale and reported demographic information. The factor …

Design Of An Osce To Assess Clinical Competence Of Occupational Therapy Students, Nancy E. Krusen, Debra Rollins

Journal of Occupational Therapy Education

Objective structured clinical examinations (OSCEs) are a series of controlled, timed stations in which students demonstrate clinical skills. OSCEs are commonly used within health professions education to demonstrate competence, prepare for clinical education, and conduct program evaluation. The body of literature addressing the use of OSCEs in occupational therapy (OT) is growing; however, there are no available guidelines for developing an OSCE specific to the profession. The purpose of this paper is to describe the design of an OSCE for OT students prior to fieldwork placement. Twelve OT practitioners participated in a modified-Delphi method to generate possible OSCE scenarios. The …

Validating Bloom's Revised Taxonomy As A Rubric For Assessing Middle School Students' Levels Of Thinking, Siri Torrence Deforest Reynolds

Walden Dissertations and Doctoral Studies

Educators in a rural charter middle school in the United States were challenged with the reliable assessment of student thinking skills even though the development of higher order thinking was an espoused goal for the school. The purpose of this study was to validate a new rubric based on Bloom’s Revised Taxonomy (BRT) to reliably assess student levels of thinking as reflected in the students’ written work. A quantitative, nonexperimental design was used. The focus of the research questions was on the BRT rubric’s reliability and validity. Interrater reliability was assessed using Krippendorff’s alpha. Validity was explored by assessing the …

Assessment Of The Social Attitude Of Primary School Students, Ari Setiawan, Siti Partini Suardiman

REID (Research and Evaluation in Education)

The implementation of Curriculum 2013 at primary school level brings about its own problems to teachers. A serious problem emerges in the assessment, especially the assessment of core competence for the social attitude aspect. This problem arises because social attitude has many dimensions and requires judgments in diverse forms. In addition, the assessment of social attitude is focused on the affective sphere. The objective of this research is to assess the social attitude of grade IV and/or V students of primary school using three integrated instrument models: self-assessment (SA), peer assessment (PA), and observational assessment (OA). This research employed qualitative …

Mapping Of Physics Problem-Solving Skills Of Senior High School Students Using Physpross-Cat, Edi Istiyono, Wipsar Sunu Brams Dwandaru, Revnika Faizah

REID (Research and Evaluation in Education)

Evaluation using computerized adaptive tests (CAT) is an alternative to paper-based tests (PBT). This study was aimed at mapping physics problem-solving skills using PhysProSS-CAT on the basis of the item response theory (IRT). The study was conducted inSleman Regency, Yogyakarta, involving 156 students of Grade XI of senior high school. Sampling was done using stratified random sampling technique. The results of the study show that the PhysProSS-CAT is able to accurately measure physics problem-solving skills. Students' competences in physics problem solving can be mapped as 6% of the very high category, 4% of the high category, 36% of the medium …

Assessing Mechanistic Reasoning: Supporting Systems Tracing, Paul J. Weinberg

Journal of Pre-College Engineering Education Research (J-PEER)

Reasoning about mechanism is central to disciplined inquiry in science and engineering and should thus be one of the foundations of a science, technology, engineering, and mathematics education. In addition, mechanistic reasoning is one of the core competencies listed in the Next Generation Science Standards (NGSS) Engineering Concepts and Practices (NGSS Lead States, 2013). Mechanistic explanations focus on the processes that underlie cause–effect relationships and consider how the activities of system components affect one another.

While some assessment work has been accomplished in engineering education, to date mechanistic reasoning is an area where limited assessment development has been accomplished for …

Winning The War On State-Sponsored Propaganda: Results From An Impact Study Of A Ukrainian News Media And Information Literacy Program, Erin Murrock, Joy Amulya, Mehri Druckman, Tetiana Liubyva

Journal of Media Literacy Education

From 2015-2016, IREX implemented a media literacy training program called Learn to Discern (L2D) that trained Ukrainian citizens to critically analyze news media messages and identify misinformation. In 2017, IREX conducted a quasi-experimental impact evaluation of news literacy skills, knowledge, and behavior using a stratified random sample of L2D participants and non-participants (n=412). The groups were matched for gender, age, region and education levels. A news literacy assessment was administered to both groups via an online survey. The assessment required participants to analyze an objective news article and a disinformation-based news article; demonstrate knowledge of the news media environment; and …

A Different Way To Provide Feedback Of Student Learning, Dr. Deborah Bracke

Education: Faculty Scholarship & Creative Works

This article provides an alternative to traditional letter grading.

Teaching To Assess: An Evaluation Of Assessment Education For Secondary Teacher Candidates, Daniel M. Frederking

Theses and Dissertations

This study examined the assessment instruction of a four-year undergraduate secondary education program and its alignment to the teaching standards and the summative teacher licensure assessment. Document analyses were conducted on the program’s syllabi as well as the assessment portions of the InTASC standards and the handbook of the edTPA. Results highlighted several areas of misalignment in the areas of categorical concurrence, depth of knowledge, and range of knowledge. Also of note, preservice teachers in different secondary subject areas are receiving different amounts of assessment instruction. Additionally, local edTPA data was examined to highlight areas of strength and weakness. Students …

The Effects Of Formative And Summative Assessment On Student's Connectedness, Satisfaction, Learning And Academic Performance Within An Online Healthcare Course, Thomas J. Wing

Boise State University Theses and Dissertations

The quantitative study presented here evaluates the effects of formative and summative assessment on student’s connectedness, satisfaction, learning and academic performance within a university three-credit 400 level online healthcare course. Literature exploring the role that formative assessment plays within an online environment is currently lacking. Additionally, understanding how assessment practices can help support the goals of online healthcare education is vitally important given the rise in popularity of this delivery format.

This study investigated student outcomes in the form of connectedness, satisfaction, learning and academic performance. Four cohorts of students were included in this study. Two cohorts were provided with …

Rsd7: Researcher Skill Development Framework (Us English Edition), John Willison, Kerry O'Regan, Sara K. Kuhn

Open Educational Resources

Created by John Willison and Kerry O'Regan.

www.rsd.edu.au

Adapted for the US context by Sara K. Kuhn.

"The seven-level Researcher Skill Development framework extends the RSD's original 5 levels of student autonomy to include the degree of autonomy required for a successful research career. It therefore addresses not only students, but also early, middle and late career researchers. This involves the extension of the same facets of inquiry that appear in the original RSD framework to include two higher levels: 6 and 7." -- https://www.adelaide.edu.au/rsd/framework/rsd7/

For more information, see: Willison, J., & O’Regan, K. (2007). Commonly known, commonly not known, …

Researcher Skill Development Framework (Us English Edition), John Willison, Kerry O'Regan, Sara K. Kuhn

Open Educational Resources

Created by John Willison and Kerry O'Regan.

www.rsd.edu.au

Adapted for the US context by Sara K. Kuhn.

"Research Skill Development (RSD) is about making explicit and coherent in regular university coursework the incremental attainment of research skills in a specific discipline. In the RSD, there are six facets of the research process, identified from the literature and modified according to Bloom’s taxonomy and our experiences of using the framework in the disciplines. The meaning of ‘research’ in this context is: students actively finding information new to themselves. Underlying this notion is the ‘degree of knowness’ of knowledge: whether research involves …

Faculty Development Initiatives: A Prerequisite For Capacity Building And Enhanced Productivity In A Medical Institution, Rehana Rehman, Hassan Mehmood, Syeda Fatima, Irfanullah Baig, Zohaib Rana, Mohammad Iqbal

Department of Biological & Biomedical Sciences

Objective: To determine the contribution of teaching, learning and assessment forum's initiatives on professional development of faculty and staff.
Methods: This retrospective study was conducted at the Department of Biological and Biomedical Sciences, Aga Khan University, Karachi, from July to December 2016, and comprised teaching, learning and assessment activities carried out from 2012 to 2015. The responses acquired from feedback evaluation were recorded at the end of activity on a Likert-type scale ranging from 1-5. Positive responses were presented for each variable with respective activity type across the study years. The association of the effectiveness of teaching, learning …

Multiple True–False Questions Reveal The Limits Of The Multiple-Choice Format For Detecting Students With Incomplete Understandings, Brian A. Couch, Joanna K. Hubbard, Chad E. Brassil

School of Biological Sciences: Faculty Publications

By having students select one answer among several plausible options, multiple-choice (MC) questions capture a student’s preferred answer but provide little information regarding a student’s thinking on the remaining options. We conducted a crossover design experiment in which similar groups of introductory biology students were assigned verbatim questions in the MC format or multiple true–false (MTF) format, which requires students to separately evaluate each option as either true or false. Our data reveal that nearly half of the students who select the correct MC answer likely hold incorrect understandings of the other options and that the selection rates for individual …

Breaking The Language Barrier: Equitable Assessment In General Chemistry, Eshani Lee

UNLV Theses, Dissertations, Professional Papers, and Capstones

Because language provides the framework through which knowledge is constructed, it is crucial to consider the ways in which students with limited English proficiencies are able to express their understanding. English language learners (ELLs) make up a significant portion of the student body in the education system and represent many ethnic and racial minorities in STEM (Science, Technology, Engineering & Mathematics) fields (Burke & Mattis, 2007). Despite the national push to build a more diversified, STEM-ready workforce, there is little research that considers the way ELLs are assessed in STEM courses at the postsecondary level. Literature reports that science tests …

Motivational Decline And Recovery In Higher Education Stem Courses, Anna M. Young, Paul J. Wendel, Joan M. Esson, Kathryn M. Plank

Education Faculty Scholarship & Creative Works

Decline in student motivation is a concern for STEM education, especially for underrepresented groups in the sciences. Using the Science Motivation Questionnaire II, 41 foundational STEM courses were surveyed at the beginning and end of each semester in an academic year at a small primarily undergraduate university. Significant pre- to post-semester declines were observed in each of five measured motivational factors (Intrinsic motivation, Career motivation, Self determination, Self-efficacy, and Grade motivation), with effect sizes ranging from 0.21 to 0.41. However, in the second semester pre-survey, four motivational factors rebounded, including three returning to initial levels, suggesting that the observed motivational …