Education Commons^™

Hitting For Average: Educational Assessment, Unidimensionality, And The Connection To Baseball Hitting Statistics, Alex Romagnoli

Educational Considerations

The traditional points system and subsequent Grade Point Average (GPA) in education perpetuates an evaluation of academic performance which reflects arbitrary weighting of assignments and/or assessments. As such, GPAs which are calculated using a traditional points system are not unidimensional in their design. The baseball batting and slugging percentage, which serves as established metrics for performance evaluations among baseball players, better reflects unidimensionality. In essence, this paper puts forth an analysis and discussion which posits that baseball batting average and slugging percentage can serve as an example for how unidimensionality can become more prevalent in educational assessments, especially as it …

The Assessment Of The Program Theory And Processes Of Books For Keeps’ Literacy Mentoring Program, Allison Shaw

Graduate Research Showcase

The significance of reading proficiency is frequently emphasized by the time a student reaches the third grade. Third-grade reading proficiency serves as a crucial indicator of a student's potential success in high school and beyond, including college. Research indicates that a failure to attain grade-level reading proficiency by the third grade correlates with increased challenges in comprehending educational requirements and meeting the demands necessary for graduation.

This study focuses on the Books for Keeps Literacy Mentoring Program in Athens, Georgia, which is designed to cultivate supportive relationships between mentors and students, fostering enthusiasm for reading. The examination of the program …

Teaching And Assessing With Taxonomies, Tony Shannon

International Journal for Business Education

The development of taxonomies which articulate learning outcomes are necessary to disconnect the silos among educators, employers and learners (Mathews, 2019; Uranis et al.) What are taxonomies? A taxonomy is a systematic classification of objects. Why do they matter? Without systematic classification and coding it is difficult to compare or combine objects. How are they relevant to teaching and learning? In the years after the Second World War, educational psychologists saw the progress made in the biological sciences with taxonomies and started to apply them in education. Among the first to appear in the mid-50s was the work of Bloom.

Educator Cultural Proficiency Insight Tool: As We Know Better, We Can Do Better!, Queinnise Miller

Tapestry: Journal of Diversity, Equity, Inclusion, and Belonging in Education

All across the U.S. student populations are evolving to reflect the increase in diversity. These students are culturally, linguistically, and economically diverse in nature (CLED). Apart from the health field, cultural competence has not been widely assessed in the education field. A purposeful sample of 362 K-12 teachers, who served 50% or more of CLED students, were administered the Educator Cultural Proficiency Insight Tool (ECPIT). The purpose of this research was two-fold: (a) develop and validate the ECPIT and (b) examine the demographic differences regarding levels of cultural proficiency of current educators. Findings indicated that the ECPIT was a valid …

Instructional Resources To Assess Applied Projects As A Culminating Graduate Communication Student Experience, Michael G. Strawser, Bridget Rubenking, Kelsey Lunsford, Margaret Gravelyn

Journal of Communication Pedagogy

This study reviews the traditional culminating graduate student experiences, theses, and comprehensive exams, as well as a newer, more professionally relevant option, applied research projects. We conceptualize applied projects as student-led, client-connected, hands-on, experiential projects that address a real-world communication problem or topic through the creation of relevant deliverables. We used Glassick et al.’s (1997) scholarship assessed model and the National Communication Association’s communication learning outcomes to determine perceived differences between culminating experiences. Survey results (N = 32) of recent alumni and current master’s level Communication students demonstrate near-equal ratings of applied projects and theses in their ability to both …

Executive Functioning In Language Learning Disabilities: Defining Executive Functioning, Executive Functioning Deficits, And Clinical Intervention Strategies To Support Children, Kaylie Trumble

Honors Theses

This thesis aims to select a working definition of executive functioning (EF) to identify possible EF deficits and clinical interventions strategies to support children. Included are EF deficits, ways of assessing EF components, and intervention strategies for executive dysfunction in LLD. Scenarios are provided to demonstrate both matching strategies to a child and strength/deficit in EF, along with a sample intervention script.

Labor-Based Grading Contracts In The Multilingual Fyc Classroom: Unpacking The Variables, Kara Kristina Larson

USF Tampa Graduate Theses and Dissertations

This descriptive, exploratory study’s purpose is to determine the effects of labor-based grading contracts on students whose historical exclusion results in their current day underrepresentation in higher education. A key component of this study is the emphasis on the student’s own perceptions and feelings about the use of labor-based grading contracts. Using a purposive sample of multilingual First-Year Composition (FYC) sections at an R1 university, I investigated the variables of labor-based grading contracts: demographics and written language characteristics, student motivation, ecological variables (i.e., perceptions of grading contracts), and academic performance measures. Research questions include: 1) How do labor-based grading contracts …

Investigating The Fit Of The Generalized Graded Unfolding Model (Ggum) When Calibrated To Irt Generated Data From Dominance And Ideal Point Models, Abdulla Alzarouni

College of Education and Human Sciences: Dissertations, Theses, and Student Research

The assessment of model fit in latent trait modelling, better known as item response theory (IRT), is an integral part of model testing if one is to make valid inferences about the estimated parameters and their properties based on the selected IRT model. Though important, the assessment of model fit has been less utilized in IRT research than it should. For example, there have been less research investigating fit for polytomous dominance models such the Graded Response Model (GRM), and to a lesser extent ideal point models such as the Generalized Graded Unfolding Models (GGUM), both in its dichotomous and …

Counseling Clients With Traumatic Brain Injury: Exploring Counselors’ Perceived Knowledge, Comfort, And Self-Awareness, Michelle Bradham-Cousar

USF Tampa Graduate Theses and Dissertations

The number of traumatic brain injury (TBI) diagnoses continues to rise each year. Counseling is a critical factor in TBI treatment, and although numerous studies have investigated TBI outcomes, a paucity of researchers have studied professional counselors’ knowledge, comfort, and self-awareness when working with TBI clients. Due to the diversity of counselor caseloads, it is likely that counselors will serve clients with a dual diagnosis that includes TBI. These dual diagnoses include depression, posttraumatic stress disorder, anxiety, psychosis, or another neurocognitive disorder. The purpose of this study was to explore counselors’ knowledge, comfort, and self-awareness when working with individuals with …

When You Can’T R.I.O.T., R.I.O.: Tele-Assessment For School Psychologists, Michael R. Hass, Brian P. Leung

Education Faculty Articles and Research

The acronym R.I.O.T., record review, interview, observation, and test, is a well-known tool for conceptualizing a comprehensive assessment. With COVID-19 and the need to provide school psychological services virtually, it is important to reconsider R.I.O.T. in light of the limitations of virtual assessment. We describe the limitations of virtual assessment and argue that in spite of these barriers, the first three elements of R.I.O.T., record review, interviews, and observations, when used systematically, can provide useful comprehensive assessment data. Specific recommendations are provided for implementing assessment virtually.

Development Of A Faculty Appreciation Of Pedagogy Scale, Carol A. Hurney, Jordan D. Troisi, Lori H. Leaman

To Improve the Academy: A Journal of Educational Development

Evidencing the value of programs and services challenges educational developers to measure a range of outcomes. While direct measures of faculty use of effective teaching behaviors and student learning are desirable, these methods are time consuming and resource intensive. We provide a scale that is easy to deploy and can be adapted to different programs. Our psychometrically sound scale measures one facet of faculty learning about teaching—appreciation of pedagogy. The scale measures awareness, knowledge integration, emotions, beliefs, and self-reported behaviors related to the appreciation of pedagogy. We also examine scale correlates, including teaching identity, confidence, and control.

Students Helping Students Provide Valuable Feedback On Course Evaluations, Adriana Signorini, Mariana Abuan, Gautam Panakkal, Sandy Dorantes

To Improve the Academy: A Journal of Educational Development

The purpose of the student evaluations of teaching (SET) are to help instructors enhance the teaching and learning experience in their courses; however, student feedback can often be more unconstructive than useful because students are usually requested to evaluate instruction with little or no formal training. As a result, SET become missed opportunities for students to effectively communicate their learning needs and for instructors to collect actionable information about how the course is perceived. This project aims to improve the quality of student responses to the open-ended questions that instructors receive by partnering with undergraduates in demonstrating to their peers …

Assessment Literacy In College Teaching: Empirical Evidence On The Role And Effectiveness Of A Faculty Training Course, Kyle D. Massey, Christopher Deluca, Danielle Lapointe-Mcewan

To Improve the Academy: A Journal of Educational Development

This research explores how faculty members’ conceptions of assessment and confidence in assessment change as a result of an instructor training course. Based on a sample of 27 faculty members enrolled in a semester-long instructional development course, this survey-based study provides initial evidence that faculty members can develop confidence in assessment while adopting increasingly complex conceptions of assessment. Based on this study’s findings, we argue that instructional development programs for college faculty have a critical role to play in stimulating faculty learning about assessment of student learning and are an important component in promoting a positive assessment culture.

Cultivating And Sustaining A Faculty Culture Of Data-Driven Teaching And Learning: A Systems Approach, Marsha Lovett, Chad Hershock

To Improve the Academy: A Journal of Educational Development

A prominent goal of colleges and universities today is to enact data-driven teaching and learning. Faculty clearly play a key role, and yet they tend to have limited time, a lack of training in assessment or education research, and few incentives for engaging in this work. We describe a framework designed to address the practical and cultural aspects of these challenges via a cycle of educational development and support: motivate, educate, facilitate, disseminate. We illustrate this systems approach with concrete examples and conclude with lessons learned from our experiences that should translate to a variety of institutional contexts.

Named But Not Known: Teaching And Assessing The Research-Writing Process, Ruth Boeder

Wayne State University Dissertations

In lived experience, the two processes of secondary research and writing overlap and intertwine interminably, creating an overarching complex system as research becomes expressed in writing and writing generates new research. This classroom study explores the two processes as one—the research-writing process—through coding of student journal responses and assessment of student research papers. Analysis reveals students to be thoughtful but not yet as nuanced in their descriptions of their research process as much be desired. They more frequently discuss writing with weaknesses in their research process than with research strengths. Further findings indicate that although it is difficult to assess …

Measuring Media Literacy Inquiry In Higher Education: Innovation In Assessment, Evelien Schilder, Theresa Redmond

Journal of Media Literacy Education

The ability to critically access, analyze, evaluate, and create media messages is crucial in the process of becoming an informed and engaged citizen throughout life. Asking critical questions is not only a valuable dimension of media literacy, but also an indispensable aspect of participating in a democracy. Yet, measuring the effectiveness of media literacy is still a major challenge for the field. It is unclear to what extent people of all ages may engage in critical questioning habits with regards to media. To address this gap, we studied the changes in critical questioning habits for college-aged students enrolled in media …

Context And Regulation Of Homeschooling: Issues, Evidence, And Assessment Practices, Janet F. Carlson

Buros Center: Professional Staff Publications

The article discusses salient factors that influence the current context within which homeschooling occurs. Individual states have applied various approaches to establish regulations that both preserve the rights of homeschooling parents and fulfill the state’s obligation to ensure that its residents receive the education to which they are constitutionally entitled. Case and ethnographic studies or research involving small and selected samples often appear in outlets associated with homeschool advocacy groups or in outlets that are not mainstream. The paucity of empirical evidence derived from methodologically strong research paradigms has led to little certainty about many aspects of homeschooling including its …

Broaching Threshold Concepts: The Trouble With “Skills” Language In Defining Student Learning Goals, Angela J. Zito

To Improve the Academy: A Journal of Educational Development

This essay argues that description of student learning goals as various “skills” presents a conceptual threshold lying between and connecting routinely dichotomized characterizations of student learning—most notably, “concrete” versus “abstract.” Qualitative analysis of instructor interviews shows that “skills” language tends to conceal abstract (that is, affective) learning goals behind more concrete (that is, cognitive) ones. Ultimately, this essay proposes that cognitive and affective student learning goals might be more clearly articulated using threshold concepts within and across disciplines, and that the recognition of “skills” as both affective and cognitive is itself a threshold concept in educational development.

The Fearless Teaching Framework: A Model To Synthesize Foundational Education Research For University Instructors, Alice E. Donlan, Sandra M. Loughlin, Virginia L. Byrne

To Improve the Academy: A Journal of Educational Development

There is often a disconnect between the unit of analysis in rigorous education research, and the types of recommendations that instructors find the most useful to improve their teaching. Research often focuses on narrow slices of the student experience, and university instructors often require broad recommendations. We present the Fearless Teaching Framework to address this gap between research and practice. In this framework, we define four pieces of effective teaching: classroom climate, course content, teaching practices, and assessment strategies. We argue that these are appropriate areas of focus for instructor growth, based on their relations to student engagement.

An Explorative Case-Study Of The Use Of Peerwise To Foster Student Centred, And Peer Supported, Learning In A First-Year Business Module, Anna Dynan, Barry J. Ryan

Irish Business Journal

Peer based learning is not a new concept in business studies in higher education; however, the integration of technology enabled and asynchronous peer learning has limited reported use and even less evidence based evaluation. In this explorative case study, the online tool PeerWise was used to scaffold and support an asynchronous peer-learning environment for a group of 212 first year business studies students. Students were required to create, answer and rate multiple choice questions on topics aligned to their curriculum within the peer constructed PeerWise question database. While there was no statistically significant correlation between PeerWise engagement and final module …

Evaluation Of The Feasibility Of A Two-Method Measurement Design For The Assessment Of Healthy Physical Activity Behavior In Youth, Amelia A. Miramonti

College of Education and Human Sciences: Dissertations, Theses, and Student Research

Purpose: Assess the reliability and validity of self- and parent-report survey responses regarding physical activity (PA), sedentary behaviors, and PA self-efficacy and determine if these data can be combined with objective physical activity monitor data to model the latent construct healthy physical activity behavior (HPAB).

Methods: 126 underserved 4^th-5^th grade students participated in a 12-week after-school nutrition, cooking, and physical activity program (WeCook: Fun with Food and Fitness). Participants and parents (n=103) completed surveys pre- and post-program and participants wore PA monitors for one week at PRE and POST. Unidimensionality and internal consistency reliability were …

Winning The War On State-Sponsored Propaganda: Results From An Impact Study Of A Ukrainian News Media And Information Literacy Program, Erin Murrock, Joy Amulya, Mehri Druckman, Tetiana Liubyva

Journal of Media Literacy Education

From 2015-2016, IREX implemented a media literacy training program called Learn to Discern (L2D) that trained Ukrainian citizens to critically analyze news media messages and identify misinformation. In 2017, IREX conducted a quasi-experimental impact evaluation of news literacy skills, knowledge, and behavior using a stratified random sample of L2D participants and non-participants (n=412). The groups were matched for gender, age, region and education levels. A news literacy assessment was administered to both groups via an online survey. The assessment required participants to analyze an objective news article and a disinformation-based news article; demonstrate knowledge of the news media environment; and …

Culturally Responsive Interviewing Practices, Michael Hass, Annmary S. Abdou

Education Faculty Articles and Research

As communities and school populations continue to become more culturally, economically, and linguistically diverse, the need for comprehensive training and explicit guidelines for culturally responsive school mental health practices also grows. School Psychologists are both expected and ethically responsible to competently assess and serve diverse student and family populations, regardless of potential language or cultural barriers. The current article is focused on describing background and rationale for culturally responsive interviewing practices as they pertain to the roles and responsibilities of School Psychologists. Building on the guidelines and principles of the Cultural Formulation Interview (CFI), developed by the American Psychiatric Association, …

Measuring Quality Standards, Assessment Practices, And Outcomes/Effectiveness Of Competency-Based Education (Cbe) Using Mixed Methods Research To Determine Cbe’S Vitality, Elizabeth Giddens

Dissertations

Competency-based education (CBE) has been around since the late 1800s but has recently served as a revamped pedagogy designed to respond to some of higher education’s most pressing issues today: low degree attainment and problems of equity; lack of alignment between education and the job market; low and slow graduation rates; high tuition; and poor academic quality. Despite the promises of CBE to resolve these issues, the approach to learning lacks much empirical data. The researcher provided a summary of current research on CBE and identified gaps in the literature. Three gaps were identified including why CBE had failed in …

The Validity Of Validity In Debra P.: Judicial And Psychometric Perspectives On Test Consequences, Charles Olney, Brent Duckor

Faculty Publications

We explore the uses and functions of ‘validity’ as a boundary marker between legal theory and psychometrics. Standardized testing regimes rely on experts to articulate the limits of validity. When challenged in courts, these limits become the subject of contestation, requiring practitioners to litigate the validity of validity. This process generates significant discontinuities, resulting from different conceptual relationships to the idea of validity. Through a qualitative textual analysis of specific case law and a quantitative examination of Lexis-Nexis database archives, we trace how legal reasoning elides new developments in psychometric research that would broaden and enrich judicial treatments while showing …

Predicting Teacher Value-Added Results In Non-Tested Subjects Based On Confounding Variables: A Multinomial Logistic Regression, Nathan Street

Doctoral Dissertations and Projects

Teacher value-added measures (VAM) are designed to provide information regarding teachers’ causal impact on the academic growth of students while controlling for exogenous variables. While some researchers contend VAMs successfully and authentically measure teacher causality on learning, others suggest VAMs cannot adequately control for exogenous influences on the classroom. Furthermore, because VAMs are primarily connected to student performance on standardized, high-stakes exams and those exams are resoundingly considered to be inadequate measures of true student learning, educators and educational leaders assert VAM results are moot. The purpose of this study was to consider the potential for student background, teacher preparation, …

Teachers' Knowledge Of Dominie Reading And Writing Assessment Portfolio Word-Level Reading, Maxine Y. Kershaw

Walden Dissertations and Doctoral Studies

There is confusion among teachers in a school district in a southeastern state about the instructional use of the state-mandated Dominie assessment for word-level reading and phonemic awareness skills for kindergarten and first-grade students. Recent assessment data indicated that 20% of students tested in kindergarten and first grade needed remediation. The purpose of this qualitative, bounded case study of a primary school was to understand teachers' perceptions about using the Dominie assessment for instruction, and how these perceptions contribute to the decline in reading scores. Using constructivism as a conceptual framework, the research questions focused on the trends in students' …

Educational Development Efforts Aligned With The Assessment Cycle, Phyllis Blumberg

To Improve the Academy: A Journal of Educational Development

Using an assessment cycle as an organizing framework, this article illustrates how educational development and assessment mutually complement each other. It describes an assessment study conducted to determine if two colleges at a small university met their strategic goals to increase the adoption of learning-centered teaching. This study served the parallel function of assessing the impact of sustained educational development efforts by the Centers for Teaching and Learning (CTL) to promote learning-centered teaching. The majority of interviewed faculty reported using learning-centered approaches. The data collection method itself also served as a teachable moment for faculty who do not attend CTL …

The Aspirational Curriculum Map: A Diagnostic Model For Action-Oriented Program Review, Eric Metzler, George Rehrey, Lisa Kurz, Joan Middendorf

To Improve the Academy: A Journal of Educational Development

When the process of curriculum mapping begins with the faculty’s articulations of the knowledge, skills, and attitudes students should master upon graduation, a curriculum map results that enables faculty to review the curriculum for effectiveness, see the workings of the whole curriculum at a glance, plan assessments, and recognize where adjustments or changes need to be made. This article explains these benefits and lays out a step by step process for building such a curriculum map that can be adapted to any institutional context. We also describe a variety of outcomes from and reactions to our process.

Evaluating Centers For Teaching And Learning: A Field-Tested Model, Susan R. Hines

To Improve the Academy: A Journal of Educational Development

This paper provides a program evaluation model, along with field-testing results, that was developed in response to the need for an evaluation model able to support systematic evaluation of teaching and learning centers (CTLs). The model builds upon the author’s previous studies investigating the evaluation practices and struggles experienced at 53 CTLs. Findings from these studies attribute evaluation struggles to contextual issues involving evaluation capacity, ill- structured curricula, and ill-conceived evaluation frameworks. This field-tested Four-Phase Program Evaluation Model addresses these issues by approaching evaluation in a comprehensive manner that includes an evaluation capacity analysis, curricular conceptualization, evaluation planning, and plan …