Social and Behavioral Sciences Commons^™

Raising The Bar — Integrating Cultural Competence And Equity: Equitable Evaluation – With 2024 Prologue, Jara Dean-Coffey, Jill Casey, Leon D. Caldwell

The Foundation Review

Editor’s Note: This article, first published in print and online in 2014, has been republished by The Foundation Review with minor updates. Whether implicit or explicit, social justice and human rights are part of the mission of many philanthropies. Evaluation produced, sponsored, or consumed by these philanthropies that doesn’t pay attention to the imperatives of cultural competency may be inconsistent with their missions. The American Evaluation Association’s Statement on Cultural Competence provides those who produce, sponsor, and use evaluation an opportunity to examine and align their practices and policies within a context of racial and cultural equity and inclusion. The …

Lost Causal: Debunking Myths About Causal Analysis In Philanthropy – With 2024 Prologue, Jewlya Lynn, Sarah Stachowiak, Julia Coffman

The Foundation Review

Editor’s Note: This article, first published in print and online in 2022, has been republished by The Foundation Review with minor updates. What if philanthropic evaluations told us that changes in the world had occurred, as well as how and why they occurred, including whether what foundations funded and grantees did contributed to those changes? What if evaluations made change pathways more visible, tested hypotheses and assumptions, and generated new insights based on what happened in the “black box” of systems change strategies? This type of learning comes from causal analysis — inquiry that explores cause-and-effect relationships. Yet currently in …

Goal-Free Evaluation: An Orientation For Foundations’ Evaluations – Updated 2024, Brandon W. Youker, Allyssa Ballard

The Foundation Review

Editor’s Note: This article, first published in print and online in 2014, has been revised for The Foundation Review with substantive additions, new examples, and minor updates.

Goal-free evaluation is a model in which official or stated program goals and objectives are unknown by the evaluator, serving as a counter to assessing impact solely according to goal achievement. Foundation-supported program evaluation, however, has historically focused on goal attainment as intuitively and inextricably linked to evaluation.

This focus has persisted despite the fact that goal-free product evaluations have been a norm for more than 75 years. Yet persuading funders to consider …

A Foundation’S Theory Of Philanthropy: What It Is, What It Provides, How To Do It – With 2024 Prologue, Michael Quinn Patton, Nathaniel Foote, James Radner

The Foundation Review

Editor’s Note: This article, first published in print and online in 2015, has been republished by The Foundation Review with minor updates.

This article argues that philanthropic endeavors should be undergirded by a theory of philanthropy. Articulating a theory of philanthropy is a way for a foundation to make explicit what is often only implicit, thereby enabling internal and external actors to pose and resolve significant questions, understand and play important roles more fully and effectively, and improve performance by enhancing alignment across complex systems.

A theory of philanthropy articulates how and why a foundation will use its resources to …

Passing In The Dark: Making Visible Philanthropy’S Hidden And Conflicting Mental Models For Systems Change, Jewlya Lynn, Julia Coffman

The Foundation Review

While the need for philanthropy to focus on systems change as a way to scale and sustain impact is now widely accepted, we see the sector largely failing to recognize that there are different mental models for how to change systems. Sometimes the approaches foundations use are based on competing mental models or models that are not a good fit for the systems, problems, strategies, or practices they are using.

We see two mental models for systems change being used in philanthropy: systems dynamics and systems emergence. Strategies that use the systems-dynamics mental model aim at points of high leverage …

Developing A Methodology For Evaluating The Sensitivity Of Rock Imagery Sites To Vandalism In Washington County, Ut, Erin C. Haycock

All Graduate Theses and Dissertations, Fall 2023 to Present

This study uses statistical analysis to examine the relationship between the characteristics of rock imagery (also known as rock art) sites and intentionally caused damages in Washington County, Utah. This project aims to create an index for public land managers to respond proactively to vandalism at rock imagery sites. Included here is an analysis of the severity and frequency of damage to the sites and an inventory of the types of site damage to determine the most common and destructive types of vandalism. Site attributes such as the number of figures in a panel, the type of images, and panel …

Evaluating An Integrated Science, Technology, Engineering, And Math/Computational Thinking Professional Development Program For Elementary Level Paraprofessional Educators, Aubrey A. Rogowski

All Graduate Theses and Dissertations, Fall 2023 to Present

For my dissertation, I looked at a training program one Utah school district used to teach paraprofessional educators science, technology, engineering, math, and computational thinking. Specifically, the program taught them about what computational thinking is and how they could use it when teaching science, technology, engineering, and math to students from kindergarten to sixth grade. While reviewing this program, I evaluated 1) The experiences the paraprofessionals had with the program, 2) Whether the paraprofessionals understood computational thinking, and 3) Whether the program prepared them to teach computational thinking to K-6 students.

I worked with eight paraprofessionals who participated in this …

Building A Roadmap For Web Archiving: Organizational Sustainability In An American Research University Library, Ruth E. Bryan, Emily B. Collier

Library Presentations

The presenters, archivists in an academic university Library, launched a web archiving program for a public university in the United States in 2018 with a three-year Archive-it contract. In the first six years of the web archiving program, we have laid the groundwork for an ongoing web archiving program through robust documentation built in anticipation of potential loss of resources, especially personnel. In this presentation, we report on a sustainability review of the program using a practical framework and a conceptual framework. The practical framework is the University of Pittsburg's Socio-Technical Sustainability Roadmap (https://sites.haa.pitt.edu/sustainabilityroadmap/). The conceptual framework is Kristin R. …

Evaluating The Organizational Advancement Of Diversity, Equity, And Inclusion In Extension Education, John M. Diaz, Lendel K. Narine, Cody Gusto

Journal of International Agricultural and Extension Education

In response to global trends and calls for greater inclusivity, the field of extension education has made significant strides towards embracing Diversity, Equity, and Inclusion (DEI) principles. This study explores the progress of DEI initiatives in the Cooperative Extension System (CES), which has aligned with industry groups, non-profit organizations, academia, and scientific societies worldwide. Through a multifaceted approach, CES has pursued strategies such as intercultural competence (ICC) training for professionals, diversity action plans, and culturally responsive teaching techniques. This study assessed the advancement of DEI in extension education using a survey of DEI specialists across extension organizations in the U.S. …

An Instrument To Evaluate 4-H Cloverbud Stem Programming, Alisha M. Hardman, Donna J. Peterson, Mariah S. Morgan, H. Elizabeth Solace

The Journal of Extension

Evaluation data is needed to demonstrate the impact of 4-H science, technology, engineering, and mathematics (STEM) programming on children and youth. However, collecting evaluation data from cloverbuds (ages 5-7) is particularly challenging given their developmental age. We adapted an observational Cloverbud evaluation tool to measure basic life skills, essential elements of positive youth development experiences, and, unique to this tool, STEM specific experiences. We developed the STEM items using educational science standards allowing Extension to document STEM in addition to youth development outcomes. Doing so, may increase the adoption of 4-H STEM programming by teachers in the school setting.

Evaluation Of The Heal™Ing Mental Health Program: A Prospective Cohort Study Of Short-Term Changes From A Physical Activity And Lifestyle Education Program For People With Mental Health Disorders Living In Rural Australia, Annette J. Raynor, Sophia Nimphius, Daniel Kadlec, Sally Casson, Caitlin Fox-Harding, Lauren V. Fortington

Research outputs 2022 to 2026

This study aimed to evaluate short-term outcomes of the HEAL™ing Mental Health program, an 8-week intervention for change in functional, behavioural and physiological health and wellbeing designed for people living with mental health conditions in rural or regional areas of Australia. A prospective cohort study was completed, reporting on 19 items (pre-program) and 15 (post-participation change), organised across seven domains. Participants took part in an Accredited Exercise Physiologist/Nurse led supervised group exercise (60 minutes) and healthy lifestyle education program (60 minutes). Separate linear mixed models with restricted maximum likelihood were used to examine the primary research question considering the effect …

Increasing Housing Stability Through State-Funded Community Mediation Delivered By The Massachusetts Housing Mediation Program (Hmp) Fy2023 Evaluation Report, Madhawa Palihapitiya, Karina Zeferino

Massachusetts Office of Public Collaboration Publications

The Massachusetts Housing Mediation Program (HMP) is a comprehensive statewide program that provides free housing mediation services as a tool to increase housing stability with the intention of preventing homelessness created by landlord-tenant disputes. It is administered by the Massachusetts Office of Public Collaboration (MOPC) at the University of Massachusetts Boston and deploys the community mediation system infrastructure with 11 Community Mediation Centers (Centers) participating and serving all 14 counties of the Commonwealth to provide free conflict resolution services for tenants and landlords/property managers with housing disputes at any stage, from the earliest point a problem occurs, up to, and …

Addressing Inconsistencies In Grading Practices., Thomas R. Guskey

Educational, School, and Counseling Psychology Faculty Publications

Coming to agreement about the purpose of grading and establishing clearer and more accurate reporting structures can pave the way for more learning-focused grading systems.

Look Beyond The Satisfaction Survey: A Framework To Evaluate Results Of Professional Learning., Thomas R. Guskey

Educational, School, and Counseling Psychology Faculty Publications

Professional Learning evaluations consider five crucial levels of data to determine effects on teachers' instructional practices and student learning outcomes.

The Value Of Descriptive, Multi-Level Rubrics, Thomas R. Guskey, Mctighe Jay, Susan M. Brookhart

Educational, School, and Counseling Psychology Faculty Publications

Single-point rubrics have become popular in schools, but they may be leading teachers astray.

Measures To Assess The Competence Of Supervisors Of Provisionally Licensed Mental Health Counselors, Jennifer S. Kennett

Antioch University Dissertations & Theses

Supervision is a foundational component of education and training for new counselors. Ensuring provisionally licensed supervisees receive adequate supervision is vital, given the variation in training and experience requirements from state to state. Therefore, the purpose of this study was to evaluate three measures of supervisor competence and experience, the Supervision Experience and Supervisory Competence Assessment (version 2; SE-SC), the Generic Supervision Assessment Test (GSAT), and the Brief Supervisory Working Alliance Inventory (BSWAI) for use in the counseling profession within the United States. The instruments were evaluated for reliability and face validity using a sample of 104 provisionally licensed counselors …