Social and Behavioral Sciences Commons^™

A Neighborhood-Based Family Center Redesign Process: Taking A Systems Perspective, Patricia Bowie, Richard Sussman

The Foundation Review

This article describes how the Hartford Foundation for Public Giving, with a subset of its grantees and their program recipients, teamed with the UCLA Center for Healthier Children, Families & Communities to redesign its evaluation process.

The foundation’s shift from traditional program evaluation to a more participatory, learning-focused approach resulted in new tools to assess variables that had been previously unexamined but were critical to program success.

This article examines the redesign process and those new tools – the data from which are being used to improve employee engagement and front-line practice as part of a cross-agency learning network – …

Marguerite Casey Foundation: Reflecting On 15 Years Of Philanthropic Leadership Through A Summative Evaluation, Mavis Sanders, Claudia Galindo, Luz Vega-Marquis, Cheryl Milloy

The Foundation Review

This article presents the findings of a summative evaluation of the Marguerite Casey Foundation that was conducted on the occasion of its 15th anniversary. The evaluation was designed to gauge stakeholders’ perceptions of the foundation’s operations to facilitate organizational learning. In sharing these results, the authors seek to elucidate the role of evaluation as a learning practice within the field of philanthropy.

The article describes the foundation’s organizational elements and evolution and discusses key themes that emerged from qualitative data collected from foundation leaders and staff, as well as findings from a survey of current grantees.

The article presents a …

Evaluation At Sunset: Considerations When Evaluating A Program As It Concludes, Blair Beadnell, Holly Carmichael Djang, Jan Vanslyke, Barbara Andersen

The Foundation Review

While the benefits of beginning evaluation efforts at a program’s inception are well known, for a variety of reasons many organizations are unable to do so and instead begin these efforts closer to a program’s conclusion.

Previously reported findings from a sunset evaluation of the Orfalea Foundation’s School Food Initiative showed positive outcomes of the initiative’s activities and provided recommendations for organizations interested in engaging in similar efforts. Because the evaluation was begun as the foundation’s activities were winding down, it required creative design approaches.

This article uses the evaluation of the Orfalea Foundation’s initiative to provide a case example …

End-Game Evaluation: Building A Legacy Of Learning In A Limited-Life Foundation, Ashleigh Halverstadt, Benjamin Kerman

The Foundation Review

This article shares the emerging hypotheses of two foundations, The Atlantic Philanthropies and the S. D. Bechtel, Jr. Foundation — each four years from sunset — about the opportunities and challenges for evaluation in the limited-life context.

Few, if any, of the problems philanthropy seeks to address can be solved within a brief, defined time frame. Limited-life foundations can only strive to move the ball down the field before they sunset, and then enlist others to carry the work forward. Given this reality, these foundations are obligated to make a deliberate effort to share what they have learned with the …

Designing Technical-Assistance Programs: Considerations For Funders And Lessons Learned, Jennifer Lyons, Sheila Dunleavy Hoag, Cara Orfield, Sonya Streeter

The Foundation Review

As foundations continue to provide grantees with technical assistance in addition to financial support, it is important to understand what works well, for whom, and in what circumstances. This reflective practice article aims to help funders who have identified a problem amenable to technical assistance to develop a strong program by providing support to a group of organizations addressing similar problems or by providing customized individual support.

Drawing on insights from evaluations of two technical-assistance programs, this article recommends five key issues for funders to consider when offering such a program: whose priorities will shape the agenda, how group composition …

How Do You Measure Up? Finding Fit Between Foundations And Their Evaluation Functions, Julia Coffman, Tanya Beer

The Foundation Review

As the number of foundations has grown, the philosophies and ways of working across the sector have diversified. This variance means that there is no one right model for how a foundation’s evaluation function should be designed. It is imperative for a foundation to think carefully about how the structure, position, focus, resources, and practices of its evaluation function can best fit its own needs and aspirations.

This article focuses on questions foundations can ask to assess that fit, and the specific considerations that can inform these decisions. It draws on 2015 benchmarking research conducted by the Center for Evaluation …

Key Ingredients For School Food Systems: An Evaluation Of The Orfalea Foundation’S School Food Initiative, Holly Carmichael Djang, Barbara Andersen, Tatiana Masters, Jan Vanslyke, Blair Beadnell

The Foundation Review

In 2007, the Orfalea Foundation launched a nine-year strategic effort aimed at empowering public school districts in California’s Santa Barbara County. The purpose was to implement food-service operations that offered nourishing meals and to create a culture that prioritized the health and wellness of children and families.

The initiative, which involved 84 schools and more than 50,000 students, assessed the capacity of the county’s school food services, including existing skill levels and equipment needs. Striving to tailor programming to specific needs, the foundation emphasized stakeholder involvement throughout the process.

The foundation took a number of steps to understand the value …

Developing A Framework For Grant Evaluation: Integrating Accountability And Learning, Shelley Scherer

The Foundation Review

Despite broad consensus among foundations on the value of capturing grant outcomes, there is no consensus on what to evaluate and how to define success, which makes it difficult for staff and grantees to navigate and apply multiple interpretations of evaluation “best practices.”

This article presents three questions designed to help foundations develop a framework for grant evaluation that reflects their beliefs about accountability and learning, balances evaluation costs and benefits, acknowledges the diversity of grants within the foundation’s portfolio, and allows their grantees to understand the foundation’s expectations for evaluation reporting.

A key takeaway from this article, drawn from …

The Unified Outcomes Project: Evaluation Capacity Building, Communities Of Practice, And Evaluation Coaching, Jay Wade, Leanne Kallemeyn, David Ensminger, Molly Baltman, Tania Rempert

The Foundation Review

Increased accountability from foundations has created a culture in which nonprofits, with limited resources and a range of reporting protocols from multiple funders, struggle to meet data-reporting expectations. Responding to this, the Robert R. McCormick Foundation in partnership with the Chicago Tribune launched the Unified Outcomes Project, an 18-month evaluation capacity-building project.

The project focused on increasing grantees’ capacity to report outcome measures and utilize this evidence for program improvement, while streamlining the number of tools being used to collect data among cohort members. It utilized a model that emphasized communities of practice, evaluation coaching, and collaboration between the foundation …

Inquiry: A Participatory Approach For Understanding Stakeholder Perceptions, Matthew Militello, Christopher Janson, Drew Tonissen

The Foundation Review

This article addresses two important and elusive issues for funded projects: quantifiable measures and deep understandings of participant perceptions.

It describes the development of the InQuiry evaluation tool, which combines Q methodology (factor analysis process to quantify perceptions) with a qualitative participatory approach. InQuiry generates both quantified metrics of what participants believe about a given topic and also a rich narrative of why participants think the way they do. These data yield metrics for understanding fidelity, outcomes, and impacts.

Beginning with the history of a program funded by the W.K. Kellogg Foundation, this article also illustrates the tool’s usefulness. The …

A Foundation's Theory Of Philanthropy: What It Is, What It Provides, How To Do It, Michael Quinn Patton, Nathaniel Foote, James Radner

The Foundation Review

This article argues that philanthropic endeavors should be undergirded by a theory of philanthropy. Articulating a theory of philanthropy is a way for a foundation to make explicit what is often only implicit, thereby enabling internal and external actors to pose and resolve significant questions, understand and play important roles more fully and effectively, and improve performance by enhancing alignment across complex systems.

A theory of philanthropy articulates how and why a foundation will use its resources to achieve its mission and vision. The theory-of-philanthropy approach is designed to help foundations align their strategies, governance, operating and accountability procedures, and …

Network Evaluation In Practice: Approaches And Applications, Madeleine Taylor, Anne Whatley, Julia Coffman

The Foundation Review

As more funders support networks as a mechanism for social change, new and practical knowledge is emerging about how to build and support effective networks. Based on extensive review of different types of networks and their evaluations, and on interviews with funders, network practitioners, and evaluation experts, the authors have developed an accessible framework for evaluating networks.

This article describes the evaluation framework and its three pillars of network assessment: network connectivity, network health, and network results.

Also presented are case examples of foundationfunded network evaluations focused on each pillar, which include practical information on evaluation designs, methods, and results, …

Grey Matter(S): Embracing The Publisher Within, Lisa Brooks, Gabriela Fitz

The Foundation Review

Most foundations don’t think of themselves as publishers, yet many of them act as such – making information available by funding research and publications, or by authoring their own. And failing to think of these activities as publishing efforts has serious consequences for shared learning in the social sector.

The shift toward knowledge-sharing strategies and approaches that embrace new search technologies, the logic of open access and open source, and the realities of the Internet as a largely decentralized and dynamic selfpublishing space offers the possibility of coordinating publishing efforts, and possibly agreeing to the use of shared practices that …

Understanding Philanthropy Consulting: A Tool To Identify The Roles And Capabilities Needed From External Support, Brian Leslie, Kelsey Noonan, Clint Nohavec

The Foundation Review

This article categorizes the distinct roles played by philanthropy consultants and presents a tool and framework for charitable foundations to identify and evaluate the roles and capabilities they need from those consultants.

The article categorizes seven capability areas, from strategy setting to talent development, that are core to all foundations. Then, it identifies trigger points within these capability areas that lead foundations to undertake projects that may require outside support. Third, the article maps the capabilities that foundations consider in determining whether and how to engage philanthropy consultants.

The resulting tool is scalable and broadly applicable, providing foundation staff and …

Redefining Expectations For Place-Based Philanthropy, Katelyn Mack, Hallie Preskill, James Keddy, Moninder-Mona K. Jhawar

The Foundation Review

This article discusses how The California Endowment has used a midcourse strategic review to refine Building Healthy Communities, aiming to provide insight for other place-based initiatives and to add to the body of knowledge about how to support transformative community change.

With Building Healthy Communities, the endowment is taking a new approach to community change using a dual strategy to build community capacity in 14 places and scale the impact of its local efforts through statewide policy advocacy and communications. In 2013, it commissioned a strategic review to reflect on what it has learned from the first three years of …

Maximizing Return: An Evaluation Of The Walton Family Foundation’S Approach To Investing In New Charter Schools, Matthew Carr, Marc Holley

The Foundation Review

The Walton Family Foundation’s social-impact goals include reform of the American K-12 education system by increasing the number of highquality schools available to low-income students. One of the foundation’s signature strategies toward this end is to support charter schools.

This article presents the findings of a study that suggests the foundation’s investment approaches to charter school startups have been successful in supporting the creation of high-quality seats for low-income students. Specifically, the foundation has invested in charter schools where test-score performance has shown greater improvements than at local district schools and charter schools that have not received foundation funding.

These …

The Impact Grants Initiative: Community-Participatory Grantmaking Modeled On Venture Philanthropy, Adin Miller, Elisa Gollub, Ilana Kaufman, Adina Danzig Epelman

The Foundation Review

· The Jewish Community Federation and Endowment Fund (JCF) launched the Impact Grants Initiative (IGI), a model of grant making based on venture philanthropy, but offering high engagement opportunities for previously unaffiliated local donors and community leaders.

· Before adopting the IGI model, the JCF used a community-participatory grantmaking approach that had become stale in engaging its donors, community leaders, and professional staff. Younger existing and potential donors were developing interests in documented outcomes, metrics, and impact, and those interests did not align with JCF’s grantmaking approach.

· IGI builds on the concepts of venture philanthropy, combining theories and techniques …

Climbing The Mountain: An Approach To Planning And Evaluating Public-Policy Advocacy, Sam Gill, Tom Freedman

The Foundation Review

· This article proposes a new methodology for planning and evaluating public-policy advocacy. The methodology is designed around a series of stages, each with a different set of strategic planning and assessment requirements.

· The article suggests that both planning and evaluative approaches that fail to take account of the necessary stages required to develop and then implement an advocacy strategy will likely assign the wrong indicators of success.

· This analysis is based on direct experience working with both policy processes and a wide range of foundations and nonprofits that have invested in public-policy advocacy, including the Rockefeller, Ford, …

Raising The Bar – Integrating Cultural Competence And Equity: Equitable Evaluation, Jara Dean-Coffey, Jill Casey, Leon D. Caldwell

The Foundation Review

· Whether implicit or explicit, social justice and human rights are part of the mission of many philanthropies. Evaluation produced, sponsored, or consumed by these philanthropies that doesn’t pay attention to the imperatives of cultural competency may be inconsistent with their missions.

· The American Evaluation Association’s Statement on Cultural Competence provides those who produce, sponsor, and use evaluation an opportunity to examine and align their practices and policies within a context of racial and cultural equity and inclusion. The use of such a lens is paramount when evaluating a program whose goals touch on issues of equity or inclusion. …

Survey Instruments Used To Evaluate Foundation-Funded Nonprofit Capacity- Building Programs: Considerations For Organized Philanthropy, Catherine H. Brown

The Foundation Review

· Alongside a growing interest in nonprofit capacitybuilding programs has come a growing concern with the impact of these programs, especially by organizations that fund them. This article describes how the McKinsey Organizational Capacity and Assessment Tool and, to a lesser extent, the Abt Associates survey have been used to assess changes in nonprofit capacity as part of nonprofit capacity-building programs.

· Drawing on field experience with both survey instruments in the context of a foundationfunded nonprofit capacity-building program, this article compares the respective benefits and costs of these instruments from the perspective of evaluators as well as survey respondents. …

If You Build It, They Will Come: Creating The Space And Support For Real-Time Strategic Learning, Jewlya Lynn, Rebecca Kahn, Phillip Chung, Scott Downes

The Foundation Review

· Learning is a key tool for foundations seeking to improve their effectiveness, and they are beginning to use evaluation to learn about and improve their strategies. The Colorado Trust took this a step further and integrated strategic learning into a grant strategy, including supporting learning coaches for 14 of their grantees.

· The strategic learning framework consisted of three steps: systematic data collection, collective interpretation of information, and the use of that interpretation to improve strategies.

· This article reviews four of the cases, including three grantees and the foundation as a case, identifying methods of learning and resulting …

Goal-Free Evaluation: An Orientation For Foundations’ Evaluations, Brandon W. Youker, Allyssa Ingraham

The Foundation Review

· Goal-free evaluation (GFE), in program evaluation, is a model in which the official or stated program goals and objectives are withheld or screened from the evaluator.

· Several obstacles must be overcome in persuading foundations and programs to consider GFE as a viable option, because both tend to view goal attainment as intuitively and inextricably linked to evaluation.

· This article presents the case for GFE as a perspective that belongs in a foundation’s toolbox. In particular, this article demonstrates GFE’s actual use, highlights aspects of its methodology, and details its potential benefits.

The Regulation And Control Of Bail Recovery Agents: An Exploratory Study, Brian R. Johnson, Ruth S. Stevens

Peer Reviewed Publications

This article explores the current status of the licensing and regulation of bail recovery agents in the United States. By reviewing state legislative and administrative codes in all the 50 states, this study found that 24 states control bail recovery agents through licensure or the imposition of other occupational regulations. These state controls include age, criminal history, and pretraining and educational requirements; some states also require continuing education and training for licensure and/or regulation. In contrast, 18 states have no licensing or other occupational requirements for bail recovery agents. These findings raise questions about the actual utility and function of …

Benchmarking Evaluation In Foundations: Do We Know What We Are Doing?, Julia Coffman, Tanya Beer, Patricia Patrizi, Elizabeth Heid Thompson

The Foundation Review

· Evaluation in philanthropy – with staff assigned to evaluation-related responsibilities – began in the 1970s and has evolved, along with philanthropy, in the four decades since. What has not changed, however, is a regular questioning of what foundations are doing on evaluation, especially since the world of philanthropy regularly shifts, and changes in evaluation resourcing and positioning tend to soon follow.

· This article presents new findings about what foundations are doing on evaluation and discusses their implications. It is based on 2012 research that benchmarks the positioning, resourcing, and function of evaluation in foundations, and follows up on …

Eyes Wide Open: Learning As Strategy Under Conditions Of Complexity And Uncertainty, Patricia Patrizi, Elizabeth Heid Thompson, Julia Coffman, Tanya Beer

The Foundation Review

· Foundation strategy is hampered by a failure to recognize and engage with the complexity and uncertainty surrounding foundation work. This article identifies three common “traps” that hinder foundation capacity to learn and adapt: 1) linearity and certainty bias; 2) the autopilot effect; and 3) indicator blindness.

· This article urges foundations to alter their mindset, questions, and processes to foster a more committed approach to strategy and adaptation. In essence, it argues for learning as strategy.

· This article draws on literature from systems theory, business strategy, and philanthropic practice as well as data from foundation benchmarking surveys.

Practice, Practice, Practice: Preliminary Findings From An Evidence-Based Practice Funding Initiative At The Peter And Elizabeth C. Tower Foundation, Nicholas G. Randell

The Foundation Review

· The Tower Foundation supported a five-year initiative to support the implementation of evidencebased practices (EBP). The average award was a three-year award of $84,050.

· The underlying grantmaking theory of change was that behavioral health providers could bring empirically tested protocols to their communities and sustain them over time if supported by long-term funding to support the real costs of implementation (e.g., training, technical assistance, adherence to program protocols, and cultural change).

· Grantees cited the high cost of training, certification, and recertification – especially in the face of high staff turnover – as a primary challenge to implementing …

Evaluation For Models And Adaptive Initiatives, Heather Britt, Julia Coffman

The Foundation Review

· Although there has been a growing emphasis on use of experimental designs in evaluation, there is also increasing agreement that evaluation designs should be situation specific.

· The nature of the program is one of the key factors to consider in evaluation design.

· Two types of programs – models, which provide replicable or semi-standardized solutions, adaptive initiatives, which are flexible programming strategies used to address problems that require unique, context-based solutions – require different evaluation designs.

· Evaluation of models requires understanding the stage of development of the model program, with summative evaluation done only when the model …

Dimensions Of Change: A Model For Community Change Efforts, Jara Dean-Coffey, Nicole Farkouh, Amy Reisch

The Foundation Review

· The Dimension of Change Model (DOCM), developed by the authors, is offered as a potentially useful tool for foundations, government, bodies, consultants, coalitions, and even individual organizations that are initiating or engaged in substantive efforts to bring about community change.

· The dimensions contained in the model - structure, parameters, intention, approach, and people - offer a frame for addressing key aspects that emerge from the literature as fundamental to all change efforts. The model is offered as a way to design, implement, adapt, and evaluate change initiatives.

· The work of First 5 Marin Children and Families Commission …