Lessons Learned from Answering the Million Dollar Question

Abstract

I want to make a difference.
I've got a million dollars to give.
How should I spend it to make the greatest social impact?

Since our founding in Spring 2006, the Center for High Impact Philanthropy has worked to answer that “million dollar question”. While straightforward to pose, the question about how to make the greatest social impact can be maddeningly difficult to answer in a way that is both informed by the evidence and actionable. Here’s what we’ve learned so far:

Keep your eyes on the prize: impact

Impact-driven decision-making does not start with the nonprofit. It does not start with the donor. It starts with the social impact.

Before embarking on any analysis, we ask, “What is the desired change-- is it a meaningful one?” We insist on clarity on the impact being sought – even if that impact is not yet easily or precisely measurable.

For example, we recently launched an effort focused on identifying how philanthropic capital can improve teaching quality. Many nonprofits, philanthropists and policy makers have prioritized this topic, and rightly so. Conclusive evidence and practical wisdom points to teaching quality as the biggest in-school factor affecting student outcomes.

Yet if you google “defining teacher quality” or “defining teacher effectiveness”, you will read about requirements for state certification and the importance of student test scores. Quickly lost is the impact that parents, teachers, and employers care about: the learning and development that leads to success in adulthood.

Existing metrics such as test scores are often proxies—-indicators of the harder-to-measure impact we seek. If we don’t stay laser-focused on impact, we risk dummying down aspirations to fit the currently available metrics. Just as important, we risk limiting solutions to current philanthropic and nonprofit practices, thereby missing opportunities for innovation.

Link considerations of cost and impact

Too often, people focus exclusively on cost (“Look at how much this program spends on overhead!”)or exclusively on impact (“Look at how much good we’re doing!”).

Costs and expense ratios are meaningless if not understood in the context of what that spending creates. A focus on impact, without consideration of costs, prevents donors and nonprofits from:

• asserting meaningful benchmarks (“this is what we’ve found it costs to increase child survival rates in this village”);
• celebrating well-earned efficiencies (“by adopting this practice, we’ve managed to increase the rates of child survival at this same cost”); or
• identifying opportunities to do more good with the same resources(“if we could figure out how to lower our transportation costs, we might be able to increase rates of survival at the neighboring village as well”).

Linking considerations of cost and impact enables the kind of ongoing problem solving needed to bring change.

When it comes to metrics and data, not all sectors are created equal

From the beginning, our Center has focused on more than one impact area. We have developed philanthropic investment guides that address U.S. education, global public health issues, and some of the most urgent domestic needs brought on by the current economic recession. Our cross-sector perspective enables us to see firsthand the disparity of information and analytic tools in different sectors.

For example, in our global public health work, we found multiple aggregators of randomized control trial impact studies, three centers focused on cost-effectiveness analyses, and even a “lives saved” calculator that pulls current, good quality, regional epidemiological information for use in estimating the lives saved by a given bundle of interventions.

In contrast, in our U.S. education work, we found that reliable figures for the number of high school dropouts nationwide was a fairly recent phenomenon. Randomized control trial studies – the gold standard for impact assessment – are few and far between. Cost-effectiveness analyses are even rarer.

Where good data and rigorous analyses are available, we’ve gratefully used them to inform philanthropic decision-making. When all we have are high-level estimates, we use back-of-the-envelope calculations to approximate impact. Both can be useful, and we are keenly aware of the differences. Donors and nonprofits need to be, too.

There will undoubtedly be many more lessons ahead for the Center. We look forward to sharing them and learning with and from others similarly focused on answering the million dollar question.