4.2: Measuring Results

Unit 4, Chapter 2.

Measuring Results

In this chapter:
Foundations of Causation | Framing Social Impact | Measuring Blended Value | Doing Evaluation & Assessment | Unintended Consequences

FOUNDATIONS OF CAUSATION

TED - Ionica Smeets - The danger of mixing causality and correlation (video)
A brief overview of bad inference: reverse causation, omitted variables, and the real-life consequences of misinformation

Poverty Action Lab - Why Randomize? (webpage)
Explains the need for a counterfactual and the challenges in creating one. Also contains a link to a table of common evaluation methods.

FRAMING SOCIAL IMPACT

SSIR - Advancing Evaluation Practices in Philanthropy (articles)
Beginning with a general overview, "Framing the Issue", this collection of articles profiles the different ways that foundations approach the question of impact evaluation and outcome assessment

SSIR - Measuring Advocacy, Yes We Can! (article)
Five measurement strategies that enable rapid course correction and continuous improvement.

SSIR - Most Charities Shouldn't Evaluate Their Work (article)
Often, the ideas used by charities don't need to be evaluated again, because they've been amply evaluated already. Charities—and funders and others—can use those existing evaluations to choose effective interventions. All the charity then needs to do is run the programs well. Charities need to be skilled at implementation—at running breakfast clubs, or community transport, or drug rehab centres. By the long-established law of comparative advantage, we should let them do what they're best at and not ask them also to get good at the totally unrelated skills of social science research.

SSIR - Humans: There's No App for That (article)
What the hype obscures, is that data and technology are tools, two of many, that make up a "data system"—a human-centered process that takes into account the reasons for collecting data in the first place and the learning that might come from their analysis. It's an inconvenient truth that most social impact organizations couldn't accurately report the number of people they reach, let alone make claims about the significantly more complicated concept of "impact."

Forbes - Size Isn't Everything When Assessing Impact (article)
Scale is sexy. In social ventures, it is the "mine is bigger than yours" metric. But, if your goal is to do well by doing good, it is critical to consider the "scope" of a venture, not just its scale. Failure to do so could mean that your venture enriches you while impoverishing those it was supposed to help.

MEASURING BLENDED VALUE

Social Return on Investment (SROI): Exploring Aspects of Value Creation (report)
In response to the lack of adequate metrics measuring value creation by nonprofits, this article addresses issues related to the understanding and measurement of Social Return on Investment (SROI).

Breakthroughs in Shared Measurement and Social Impact (report)
This paper, based on six months of interviews and research by FSG Social Impact Advisors, examines twenty efforts to develop shared approaches to performance, outcome, or impact measurement across multiple organizations. The accompanying appendices include a short description of each system and four more in-depth case studies.

DOING EVALUATION & ASSESSMENT

Dodge Assessment Initiative Online Workshop (guide)
Through the Assessment Workshop led by former Dodge President and CEO David Grant, non-profit organizations tackle a difficult and important topic: improving the performance of their organization through a more thoughtful, sustained and sophisticated approach to assessment.

Kellogg Foundation - Logic Model Development Guide (guide)
Nonprofits today are being pressed to demonstrate the effectiveness of their program activities by initiating and completing outcome-oriented evaluation of projects. This guide was developed to provide practical assistance to nonprofits engaged in this process.

SSIR - Six Theory of Change Pitfalls to Avoid (article)
Simply putting boxes and lines down on paper doesn't guarantee that your organization will make better decisions. In our ten-plus years of supporting clients in theory of change work, we've found six major pitfalls that, if avoided, can help nonprofits create actionable theories of change: 1) confusing accountability with hope, 2) creating a mirror instead of a target, 3) failing to take external context into account, 4) not confirming the plausibility of your theory, 5) creating a theory that isn't measurable, 6) assuming you've figured it all out.

Examples of social venture impact reports: One Acre Fund, Nyaya Health, Pencils of Promise.

UNINTENDED CONSEQUENCES

TED - David Damberger - What happens when an NGO admits failure (video)
International aid groups make the same mistakes over and over again. David Damberger uses his own engineering failure in India to call for the development sector to publicly admit, analyze, and learn from their missteps.

Unreasonable Institute - Our Failures (webpage)
"We believe deeply in the work that we do. And we're proud of it! At the same time, we believe deeply in the value of militant transparency. As a team, we've experienced a number of failures and false starts over the past few years. And to keep us humble and to ensure that we always learn from them, we enumerate them here."  

Anything missing?  Broken link?  Let us know at sii@brown.edu!

</