Srik Gopal, who co-leads social sector consulting company FSG’s Strategic Learning and Evaluation practice, recently blogged at SSIR about evaluating complex social initiatives. He discusses two funders of large-scale, complex initiatives and their attempts to change the way they evaluate success, because as he says:
They are building on the recognition that a traditional approach to evaluation—assessing specific effects of a defined program according to a set of pre-determined outcomes, often in a way that connects those outcomes back to the initiative—is increasingly falling short.
He continues, stating that because complex systems are always changing, the evaluation tools used need to be “adaptive, flexible and iterative.” I’m a big fan of the idea that we need to get beyond simplistic “cause and effect” models of evaluation, which often ignore context. This is particularly true when dealing with complex initiatives and initiatives launched in unstable/changing or multiple environments – it’s important to go beyond whether something works in each location and get into why it does or doesn’t work.
The blog post gives a good feel for the flavor of the work that FSG has done to try to recognize this complexity and deploy tools that can capture the information needed to create evaluations that can adapt to changing circumstances and capture not just outcomes but relationships and system dynamics, including a chart that shows 3 of their 9 propositions for evaluating complexity alongside what those propositions mean in the real world and some existing tools that can be used in order to capture the needed info.
However, this blog post doesn’t go beyond giving some flavor. It’s worth clicking the link they offer and going through the free signup process to see the full 30+ page report they offer. This includes all 9 propositions, both in snapshot form (page 5) and with full descriptions and case studies. There is also a chart similar to the one in the blog post on pp 31-32 that shows each proposition alongside a brief description of the proposition and some helpful evaluation tools/methods. Beyond the charts and descriptions, there are also 3 case studies.
This looks to be a great resource for those looking to design a thoughtful evaluation process for complex initiatives, as well as a way to re-think what organizations may be hoping to learn and capture for even more simple evaluation practices.
business *is* personal 