“The growing magnitude and complexity of socioeconomic problems facing societies throughout the world transcend the capacities of individual organizations and sectors to deal with them adequately.”
Austin and Seitanidi, 2012
Few would argue with the perspective of James Austin and May Seitanidi, particularly when looking at the last few years of world events. Collaborative action is flourishing – around the world. There is a growing number of multi-stakeholder partnerships, platforms, alliances, and coalitions formed to tackle complex problems, and transform the systems that underlie them. In the flurry of designing partnerships, bringing the right people together, and supporting innovative collaborative ventures, we’ve squeezed out the space required for critically reviewing our progress – for examining and reflecting on what our partnerships have done, and the differences they have made.
As a result, many partnerships struggle to demonstrate their progress and achievements, pivot or course correct when circumstances change, or secure financial support for their ongoing efforts.
But what if learning and evaluation wasn’t an afterthought? What if learning and evaluation was seen and recognised as an essential requirement for a partnership to deliver on its goals, and to contribute meaningfully to the complex systems it intends to change?
Change and complex systems
How complex systems change, and how to bring about such change, has been a focus for many, including those working in partnerships. The Intervention Level Framework (built on earlier work by Donna Meadows), identifies 5 levels, or places to intervene, in complex systems:
- Paradigm
- Goals
- System Structure
- Feedback
- Structural Sub-system Elements
Starting at level 5 and moving to level 1, the effect on the system increases dramatically, yet so too does the challenge of bringing about change.
For example, changing the physical features of a sub-system (such as a building, a road, a school, or a sewage treatment facility) is often more attainable than shifting the underlying beliefs that influence how complex systems respond to the climate emergency. Yet changing the latter, is likely to yield more profound systems change.
Learning and evaluation as a key tool for partnerships to effect systems change
Multi-stakeholder partnerships hold promise as vehicles for influencing each of the ILF’s 5 levels. Indeed, interest in partnerships has risen because of their potential to influence systems, and therefore to address the challenges enshrined in the SDGs, and other ambitions. And they do this through a variety of ways – fostering new connections among system actors; creating, gathering and mobilizing knowledge and evidence; shifting the narratives that shape public debates, discourse and decisions; influencing the flow of financial and other resources; and providing space for innovations to be seeded, created, tested and launched.
In doing so, partnerships like other collaborative ventures, create value for the system (as above), as well as value for its individual partners. Partners may derive different types of value from a partnership, including an enhanced ability to achieve their own organization’s mission and goals; greater social or political capital; increased legitimacy; market advantages; and greater influence and strategic positioning.
And to generate these dual layers of value – value for the system, and value for individual partners – partnerships require a few features, such as trust among partners, reciprocity, clear decision making, a shared vision for change, and mutual accountability.
To this list, we’d add a commitment to continuous learning and evaluation.
Moving toward a culture of learning and evaluation for multi-stakeholder partnerships
Like the feedback element of the ILF, continuous learning and evaluation for partnerships is a powerful lever for improving the strength of the partnership, and its ability to generate value for its partners, as well as the system it is seeking to influence. Learning and evaluation focuses on the structural features of the partnership, the activities undertaken as part of the partnership, as well as the outcomes it generates, for its partners and the systems of interest. Learning and evaluation also examines the linkages between these elements – what actions lead to what outcomes, how and why.
Increasingly, partnerships are investing time, energy and resources into learning and evaluation systems. Gavi – The Vaccine Alliance – provides a powerful example. Gavi was established in response to a complex problem: how to increase childhood immunisation coverage globally, and across disease categories. Since its beginnings, the influence of Gavi has been profound: on how pharmaceuticals are discovered and delivered, how immunisation programs have been financed and funded, and ultimately on the number of children vaccinated through routine programs. With learning and evaluation at the centre of what the Alliance does, Alliance members and the Secretariat are able to continually learn about and adapt to the challenges of the day (e.g. COVID-19), and to develop, test and deliver innovative solutions.
Benefits and challenges of learning and evaluation
The benefits to partnerships such as Gavi, that invest in learning and evaluation is clear. Partnerships are able to demonstrate their progress and achievements, both to their partners, as well as important system stakeholders. Insights gained from continuous learning and evaluation helps partnerships to course correct, pivot and change – adapting to new contexts and conditions. Evidence of progress supports efforts to secure funding and financial support for the work of the partnership, as well as attract and retain partners who are committed to a shared vision.
And learning and evaluation generate knowledge for our global efforts to be better partners, to create better partnerships, and implement more effective systems change efforts.
Yet the pathways to a culture of continuous learning and evaluation are paved with obstacles: fear and lack of money, time, enthusiasm, skills, and capacities being among those commonly encountered. There’s no one way to overcome these obstacles, and different partnerships will require different actions at different points. In our experience, a Learning Partner is a useful addition to the team. Learning Partners come in different forms – from an individual, to an organization – and are committed to championing critical reflection, shared wisdom, a culture of inquiry, and valuing of evidence (in a range of forms). Investing in a Learning Partner is an important signal to the partnership and its systems of interest of a sophisticated and energised partnership – where curiosity and improvement are rewarded and encouraged.
Where to start
We encourage all partnerships to strive toward a culture of inquiry that supports learning and evaluation. And to do so, those working in partnership might consider the following three actions as places to start:
- Identify the pain points in your partnership where learning and evaluation may help: these may relate to challenges in communicating your partnership’s unique value and past achievements; securing financial resources for the work of the partnership; making decisions about what to do more of, less of or stop; or attracting new partners.
- Find and foster champions: go with where there is energy in your partnership. Find those partners who are interested, capable or engaged in learning work, and bring them together.
- Start small: gather what information you have at hand; celebrate the successes of what the partnership has achieved; find and share examples from other partnerships who are invested in learning and evaluation.
Find out more about learning and evaluation in partnerships and platforms and get in touch with us at dayfourprojects.com.
Dr Cameron Willis
Cameron has worked in research, policy and practice settings since 2005, exploring how to design, implement and evaluate complex interventions that have genuine impact.
He has a PhD in Epidemiology and is comfortable digging into detail and applying his qualitative and quantitative data analysis skills. Cameron designs projects that help capture useful information for answering difficult questions: questions like what works, for whom, and under what conditions.
Dr Nic Vogelpoel
Nic is fascinated by the complexity of human health and by the places, spaces and experiences that affect us all. He has worked in research, policy and practice across health and human services in government and for national and international charities, investigating ways to design, implement and evaluate integrated health interventions.
Nic has a PhD in Public Health and is committed to promoting understanding of multi-stakeholder partnerships. He has worked with the Wellcome Trust, World Economic Forum, UKRI and held senior management and operational positions with Sense, Cerebral Palsy League and Movember Foundation and board director positions with Amnesty International UK and Engage, the visual arts charity.
