Learning in Public Policy: Review of the OECD Guide to Peer-to-Peer Learning

Public Policy and the Ongoing Flint Water Crisis: Community Perspectives — Source: Gerald R. Ford School of Public Policy

What  if recipients of foreign aid would be directly involved in structuring  the way the money is spent? What would policy makers learn about the  internet as a public good, if they joined in on hacker community events?  How can I work together with my neighbours and my municipality to tackle the problem of speeding traffic in my street?

With such questions in mind, I engaged with“A Guide to Peer-to-Peer Learning”.  This guide, targeting learners, and facilitators in the public policy  space, is a collaboration between Matt Andrews, professor of Public Policy at Harvard University, and Nick Manning of the World Bank, and .

The reason they’ve compiled a guide is that they see

“a  strong current interest to shift the focus for achieving improvements  in public organisations and in public service delivery from  pre-determined solutions to more applied approaches for supporting  reforms in contested and complex contexts”

The  guide gives an interesting look at how peer learning is applied in  policy making. As part of making the guide, the authors have studied 52  initiatives where civil servants were in some way linked to their civil  servant peers in other contexts to learn from each other about policy  issues relevant to them.

Based  on these initiatives, and their own experience in peer learning  facilitation, the authors deconstruct several phases of a peer learning  program, which they present as the backbone of peer learning program  design:

Phase 1a. Curation of topics,…
The first step is to understand what the important topics are within a  group of learners. Here it is essential to base the learning needs  assessment on people, and what those people are currently doing.

The  guide is a little light on examples, but we’ve found it works well to  think of this like brokering connections around topics of active  intrerest to people. For larger unconferences, we gather different  community ambassadors to find common topics. For smaller, more in-depth  programs, we have 45-minute ‘calibration’ calls with every participant,  to understand their world and challenges.

Phase 1b …and then engagement
Once  you understand the topics, then you can find ways to draw in wider  group of peers, and engage them in the program to start their learning.

The authors only mention the importance of appropriate formats, and tools  to support engagement, but unfortunately provide no guidance in this  publication. It’s solid advice though, that’s easy to underestimate. For  example, it might seem simple and easy to gather a group of 6  entrepreneurs for dinner and have them advice each other, but that often  ends up with one person dominating the conversation. Structured formats (like Fixer Sessions - soon to be shared as part of our Peer Learning facilitation kits) have been honed to allow delegating complex problems to a  multi-disciplinary team of experts, with consistently actionable results.

Phase 2. Ongoing learning, and engagement.
Keeping a rhythm of engagement going is important for learner progress. However the authors' research for the guide finds that this is rarely done. Most peer  learning initiatives are one-off. Another interesting point about this  phase is that initiatives rarely measure the things that peer learning  delivers on (eg. relationship building), but rather measure activities,  and formal products that result from the learning.

When we built the first Lean Startup communities at Leancamp, we had this  exact struggle. The reason is that participants become more aware of the  ways they can pull value from the community, and how to deliver value  themselves to others. This experience is lost when the event is  organised as a one-off, but it was hard to keep momentum in each of our  30 cities. You make all the investments in get the community together,  but there’s no awareness building with the participants around  optimising the value that the format provides.

Phase 3. Achieving learning outcomes.
For  sustained learning, and getting organisational support, the authors  argue that comparison of the learning achievement of programs with  predefined learning goals important, but comparison is rarely performed.

“Examples  of more successful peer learning initiatives are clear about the kinds  of peer sharing and learning they hope to generate. However, most peer  learning engagements do not specify the details of what kind of learning  is expected or hoped for.”

Feedback,  and course correction based on feedback has become the basis of how we operationalise peer learning at Source Institute. For different program design, we always tailor feedback systems to support the direction towards the overall  program goals. For instance in the Leaders in Innovation Fellowship for the Royal Academy of Engineering, we’ve set up feedback with the  learners in such way, that we can monitor whether we’re on track to addressing academic engineers’ entrepreneurial mindset during the delivery. How to really measure if learning happened is one of our main challenges at Source.

Phase 4. Diffusion from the individual to the organisation
In  this last phase the focus is on promoting the learning structure that  learners have gone through outside of their organisation, and applying  those inside.

Our Source Institute alumni are instrumental for propagation of peer learning. We involve alumni as mentors for new participants. For in stance in the  online course we developed on African Entrepreneurship, we used insights  from interviews with participants to the Africa Prize for Engineering  as course material for budding African entrepreneurs to learn from.

Would policy reform concerning climate change be more assertive, if the UN’s Subsidiary Body for Scientific and Technological Advice (which advises policy makers on effective adaptation strategies to climate change, measurement methodology, etc, in settings like above) were run as a peer learning program? Photo source: http://unfccc.int

Interesting points from the guide:

  • Matching  of peers should be on appropriateness, rather than seeking out a  matches with best-in-class by default (which is often a bias that leads  to a miss-match)
  • Recognition  that learners need to be helped with a format for interaction, and  collaboration, rather then leaving peers to themselves to figure out.
  • This statement relates to what we see that peer learning works best in fast-changing, frontier environments.
  • Design matters for peer learning
  • Create  safe space, where politics is removed from the actual learning goals.  Tied to this in the public policy arena is also the importance of  authorising peers to collaborate.
  • There  is a benefit of combining peer learning with training, where peer  learning elicits sharper demand for specific training content.
  • One  of the main principles of peer learning we share with  authors, is that success is determined by what the learner becomes.
  • The  definition on peer learning is opaque, and doesn’t distinguish itself  from other types of learning. In this guide it is defined as:
  • Matching  of peers on an individual level by a facilitator is identified as key  to succeed in peer learning. But there is no guidance on how to actually  facilitate those connections.
  • The  guide focuses on peers who collaborate in the same domain (civil  servants). But it doesn’t mention the forging peers by bringing people  together from different domains.
  • The  use of the word “guide” in the title, creates expectations that the  work provides guidance for design. However, there is little actual  guidance, other than some questions that the learner or facilitator can  ask themselves in each of a subsequent set of phases. Also “correct  tooling” is mentioned as critical to peer learning, but neither what is  correct nor the tooling is explained.

Conclusions
The Guide is an interesting peak into the state of the art in of the  understanding of peer learning in leading organisations like the OECD  (The Organisation for Economic Co-operation and Development). Its authors point to the emergent nature of peer learning, and that design  needs to be better understood. This is reflected in the struggle you can  sense in critical aspects of their writing, where they try to

a) get  the reader to understand what peer learning is, and

b) indicate how to design a learning program.

The upshot is that this work is more an interesting collection of insights on peer learning then it is a guide.

From our experience in getting people to understand what peer learning is, what works best is to have them participate in it. What comes closest to the experience of participating, is hearing the stories about people, their goals, and how they interacted with their peers.  It’s hard to convince otherwise. So what's needed is more stories to spread the message.

Secondly, regarding guidance on peer learning design; this is something can ultimately directly support the agency of the learner, making scale, and impact, dependant on what the learner does with their agency. This dependency is captured nicely in this last quote from the guide, where the authors  say:

“On  the one hand, facilitators target peer learning ‘at scale’ (in  countries and organisations and cities) — given a theory of change that  results at scale require diffusion of lessons across a significant body  of individuals — but on the other hand the peer learning actually  happens more discretely in the hearts and minds of individuals,  partaking in specific personal relationships.”

The indirect outside impact on the organisation, or on reform (as the authors try to argue for), thus depends on what the learner does with their agency. So it's hard to make the claim between introducing peer learning, and reform.

But in light of progressing towards reform, it can be said that introducing more peer learning in the context of policy making can only lead to more useful developments.

Imagine how a hard-to-grasp-yet-critical topics like climate change would reform, if policy makers could start collaborative relationships with  science fiction writers to make visions of climate impact. Would the policy makers in Flint, Michigan, have responded sooner to the disaster,  if Egyptian engineers, accustomed to dealing with distributing drinking water to the population under high scarcity, would helped figure out  options for quickly and affordably providing alternative water sources  for the time that was required to fix the lead water mains? I can only guess policy making would start coming to grips with its fast-changing  environment.