The European Climate Foundation’s monitoring and evaluation journey

“Why does the European Climate Foundation (ECF) monitor and evaluate?” This was the question that started the ECF’s M&E journey. The answers include: to help see where we are going, to see if we need to change direction, to help make better plans for the future and to make our work more effective. All these are underpinned by the concept of doing things better. At the ECF we wanted M&E that supported achievement of greater impact. The next logical question was “How?” The answer became PARL – our online platform for Planning, Assessment and monitoring, Reporting and Learning.

The ECF was established in 2008 as “a foundation of foundations”, a philanthropic initiative to help Europe foster the development of a low-carbon society and play an even stronger international leadership role to mitigate climate change. The ECF is guided by the co-development of strategy with our partners, which include grantees, funders and other stakeholders. The ECF portfolio encompasses sectoral, cross-cutting and regional initiatives with accompanying programme teams. Three years ago, these teams worked independently of each other with their own ways of planning and monitoring their programme strategies. There were also variances in the language used in articulating the strategy within teams, across teams and externally to partners. With a new M&E system the aim was to achieve better integration, and to have “one way” to present the theory of change, monitor progress and evaluate effectiveness. Thus PARL was born.

PARL is an online database that holds the ECF strategy. It displays each ECF strategic objective as a logic model providing a graphical depiction of the contribution of a grant to the achievement of an objective. There are four stages to system completion planning, monitoring, reporting, and learning:

1. In the planning stage the ECF strategy is captured in PARL. This starts with the Executive Management Team (EMT) agreeing the initiative theme and associated objectives. The programme staff then describes the bottlenecks that are preventing achievement of the objectives or conversely the drivers that if levered correctly can support achievement. Outcomes are the shorter-term impacts that are required to mitigate the bottlenecks. The programme staff then engage grantees, developing the grants to deliver the activities and outputs, which will result in the realisation of the outcomes. The planning stage ends with the approval of these grants by the ECF Board. Additional information is captured in PARL at each strategy level (such as description, context, indicator, score, progress, key learning, etc.).
2. Strategic monitoring is documented in the system quarterly by the programmestaff, through a statement of progress and a score at three strategy levels: grants, outcomes, and objectives. The score is a colour-based on a traffic light system (see diagram).
3. Planning and monitoring reports are generated from PARL based on the differing user requirements e.g. funders and the board.
4. Review and learning is arguably the most valuable stage. Where outcomes are not being achieved despite good progress at grant level, the theories of change need to be reconsidered, triggering a potential change in the strategy. Where outcomes are being achieved it is important to understand the reasons for this so the achievement can be replicated. In the ECF we are still working at implementing this stage in a way that will enable us to use learning to improve/adapt/replicate the strategy as required.

Implementing PARL in the ECF meant a significant change to work practices for the programme staff. PARL was launched in 2014 and while some staff quickly embraced the system the majority were concerned about what the change meant. Phrases used were, “I’m not sure about this”, “this will mean extra work” and “I don’t want to do it”. Gradually all teams started to accept the need for PARL, recognised the benefits and let go of old ways of working. Two years on PARL is generally accepted as the “way we do things around here” and is promoted by staff. There were a number of factors that supported the journey:

• The project had the ongoing support from the ECF CEO. He was involved in the initial concept for the system and promoted PARL at all staff meetings.
• The development of the platform, (design phase to launch), was led by the ECF COO and the implementation team had the right mix of skills that included IT, programme knowledge, change management and communication.
• There were regular communications between the implementation team and the programme staff to give and receive feedback. This communication became essential in ensuring PARL was developed in a way that met the user needs.
• There were ‘system champions’ on the programme teams who were enthusiastic about PARL and promoted it. This enthusiasm helped sway others.

The M&E journey has had successes and challenges, which have made the journey itself more interesting and provided useful insights. A key success is that strategic planning monitoring and evaluation are interlinked in one system. By having a well thought through strategy, which at the outset defines what success looks like and how it is expected to be realised, means the monitoring and evaluation of effectiveness is a natural consequence.

PARL has supported increased clarity in the strategy and provides a common strategic language within the ECF and the wider community (funders and grantees). It is increasingly used as the basis for strategic discussions between the programme teams and partners and the ECF is starting to share the tool with other foundations who have the same aim of achieving a greater impact. The graphical depiction of the strategy and its progress presents an attractive, simplified visual of how change happens. However it can be challenging for teams to articulate the theory of change in this format and sometimes the logic model is criticised for being too rigid or oversimplifying complexity. PARL is also only as good as the information it holds, and time and effort have been spent on achieving consistency in the quality of information, particularly in formulating tangible indicators, writing robust progress statements and uncovering the key learning.

PARL has achieved the aim of having one way to present the organisation’s theory of change and monitor and evaluate effectiveness. The next steps of the journey are about using PARL in a way that gives insight into that theory, understanding the ECF’s underlying assumptions about how change happens and questioning their validity. So as the journey began it continues with questions.