These monitoring, evaluation and lesson learning guidelines are for projects supported by DFID’s Development Awareness Fund (DAF).
The guidelines explain why it is important to develop good systems for monitoring and evaluation and for learning and sharing lessons. They provide details on the DAF requirements for monitoring and evaluation, with specific guidance on project frameworks, monitoring and evaluation plans and reporting requirements.
Good monitoring and evaluation is important because these activities:
Monitoring is a continuous internal process, conducted by project managers to check the progress of development interventions – in this case DAF projects – against pre-defined objectives and plans, as set out in the project’s planning framework.
Evaluations usually take place towards the end of the project. They ask what has actually been achieved, what happened and why, and answer specific questions related to the relevance, effectiveness, efficiency, impact and sustainability of the project. Evaluations make use of the information recorded during the monitoring process.
The DAF uses a mixture of monitoring and evaluation to fulfil both lesson learning and accountability functions. The DAF intends that important lessons are learned from and shared so that they can contribute to improvements in development awareness policy and practice.
Monitoring and evaluation should be seen as an integrated reflection and reporting system for project managers and others engaged with the project. Monitoring and evaluation must be planned for, managed and resourced. These tasks are essential for good project management and should not be considered as just a statistical task or an external obligation.
The main project planning tool used as the basis for DAF project monitoring and evaluation is the project framework document.
The project framework was introduced as a compulsory component of DAF proposals in 2009 to strengthen the design, implementation, monitoring and evaluation of projects. It is an important management tool to describe logically how particular activities are expected to contribute to the achievement of the outcomes or changes that will together produce an overall change as described in the project purpose.
The project framework is useful for anyone involved in a DAF project, as it:
To ensure these processes work well, DAF projects should:
At the proposal stage the framework helped to demonstrate the project design and the logical link between activities, outcomes and purpose and the indicators helped to clarify expected, measurable changes. Throughout the project implementation period, the project framework will be used as a key reference document for project management, monitoring and evaluation. The framework will be reported against and reviewed in the Annual Reports and at the end of the project in the Project Completion Report and any project evaluations. As the project framework is such an important tool for both project management and monitoring, at the beginning of the project it is considered necessary to review the document prepared at the proposal stage, and if necessary amend and clarify the content to ensure that it satisfies these requirements. This review process will be undertaken as part of a Project Inception Plan process, to be followed on all DAF projects starting in or after 2009.
Within three months of signing the DAF Grant Arrangement, all new projects (from 2009 0nwards) will be required to prepare a Project Inception Plan that comprises:
At the Project Inception Plan stage, the outcomes and indicators prepared for the proposal may need to be refined to ensure that they are appropriate for use as a basis for project monitoring and evaluation. However, it is important to note that any changes made to the purpose and outcomes must not represent fundamental changes to the project focus as presented in the proposal and all changes to the project frameworks will need to be approved in writing.
At the Project Inception plan stage, the key elements of the framework need to be reviewed using the following guidelines:
Project Purpose: The project purpose should summarise what overall change the project will achieve and clarify the link between the project outcomes and the selected DAF programme objectives. The purpose should be fully achieved by the time the project is finished.
Project Outcomes: The outcomes should describe the main changes in knowledge, understanding, skills and attitudes expected to result from the project activities and contribute to the achievement of the project purpose. For some projects, the expected outcomes may be in the form of changes to the behaviour, policies and practice of individuals or organisations. The outcome statements should clarify the specific target groups and the nature of the anticipated changes.
Indicators: DFID defines an indicator as “a quantitative or qualitative factor or variable that provides a simple and reliable means to measure achievement, to reflect the changes connected to an intervention, or to help assess the performance of a development actor”. The indicators should provide clear and timetabled measures of progress and achievement of targets compared to a measured baseline state.
In order to work effectively as a management tool, the project framework must include indicators that are carefully selected to be clear, relevant and practical to measure. Generally, a few strategically selected indicators will be more useful than many.
Indicators must also be SMART: i.e. Specific, Measurable, Attainable, Relevant and Timebound.
Baseline Data: In order to measure the nature and extent of changes resulting from project interventions it is essential to establish the pre-project, or baseline situation as a reference point. The purpose and outcome indicators included in the project framework and monitoring and evaluation plans must therefore include comparisons to relevant baseline information. This ‘baseline data’ may have already been collected as part of the project planning process, but if not, it will need to be established during the project inception phase.
Activities: The main actions or tasks that need to be undertaken to achieve the project outcomes.
Activities also need to be scheduled over the life of the project and details provided to clarify the quantitative and qualitative targets e.g. the description of training activities should include the number of planned training activities, numbers of people to be trained, the target group and the expected outcome of the training activity.
The DAF 2010-11 Project Framework and Activity Schedule form can be used to prepare the refined project frameworks with scheduled activities.
The project monitoring and evaluation plan identifies how progress against the indicators will be measured, by whom, when and how frequently. A suggested format for the monitoring and evaluation plan and an associated checklist is available. This format may be adapted to suit particular project circumstances, but should include all the key components.
Most project plans also include provision for an independent external evaluation to be carried out towards the end of the project. This provides an independent assessment of achievements and positive or negative factors affecting performance.
All DAF projects are expected to contribute to the achievement of one or more of the four published DAF programme objectives. The project framework document identifies which of the DAF objectives the project is supporting, and the purpose statement should help to clarify the link between the project outcomes and specific DAF programme objectives.
The Inception Plan review and appraisal will aim to ensure that project frameworks, monitoring and evaluation plans and activity schedules enable both grant holders and DFID to assess project progress and the project’s contribution to the achievement of the DAF programme objectives.
The inception plan project framework, activity schedules and monitoring and evaluation plans provide the basis for project monitoring and progress reporting to DFID (via Triple Line) through a number of reporting mechanisms. These include expenditure reports (submitted as part of the payment claims process, usually quarterly), annual progress reports, project evaluation reports and project completion reports.
The monitoring and evaluation and reporting requirements for DAF projects are listed in the table below:
Requirement of grant holder
DFID/Triple Line responsibility
Statements of expenditure confirming actual expenditure against budget.
Where undertaken, final evaluations should verify achievements and contributions towards DAF programme objectives and note what happened and why, and identify lessons from the process of implementation that may be of use to others.
In addition to the above, grant holders may also be asked to provide material for case studies to be used in DFID internal and external communications to illustrate the impact of our work.
All reports require grant holders to complete an Achievement Rating Scale (ARS). The format for this is shown in the templates for each type of report. It is important that the overall score and justification is filled in.
The agreed project framework forms the basis of the contractual relationship between DFID and the project team. DFID expects that the project will achieve what was stated in the purpose by the end of the project in exchange for the funding supplied. However, as the project framework is a prediction, the situation on the ground may change during the course of project implementation, requiring adjustments, particularly to the activities required to achieve the outcomes and purpose. The indicators may also need to be amended, particularly early on in the project when baseline data becomes available. The project purpose and outcomes are not expected to change during the lifetime of the project. Any proposals to change the project framework documents should be communicated to Triple Line as soon as possible via email and no significant changes should be implemented until written approval is received. Once agreed, any revisions to the plans will be used as the basis for further monitoring, evaluation and reporting. By the end of the project, outcomes may not be exactly as planned. However, there will still be some impact and change, both intended and unintended, positive and even in some cases negative changes. This information, provided in Annual Reports, Project Completion Reports and evaluations needs to be recorded so that important lessons can be learned and used to inform future project and programme design.
During the lifetime of a project many lessons will be learnt. Sometimes there will be lessons from success and sometimes they will be hard but useful lessons learnt from failure. These lessons may help others to build on what worked well or avoid similar challenges.
To tease out lessons, project teams need to understand what works - or has not worked - where, with whom, under what circumstances, and why. This means being able to analyse the context in which the work was done, what work was done and what the outcomes of that work were.
Lessons might arise from interesting problems and how they have been tackled, in what circumstances and with what success. There might be other lessons about the assumptions that were made and how these turned out not to be right. Other people may avoid making the same assumptions. Risks may have been poorly identified, or could have been managed better.
Lesson learning can only happen when there is time to reflect on practice, identify lessons, publicise them to others and when others have the chance to absorb and apply the lessons. Ideally project teams should schedule regular meetings for this type of reflection on the project and to record monitoring information. In the first instance monitoring should be giving the project team and stakeholders the chance to learn lessons and improve their practice. This requires open-mindedness in reviewing progress so that difficulties can be acknowledged rather than disguised.
Lessons need to be targeted to be shared effectively. It may be helpful when listing lessons to name whom you think the specific audience for the specific lesson might be. This will help others to organise the information in the most accessible way. Different audiences might include DFID, other funding agencies or organisations engaged in development education/development awareness
The Annual Report and Project Completion Report provide opportunities for documenting lessons learned. DFID can use these lessons internally to inform future planning, policy and strategy. Projects should also consider how information about the project and lessons that may have widespread value can be disseminated through their own networks, using newsletters, publications, workshops and seminars and by allowing regional staff to meet together to share experiences.
What are Bookmarks?
DFID around the web:
© Crown copyright 2010