This is one of the best posts on basic fundamentals of Monitoring and evaluation but generally speaking, if you like to comprehend the concept then you should not only read this post but also consult many of the available resources on the topic. You should give appropriate time and focus on its process & tools and in-depth details. So, are you ready to improve your M&E skills? Start reading this post very carefully which will enable you to respond well to any of the basic questions about this field and able you to interlink it with the program and other project areas. The information in this post will surely help those who want to enhance their M&E knowledge which is essential for career growth as well. Let's start with the introduction:
Introduction, roots and importance
We like to introduce Monitoring and Evaluation for readers so the first thing to remember is this is one of the important process groups of Project Management and is strongly linked with the other four process groups (initiation, planning, execution and closing) of the project life cycle. A meaningful M&E system provides valuable insights to the donors and management to keep ongoing processes & activities on track. It empowers decision-makers with Control features; it works by identifying delays, ambiguities or bottlenecks during the course of process monitoring, collating data and generating monitoring reports for program managers for taking timely corrective measures for the current projects. A functional monitoring, evaluation, and accountability & learning system proposes recommendations against key challenges, compiles lessons learned and emphasizes reinforcing good practices. The process also helps to present project success by claiming the achievement of desired results based on evaluative evidence through selective evaluation exercises. Don't worry if you are getting confused here, below are the details of everything in simple words to equip and strengthen your knowledge of M&E functions.
The field is evolving continuously so we have to keep updating ourselves consequently. In a large organization, there is a separate monitoring section that works closely with the program, policy and other departments. Institutions/NGOs use different titles for their M&E department for example; Monitoring, Evaluation & Learning (MEL), Program Monitoring, Evaluation and Research (PMER), Appraisal Monitoring & Evaluation Unit (AMEU), Monitoring, Evaluation Accountability and Learning (MEAL), etc. however purpose of all almost same.
What is the monitoring and evaluation framework?
The framework provides qualitative and quantitative measurement and performance assessment of a project to timely predict the project's success or failure which increases its importance in organizations and empowers program managers to take steps for fixing the issues in the project implementation plan in light of areas of improvements identified in monitoring reports to avoid unexpected results or project failure.
The System primarily answers the management & donors about performance by measuring & assessing output & outcome level achievements, results and impacts against project activities. In actuality the M&E field is not separate from the program instead, it is an important and integral part of the program similar to other instruments. The process is not a one-time event rather it is carried out multiple times to provide indicator-based systematic & coherent information to compare planned targets with accomplishments so that the execution of the project is assured in the right direction and on track. The chain of results of M&E are closely interlinked as reflected in the below model however each of the two functions yields different information to fulfill the data and reporting needs. Let’s discuss each function in more detail:
Define Monitoring
- Monitoring is regularly carried out during project execution to objectively observe the processes and it supports the project management to see if the implementation of activities is aligned with the project design, log frame, M&E Plan and DIP?
- The monitoring is taken place to track progress periodically and analyze achievements in light of agreed criteria & standards. Moreover, the monitoring department also collaborates with the program to ensure that relevant data have been collected, stored, analyzed and reported timely to donors and stakeholders as mentioned in PMP.
- Moreover, a monitoring process reports on lacking & delays side by side of the project execution to the field team for corrective actions.
- It is carried out more frequently to observe the ongoing activities as it starts with the project initiation and continues till the completion of the project.
- It is a repeated process of observation, data collection, comparison, progress tracking, assessment and measuring the performance at the output level for the target indicators of project activities and generating monitoring reports on the basis Project Monitoring Plan (PMP) or Monitoring Plan or Framework document on different types of templates/formats and monitoring checklists.
Define Evaluation
- Evaluation is an exercise to examine what activities of the project worked well and what went wrong? and record the experience for learning and improvement.
- It probes for example; has the Outcome level progress achieved as it was planned or not? has planned outcomes generated different types of expected impacts? Can we logically claim the ultimate achievement of the Goal of a project?
- So, evaluation provides answers to all of these types of questions by using systematic research methods, tools and approaches. All such questions are asked to know the value, relevancy, and development effectiveness of an intervention.
- The data collected in the result of an evaluation activity have useful information that is very influential and much needed for future planning of projects & making long-term policies or modifying existing programs or use in replicating the same project for different populations and target areas.
Why Evaluation?
The answer is simple; in the past, it was enough when NGOs and Government agencies were just reporting on how much budget they spent and what outputs were achieved but nowadays fashion has totally changed and result-based management (RBM) is more focused so results-oriented monitoring & evaluation system is workable in the present scenario. Therefore not only funding agencies but also implementation organizations emphasized result-oriented reporting and are interested in discovering the effectiveness of aid in all measurable forms of data. Basically, they want to see and share information with project stakeholders on how much an intervention brought value and change in the conditions. For example, a WASH project reduced 30% of diarrhea cases of under 5 years of children and how much population access to installed pumps? and up to what extent people meet their needs in a target population as a result of the installation of clean drinking water pumps. Moreover, they also study the sustainability of achieved results. That's why Evaluation has taken place at selective points of a project lifecycle.
In the design stage of a project, the program development team and MEAL experts prepare a logic model & log frame and carefully finalize what activities of an intervention are to be evaluated because a sufficient budget is required to conduct it. Mostly following evaluation models are used and each produces information for unique needs. Further details are given with each type:
04 Main Types of Evaluation
There are many other forms of evaluation but we discuss here four main types of evaluation mostly used in practical situations. Each of the type serves for a unique purpose and provides desired information as per the needs of project.
1. Formative Evaluation - It is a type of assessment that takes place before the implementation of a new project to evaluate the actual needs of the target population. In this activity, we assess whether this is the right time for proposed components and will satisfy the needs of the target population?, understanding & acceptability level of proposed actions in the area, historical background with relevant statistics of problem against suggested intervention Or sometime you may conduct an assessment during project implementation to find out the root cause of problems and challenges to modify or undo some actions to ensure keep the project successful. In another scenario, you may have to go for assessing the compatibility, capability & executability of an existing program for another geographical location and for a different population so that you can quickly initiate an effort and replicate good practices in light of acquired experience.
2. Process Evaluation - It is a selective exercise to assess implementation progress at many intervals. This is also known as implementation evaluation and some organizations name it program monitoring. It is initiated after a process or its part is completed in the form of multiple activities or as a single activity as per plan. Researchers collect data by designing a Household survey or FGD questionnaire for project stakeholders and beneficiaries to know what the project achieved so far & for whom? whether activities are implemented at the right time and many other questions to objectively evaluate the performance of output level indicators against various components of a project.
3. Outcome Evaluation - Let's understand it with an example; There is a primary healthcare project and one of its activities is to provide Infection Prevention Control (IPC) training to Doctors and Paramedical staff for example in 04 number of training 28 medical staff trained is the output number whereas its outcomes would be a change in behavior of trained staff in terms of adopting recommended methods of IPC, counsel & deal patients accordingly will be reduced chances of further spreading infection, improve in awareness level of affected population is also an outcome. So we conduct an Outcome or objectives-based Evaluation to get in-depth details in quantifiable forms to see how certain outputs left effects on project beneficiaries, changed the behavior and improved practices. These are some kinds of objectives of a project and we measure and assess these effects under this kind of evaluation.
4. Impact Evaluation - A project goal is the highest level of the target. To claim whether our project goal has been attained? or up to what extent initiatives contributed to modifying the current dilemma in a positive or adverse way. What was the extent of the intended achievement? Also, it is very important to overarch contextual and external factors that affect performance. NGOs usually take the services of a third-party consultant to conduct an impact evaluation after the completion of a program or can plan it at the main intervals of a project so that they can further improve aid effectiveness. A project goal is straightforward a big statement describing what will be ultimately accomplished as a result of an intervention. In the above paragraph on outcome evaluation, we discussed a healthcare project example and its effects where objectives like change in behavior and improvement in treatment protocols will be achieved whereas the Goal of the same project could be "improved healthcare services in government hospitals to reduce morbidity rate due to COVID-19 infection". In impact evaluation, we analyze to which degree the morbidity rate reduced in percentage/numbers in comparison to what has happened in areas at the start without an IPC project and after completion of the project. Overall how much has our health system strengthened and in what areas for a particular population? We can measure for example what major & minor impacts were noticed in reducing morbidity? how to correlate various hypotheses like raised awareness level of the community on the pandemic with the ultimate goal and still what expectations didn't meet which affected the achievement of the goal to a certain degree, what are the reasons and how to overcome the encountered problems in future intervention?
Conclusion
Monitoring and Evaluation are two distinct functions of program performance measurement however at the same time, it is a combined system and a process of project management that helps organizations in analyzing and tracking the progress of ongoing activities of a project, highlights obstacles to timely address and collates accountability and lesson-learned data. The system evaluates program activities, outputs, outcomes and impacts to assess the effects, relevancy, effectiveness, efficiency, coherence and results of a project to improve planning, design and implementation for the upcoming projects or modification in the existing projects.
Keep in mind always the objectives and goals of projects are social development, the well-being of the target population, the strengthening of the local systems by a participatory approach and shifting from a poor to an improved condition, and so forth. Therefore the purpose of M&E actions is to support the program to achieve the objectives and ultimate goals as per target indicators, planning & design by providing concrete feedback analyzed data, and reports for informed decision-making and to improve the performance of programs.
Comments
Post a Comment