RANSFORD

=======

STARTING DATE

Now

At your convenient time

MODULES

5

QUIZ

5 evaluation questions

HOURS

24 - 120 hours

WEEK

1 -2 weeks

FOR WHO?

Early-and-Mid-Career Professionals

Aspiring and existing Project Managers

Non-Profit Leaders

WHAT YOU WILL GAIN

Higher Skills in Monitoring & Evaluations

Certificate of Completion

$20 (NGN12,000)

1.1 What is Monitoring & Evaluation?

 Monitoring is defined as the concurrent process of tracking the implementation of activities of the project and attaining its planned outputs. It helps to provide real time information of the progress of the project in terms of completing its activities and achieving its immediate outputs, both in terms of quality and target.

Monitoring, thus, is an activity to see if an ongoing project is proceeding on track. It involves the process of systematically collecting data to provide real time information for all stakeholders (managers, funders, participants) on the progress of implementation and the achievement of desired outcomes.

 The critical functions of monitoring are: to gather feedback from the participants; collect data; observe the implementation of activities of the project; analyze contextual changes; and provide an early warning system of potential challenges. Analysis of monitoring data is critical to ensure that the project is being implemented in the right direction for it to achieve its intended outcomes. In case the project is not moving in its intended direction, mid-course correction should be done.

Monitoring is applicable to all programme levels (from input, process, output and outcome). Most commonly, the focus is on output data, although it is also important to track the goals and the objectives. Monitoring should ideally be an internal function of the project management team. Monitoring, thus, plays a critical role in the success of a project.

 Monitoring of results helps to:

 Improve strategies and targeting. Enabling decision makers to focus the project resources on areas where they can get the maximum output.

 Understand project implementation barriers or challenges in real time and suggest course correction measures.

 Ensure that the project is more effective and result oriented. It also focuses on impact level changes throughout the project, rather than just at the end of project evaluation.

 Evaluation is defined as systematic research to see if a programme can achieve its intended outcomes and impacts. Evaluation is done firstly to see whether the envisaged objectives and goals have been achieved or not, and secondly, to see whether the achievement is because of the project interventions. It should assess the magnitude of change in the outcome and impact and whether the change in the outcome or the impact can be attributed to the project intervention.

Evaluation assesses if there is any deviation from the goals and the objectives, and whether it can confidently be said that the objectives are achieved only because of project intervention. Evaluation, then, is a type of causal research that establishes the cause-effect relationship between the activities and the outputs on the one hand and the objectives and the goals on the other.

 While monitoring facilitates mid-course correction in attainment of project outcomes, evaluation helps analyze variances from envisioned objectives and goals. By providing feedback to the project functionaries, M&E facilitates learning by doing. Development and enhancement of in-house capacities to anchor the M&E functions is, thus, a prerequisite for learning organizations.

In this course, you will develop an understanding of the fundamental concepts of M&E, its importance for project management and learn the difference between the two. You will also understand how M&E is juxtaposed with the results chain of a project. Finally, at the end of the course, you will understand about indicators and learn how to design monitoring and evaluations plans and implementation.

1.2.  M&E Typologies

Some popular typologies and terminologies commonly used in M&E are discussed below. Monitoring Typologies Monitoring is a task that is inherently undertaken by doers or implementers of the project themselves. Therefore, when monitoring is performed by the project team itself or internally by the project implementing team, it is called internal monitoring. Sometimes, when projects involve parties or organizations external to the project for facilitating the monitoring functions, it is classified as external monitoring. 

If the project implementers restrict the monitoring process to themselves, it is called non-participatory monitoring. Project stakeholders, including the communities remain mere information providers and have no role in analyzing the information and providing inputs for project implementation.

When functional participation of key stakeholders of the project, including the target community for the project is solicited, it is called participatory monitoring. In the case of the sample project about working to make its target area ODF, a committee from the local community can be involved in formulating the monitoring plan, collecting information and analyzing it. The community monitoring committee can keep a check on activities like communication campaigns and toilet construction (both in terms of quantity and quality).

Evaluation Typologies

While monitoring is inherently an internal activity, evaluation is an external activity usually done by those external (individuals/agencies/ institutions) to the project. Generally speaking, evaluation is external evaluation. However, when project implementers choose to undertake evaluation by themselves, it is called internal evaluation. Evaluation can be best defined based on the timing of conducting the evaluation.

Evaluation per se is a less frequent activity generally undertaken at the completion of a project for assessment of attainment of the project objective. This is the post-project or post-facto evaluation. In the case of many long duration projects, evaluation is also conducted midway through the project implementation.

Mid-term evaluations help to ascertain the level of achievement of long duration projects half way through the project. When the programme funders or the management is interested in a more regular assessment of the achievements of the project, the outcome and impact level assessment is done at a six monthly or a yearly period also. This time series design of evaluation is commonly known as concurrent evaluation.

To relate the achievement of objectives and goals directly to the project, it may also be necessary to compare the status in the project area with an identical non-project area, which forms the control group while the project villages are the treatment group. This kind of evaluation design is called the control-experiment design. Depending upon the time when the evaluation is implemented, it can be concurrent, mid-term or post-facto and internal or external. 

1.3. Indicators

The concept of indicators is pivotal to M&E. As per its dictionary definition, an indicator is defined as a sign or a signal. In the context of M&E, an indicator is said to be a quantitative standard of measurement or an instrument which gives us information (UNAIDS, 2010). Indicators help to capture data and provide information to monitor performance, measure achievement, determine accountability and improve the effectiveness of projects or programmes.

Designing indicators is one of the key steps in developing an M&E system. As mentioned above, indicators are units which measure information over time to document changes in the specific conditions. With respect to the various M&E levels and the result chain of the project, specific indicators need to be developed for each stage of the results chain. Thus, there should be a different set of indicators at the impact level, at the outcome level as well as at the output, activity and input level. Also, for each level, there can be more than one indicator.

An indicator may be quantitative or qualitative based on the characteristics of information that it provides. Those that deal with information that can be expressed in numbers are quantitative indicators, while those dealing with information units expressed in any form other than in numbers, e.g., statements, are qualitative indicators.

Another important attribute of quantitative indicators is that arithmetic functions can be applied to its corresponding data while this is not possible in the case of qualitative indicators. For qualitative indicators, their count or frequency may be considered. Income measured in rupees, the weight of a baby measured in kilograms and the number of toilets built are examples of quantitative indicators. If the same information of income or weight is collected in categories of high, medium and low, they are qualitative indicators. 

SMART and SPICED Indicators

The SMART criteria is widely used to judge project objectives. A project objective is said to be SMART if it fulfils the following criteria:

• Specific
• Measurable
• Attainable
• Realistic
• Time-bound

In order that development interventions are more result oriented, projects must be made SMART. The acronym SMART for developing project objectives is mentioned above, and will be discussed in detail later. At the same time, it is important to make sure that the indicators or the performance measures also fit the SMART criteria. SMART indicators play an important role in results based project management  as  they  ensure  accountability  (MDF  Training  &  Consultancy,  2016).  They  have  the following  characteristics:

• Specific
• Measurable
• Attainable 
• Realistic
• Time-bound

Another  school  of  thought  advocates  qualitative  indicators  represented  by  the  acronym  SPICED,  which stands  for  the  attributes listed  below:

• Subjective
• Participatory
• Interpreted  and  communicable
• Cross-checked  and  compared
• Empowering
• Diverse and  dis-aggregated
  

1.4. Design Indicators

Indicators are essential instruments for M&E, thus, practitioners need to keep in mind some of the critical points while designing or formulating them.

Firstly, creating new indicators or reinventing the wheel should be avoided till the time it is absolutely required. Over the years, development professionals have worked to provide M&E practitioners with sets of well tested and proven indicators and these sets of indicators should be referred to while formulating indicators for any project.

Secondly, while designing indicators, it should be made sure that they fulfil either the SMART or the SPICED criteria. Indicators can document change therefore, any indicator finalized should essentially be able to capture change in the condition that is being assessed using the indicator.

A good indicator is therefore:

• Simple: As all the good things in the world are
• Measurable: Provides a measure for depicting change
• Precise: Has a definition so that it can be defined in the same way by all
• Consistent: Has consistent measurement results. On measuring the same thing, its value remains consistent and does not change over time 
• Sensitive: Can capture the smallest amount of change in the indicator value  Action Focused: Captures information that is eventually useful for stakeholders and leads to some action.

While designing indicators, it is very important to collectively brainstorm to identify candidate indicators for a specific condition. Once several indicators are listed for a given specific condition, the next step is to assess each of the indicators using the characteristics of a good indicator to find out whether the candidate indicator is simple, measurable, precise, consistent and sensitive. The source of data for the indicator and the reliability of the sources is also considered. The cost incurred in collecting data for this indicator is also considered while finalizing the indicators. Candidate indicators that satisfy the criteria are then taken as indicators for assessment of that condition. Candidate indicators are also modified till they acquire the characteristics of a good indicator.

For example, in the case of the project which aims to make its target area ODF, the output level indicator is ‘The number of individual household latrines (IHHL) constructed’. As constructing toilets is one of the key outputs expected from the project, this indicator helps in measuring the same. Similarly, as creating awareness about sanitation is another key activity, ‘The number of village level meetings conducted to create awareness about sanitation’ is another output level indicator.

Considering another example of a project which aims to improve maternal and child health (MCH) in its target area, ‘Maternal Mortality Ratio’ (MMR) and ‘Infant Mortality Ratio’(IMR) are the result level indicators for this project. At the outcome level, the indicators are, ‘The number of women with incidences of serious health problems related to child birth’ and ‘The number of women consuming iron fortified food or iron supplements during pregnancy’. At the output level, the indicators are ‘The number of deliveries conducted by skilled health professionals’, and ‘The number of women receiving at least three antenatal care (ANC) visits’. 

Defining Indicators

After selecting suitable indicators, it is very important to fully define them. No indicator should be deployed without fully defining it and making sure its essential components are lucid and concrete (UNAIDS, 2010). Each indicator definition should have the following components:

Title: A brief heading that captures the summary or focus of the indicator.

Definitional: A lucid and to the point definition of each indicator so that everyone can interpret it in the same way.

Source: The source i.e., the tool used for getting this indicator value and the respondent from whom this information is collected is also defined. 

Data Collection Frequency: The frequency at which the data is collected to derive at the indicator value is defined. This could be at quarterly, half yearly or annual intervals etc.

Numerator: The variable that is included above the line in a common fraction.

Denominator: The variable that is included below the line in a common fraction.

Calculation Method: The method for calculating the indicator value is defined.

For instance, the complete definition of the indicator, ‘The number of deliveries conducted by skilled health professionals’, is stated below:

Title: The number of deliveries conducted by skilled health professionals

Definition: This indicator measures the number of deliveries conducted by skilled health professionals. The term skilled health professional refers exclusively to people with midwifery skills (auxiliary nurse midwives (ANMs), doctors and nurses) who are trained in the skills necessary to manage normal delivery cases and diagnose, manage or refer obstetric complications

Source: The sample survey conducted as part of the baseline, midline and endline survey

Data Collection Frequency: In the first quarter of the first year and in the last quarter of the third and in the fifth year of the project.

Numerator: The total number of deliveries attended by skilled birth attendants (SBA) as reported in the sample survey.

Denominator: The overall sample size of the number of women who had deliveries in the last two years.

Calculate: This indicator value is calculated by dividing the number of births attended by skilled health professionals by the total sample size of the number of deliveries conducted in the last two years. 

Download

full course guides

To complete this program, download the under listed course modules in PDF.  Study and complete the Monitoring & Evaluations quiz to get your certificate.