Terms of Reference for Evaluation

EVALUATION SUMMARY

BACKGROUND OF PROJECT

CBM started working with the National Union of the Disability Organizations of Rwanda (NUDOR) since 2015 until now. NUDOR exists to strengthen the voice of the disability movement in Rwanda. It is an umbrella organisation established in 2010 and composed by 15 Organisations of persons with disabilities. Together NUDOR and its fifteen members are working so that persons with disabilities can enjoy the equal rights to which they are entitled.

CBM’s and NUDOR partnership was established and formalized through contracts and Memorandum of Understanding (MoU) on the project entitled “Promotion of anti-discrimination practices for Persons with Disabilities in 5 districts in Rwanda” (P3898/BMZ 4287) which ended in January 2023 and the project “Promoting the Livelihood of Persons with Disabilities in 4 districts” and the current project “People with disabilities in Rwanda improve their economic situation - Zigama Ushore Ubeho Neza (ZUUN) (P4170/BMZ 6279).

The project “Improving the socio-economic living conditions - with a focus on savings and health - of people with disabilities in 5 districts in Rwanda - Zigama Ushore Ubeho Neza (ZUUN)“ is targeting the following direct and indirect target groups:

The direct target group of this project are the 22,500 future members of the 750 village savings groups to be founded in this project, of which each group will have around 30 members. Around 60% of the members will be female and 40% male1. 70% of the members have a disability, 20% are caregivers of people / children with disabilities and 10% of the members have no disabilities2. From existing NUDOR projects we know that more than 60% of the people with disabilities have a physical disability including visual or hearing impairment. More than 10% have a cognitive disability / learning disability and more than 10% have a multiple disability. The remaining 20% have other disabilities such as albinism or short stature.

The minimum age for membership in a village savings group is 18 years. Another criterion for membership in the groups is belonging to the same village (members must already know each other).

The indirect target group includes all people with disabilities living in the five districts will benefit from more accessible financial services. With an estimated prevalence of 15% according to WHO3Data, this is almost 318,000 people. We would also like to reach around 30% of the total population in the project area with awareness-raising measures (635,551 people).

The interim evaluation of the project “Promoting the livelihood of persons with disabilities in Rwanda” (CBM / NUDOR, 2020) showed that only 18% of the savings groups in this project are led by women, although almost 60% of the members are women. Therefore, in this project, special attention should be paid to the participation of women in the savings groups.

We want to give 10% people without disabilities access to the village savings groups, as this is absolutely necessary for inclusion and social acceptance based on the experience of past projects (mid-term evaluation 2020, village savings groups in Rwanda, CBM and NUDOR). These members belong to other disadvantaged groups, e.g. are she Tested positive for HIV or survivor of the genocide. However, the group leader may only take on one person with a disability.

World Disability Report, 2011

The overall objective of the project “People with disabilities in Rwanda improve their economic situation - Zigama Ushore Ubeho Neza (ZUUN)” is that by 2024, the socio-economic inclusion of 22,500 people with disabilities in the districts of Gisagara, Burera, Ngoma, Rubavu and Muhanga in Rwanda will be improved.

Specific objective: 22,500 trained people with and without disabilities in the project area benefit from better financial education, the ability to save and more participation in social and economic life.

 Under the specific objective, the following outputs will be reached through a combination of the following 4 results/outputs:

• Result/output 1: 22,500 people with and without disabilities are organized in functioning, inclusive village savings groups.
• Result/output 2: 22,500 trained VSLG members acquire skills in sustainable agriculture or other business models.
• Result/output 3: At least 18,000 of the VSLG members participate in community activities and / or community politics at the district level
• Result/output 4: State and private providers in the health, education and rehabilitation sectors are better networked.

In order to achieve those objectives, the project`s action plan considers the following interventions:

Objective 1: 22,500 people with and without disabilities are organized in functioning, inclusive village savings groups

• Mappingof people with disabilities in the project area
• Trainingof trainers for business mentors (savings group concept)
• Foundationof 750 and savings groups in 5 districts and training of members in the savings group concept.
• Trainingof trainers for business mentors (management, financial knowledge, entrepreneurship)
• Exchangeof experiences between new and old savings groups
• Purchaseof project vehicle
• Trainingcourses for microfinance institutions in inclusion

Objective 2: 22,500 trained VSLG members acquire skills in sustainable agriculture or other business models.

• Trainingof 200 savings group members in various income-generating measures as multipliers.
• Trainingof 200 savings group members in marketing and value chains as multipliers
• Coachingthe target group in setting up income-generating measures
• Trainingof 100 savings group members in modern beekeeping (EUR 18088, BL 2.2 and 100 beehives
• Smallstudy on affordable access to water for income-generating measures

Objective 3: At least 18,000 of the VSLG members participate in community activities and / or community politics at the district level

• Participationin the International Day of People with 
• Participation in the network meeting JADF (Joint Action Development Forum) at district level.
• Raisingawareness through 
• Officialproject start and stakeholder coordination 

Objective 4: State and private providers in the health, education and rehabilitation sectors are better networked.

• Creation, production and distribution of the overview map of rehabilitationfacilities
• Collaborationmeeting with key partners (UR and other NGOs)
• Trainingof business mentors in sign language
• Training in tactile communication and functional assessments of people with hearing impairments

 EVALUATION OBJECTIVE, SCOPE AND INTENDED USE

 The Final evaluation of the project” People with disabilities in Rwanda improve their economic situation - Zigama Ushore Ubeho Neza (ZUUN)” (BMZ: P6279) during the implementation period of 1 December 2021 to 31 December 2024 it is intended to provide evidence of impact of the project (programme) at final stage with main purpose to facilitate the process, which will document project outputs and broader impact. Eventually, the process should also mobilize the various stakeholders to take action based on this documentation.

SCOPEOF THE EVALUATION

The final evaluation is being commissioned by CBM in order to understand and document the main results and changes brought on by the livelihood project in the lives of people with disabilities and how other stakeholders (public and private) have contributed to those changes / or role they have played in those changes. The final evaluation will focus on assessing the programme as a whole. It will be an impact evaluation and should assess formative aspects.

It will reflect on the process of developing the project, examine how far the project has achieved the expected the impact towards its overall and specific objectives, understand problems that need to be addressed and what lessons can be drawn. This will inform CBM and BMZ on key impact areas and support in understanding improvement needs for future similar programming.

The objective of this final evaluation is review and analyse the overall project implementation according to the project plan from the project start up to the moment of final evaluation takes place. The scope of the evaluation is the entirety of activities as per Project Contract.

The final evaluation aims to achieve the following objectives:

• AssessPartner Performance: Evaluate the performance of project partners in relation to the project plan, contractual obligations, and implementation activities across the regions of Burera, Gisagara, Ngoma, Muhanga, and
• Accountabilityand Learning: Review the overall project results achieved to date, with a focus on accountability for donor reporting and supporting organizational learning.
• Support Project Management and Stakeholders: Provide project management and stakeholders with an clear objective, external assessment of the project’s current status.

This evaluation will help identify key successes, challenges, and areas requiring attention.

Key Evaluation Focus Areas. The evaluation will explore the following dimensions: Relevance, Coherence, Efficiency, Effectiveness, Impact, Sustainability, Coverage, Gender considerations, Child Safeguarding, Environmental impact, Disability Inclusion.

 The evaluator will be required to evaluate the following:

• Evaluatethe overall impact along the OECD/DAC criteria
• Evaluate critical issues relating to results achievement and the factors affecting successful impact of the project.
• Generatelessons learned from the implementation of the project’s activities and the outcomes achieved that will be useful for similar projects in future for same context; Identify the good practices that can be reached to scale in other projects.
• Develop specific recommendations for major stakeholders to promote sustainability and long-term impact to the beneficiary communities.
• Thestrategic positioning and 
• Sustainabilityof results

The final evaluation is expected to cover the following project components:

• Project Management and implementation and collaboration with other stakeholders at national, local level and NUDOR level;
• VSLGmember’s engagement and VSLGs activities including savings, loans, social funds, Income Generating Activities, collective marketing and value added activities – honey production for example;
• TheGroups cohesion and effectiveness of leadership;
• Theconsistency in savings and loan repayment  default rates;
• Thedropout rate and reasons for the same;
• Assesswhether persons with different disabilities are equitably participating in VSLAs including persons with deaf blindness
• TheVSLGs’ Financial Linkage to Micro Finance Institutions, SACCO and mainstream banks (at individual and Savings Group Level);
• The linkages created with the NCPD and INGOs implementing VSLAs, linkage to government social protection programmes (Para Social model and VUP).
• Thecollaboration with UR and other NGOs
• Analyze gender balance and impact programme had on women and girlsempowerment and children with disabilities;
• Sustainabilityof the project results (link with existing government programmes).

In addition to the above component, a qualitative analysis of beekeeping / honey production and improved banana production to develop qualitative case studies to complement the end line evaluation will be undertaken independently in parallel.

This will help to:

• Understand the outputs and outcomes brought about by the ZUUN project beekeeping and improved banana production project activities for persons with disabilities and their families.
• Demonstrate how to manage, promote and use the knowledge generated in the course of related project activities, and future project development.

The major purpose of beekeeping / honey and improved banana production case studies analysis is to draw lessons and obtain recommendations from the end line external review that can offer inputs to improve quality in designing follow up beekeeping / honey and improved banana (good agricultural practices) activities for similar sustainable livelihood projects going forward.

This collected information and analysis will serve as a foundation for identifying potential opportunities to foster growth and advancement of beekeeping, improved agricultural production practices (banana cultivation). To assist with the development of future NUDOR / CBM Rwanda disability inclusive sustainable livelihood activities as income generation / livelihood opportunity for persons with disabilities / VSLGs.

The final Evaluation has the following principal tasks:

• Evaluate the relevance, coherence, effectiveness, efficiency, impact, and sustainability of project;
• Assessthe project design in terms of its relevance to the overall development situation at the national level; relevance to national strategies, relevance to NUDOR mandate and relevance to beneficiaries;
• Assessthe cost-effectiveness and efficiency of the project interventions;
• Assessthe socio-inclusion situation impact on improving the socio-economic participation of persons with disabilities in the community;
• Assessthe relevance and effectiveness of the project’s strategies and approaches for achievement of the project objectives;
• Assess the performance of the project in terms of effectiveness, efficiency, and timeliness of producing the expected outputs;
• Assess the quality and timeliness of inputs, the reporting and monitoring system and the extent to which these have been effective;
• Assessthe relevance of the project’s management arrangements, identify strengths, weaknesses, and lessons learnt with regards to the project
• Analyseunderlying factors beyond CBM control that affected the achievement of the project’s results;
• Provide recommendations to key project partners and stakeholders for the next period of the project;
• Assess whether the project is meeting the results (outputs) and overall objective (outcome).
• Make an economic analysis and honey production cost of beekeeping: Developa case study of beekeeping and honey production cost of beekeeping VSLG and individual income generation activities of households supported by ZUUN projectDetermine the level of technology-use, and to specify the number of hives (fromtraditional to modern) to be supported by determining what should be the minimum production capacity.
• Analyse the outcome indicators in beekeeping enterprises within VSLGs, banana production, and improved sustainable agricultural practices at householdlevel (profitability, productivity, income from beekeeping, banana production, improved agricultural