Evaluation Summary 

Simlar jobs in Rwanda
Learn More About Christian Blind Mission (CBM)
Christian Blind Mission (CBM) jobs in Rwanda

1.  Background of Project

eveloping the project, examine how far the project is progressing towards its overall and specific objectives as well as its results/outputs (i.e. review the overall implementation according to the project plan from project start to date), understand problems and challenges that need to be addressed and what lessons can be drawn for the project part duration and give recommendations for improvement to address identified challenges. 

It is also intended to be used primarily by NUDOR, CBM and BMZ to inform on necessary and/or desirable adjustments and improvements to the project and its implementation. 

The evaluator will be required to evaluate the following:

• The results achievement to measure if the project is making satisfactory progress in timely achievement of outputs as per logframe and related delivery of inputs.
• Evaluate other critical issues relating to results achievement and the factors affecting successful implementation of the project.
• Evaluate the overall project set-up and implementation along the OECD/DAC criteria
• The strategic positioning and partnerships.
• Sustainability of results and exit strategy.

The Mid-Term Evaluation is expected to cover the following project components:

• Project Management and implementation and collaboration with other stakeholders at national, local level and NUDOR Level;
• VSLG member’s engagement and VSLGs activities including savings, loans, social funds, Income Generating Activities;
• The Groups cohesion and effectiveness of leadership;
• The consistency in savings and loan repayment incl. default rates;
• The dropout rate and reasons for the same;
• Assess whether persons with different disabilities are equitably participating in VSLAs including persons with deaf blindness
• The VSLGs’ Financial Linkage to Micro Finance Institutions, SACCO and mainstream banks; 
• The linkages created with the NCPD and INGOs implementing VSLAs.
• The collaboration with UR and other NGOs
• Analyze gender balance and impact programme had on women empowerment and children with disabilities;
• Sustainability of the project results (link with existing government programmes). 

The Mid-term Evaluation has the following principal tasks: 

• Assess the project design in terms of its relevance to the overall development situation at the national level; relevance to national strategies, relevance to NUDOR mandate and relevance to beneficiaries;
• Assess the cost-effectiveness and efficiency of the project interventions;
• Assess the socio-inclusion situation impact on improving the socio-economic participation of persons with disabilities in the community;
• Assess the relevance and effectiveness of the project’s strategies and approaches for achievement of the project objectives;
• Assess the performance of the project in terms of effectiveness, efficiency, and timeliness of producing the expected outputs;
• Assess the quality and timeliness of inputs, the reporting and monitoring system and the extent to which these have been effective;
• Assess the relevance of the project’s management arrangements, identify strengths, weaknesses, and lessons learnt with regards to the project Analyse underlying factors beyond CBM control that affected the achievement of the project’s results;
• Provide recommendations to key project partners and stakeholders for the next period of the project.

2.2.     Target audience and intended use

The evaluation shall provide a detailed account of the achievements and challenges of the project for the partner organisation, for CBM Country Office and other stakeholders.

The final report shall be used as a demonstration of accountability to the donor/BMZ. An Executive Summary and graphical depictions of findings together with a limited number of useful recommendations shall allow stakeholders to use it for adjustment and learning. Moreover, it shall be used by partners as a feedback by the target group and how well intended results and positive (or negative) results have been achieved. A specific format such as a slide presentation or poster in accessible format shall be used.

The evaluation team should conduct the process in a participatory manner as much as possible and enable all stakeholders, including NUDOR and CBM CO staff, to reflect on their role in the design and implementation of the project itself. Their perspectives on the process and results, identifying strengths and weaknesses, successes and failures, enabling and inhibiting factors as well as challenges should be sought and incorporated.

The above audience will also be including NUDOR board member’s representative and individual key people with disabilities that were/are involved in the project implementation in supporting for the organisation of awareness-raising campaigns, technical support in the development of training modules and implementation of training measures and participation in project-related meetings in a supportive capacity.

3.  Evaluation Questions

The Evaluation shall measure the progress of the project and assess the relevance, coherence, coverage, efficiency, effectiveness, impact, and sustainability of the interventions in line with project objective and output deliverables. Hereby, the mid-term evaluation shall refer to timelines, budget and quality of implementation against project targets as per Project Contract.

Moreover, the evaluation needs to consider Gender, Child Safeguarding, Disability Inclusion and environment aspects as mandatory areas of enquiry. The evaluation will identify and analyse good practices and success stories for sharing and learning.

The following key questions should be used as a guidance for the evaluation team to address core aspects of the enquiry.   

• Relevance, quality of design and appropriateness:

• Was the project designed in ways that respond to the needs and priorities of all participants (irrespective of gender, age, disability status)?
• Are the activities and outputs of the project consistent with the overall goal and the attainment of its objectives?
• Are the activities and outputs of the project consistent with the intended impacts and effects?
• To what extent does/did the project design reflect the rights of persons with disabilities and other marginalised groups and include feedback from a diverse range of local stakeholders? Is the project aligned with national government’s policies and strategies? 

• Coherence

• How well do the interventions fit?
• To what extent do other interventions (particularly policies) support or undermine the intervention, and vice versa?
• How relevant is the project to future interventions or plan in other parts of the region?
• How do you assess the synergies and interlinkages between the intervention and other interventions carried-out by the same institution/government, as well as the consistency of the intervention with the relevant international norms and standards to which that institutions/government adheres (internal)?
• How do you assess the consistency of the intervention with other actors’ interventions in the same context, including complementarity, harmonization and coordination with others, and the extent to which the intervention is adding value while avoiding duplication of efforts (external)? 

• Coverage:

• How have the population groups including the most marginalised been reached? 

• Effectiveness:

• To what extent have the planned objectives been achieved so far? Specifically, to what extent have the annual monitoring indicators (outputs) and final impact indicators (effects and impacts) been successfully reached so far?
• Are beneficiaries satisfied with the services received and do they note a noticeable gain?What were the major factors influencing the achievement or non-

achievement of the objectives?

• To what extent does the project contribute to strengthening the capacities in the project area?
• What are the measurable success indicators since the start of project implementation?
• What lessons learned can you draw from the project to increase the effectiveness of the project in the second wave of the implementation period and of future projects? 

• Efficiency or cost-effectiveness (of planning and of implementation):

• Have project funds been spent appropriately? Was the achievement of project objectives on time?
• How does the project co-ordinate with other similar interventions to encourage synergy and avoid overlaps?
• How could the efficiency of the project be increased?

What recommendations do you have for future projects?

• Are existing lessons learned such as reflection, internal review sufficiently used to keep project plan updated? 

3.4   Impact - Contribution to change:

• What real difference has the project made to the beneficiaries?
• Which changes took place based on the project´s interventions in terms of attitude, practice or effects on individual / family / community / institutional level?
• How is the project perceived by the public? Does it receive visitors, requests etc.? Does it receive media attention and coverage?
• What contribution has the project made in terms of establishing an inclusive society?
• What was an effect that has not been anticipated or expected?
• Were there also negative effects and consequences?
• What should be considered to increase the impact of future projects? 

• Sustainability:

• What are the obstacles faced during the project implementation and those putting the project sustainability at risk? What are mitigating and coping recommendations for the future?
• What are the major factors influencing the sustainability of the project?
• How did persons with disabilities participate in the project design, implementation and monitoring or other assessment?
• To what extent are the project’s positive actions likely to continue after the end of the project?
• Have the projects awareness raising activities and advocacy work led to a change in the behaviour of the target groups? If yes, in what respect?
• What are lessons learned from this project? What specific future needs can the evaluation team recommend to be prioritized for future institutional support activities?
• What were the most significant constraints and/or difficulties in implementing the project and, where appropriate, how did the project overcome them? What lessons learned does the consultant team identify that have implications for future program design?
• Is the feedback system effectively contributing as an information tool to improve the services/activities/trainings?
• How could activities have reached an even higher level of sustainability? 

• Gender:

The evaluation must consider gender aspects of the project and report on this. Data collected must be disaggregated according to gender.

• Has there been any change in the participation of women and men in community, social, economic or political/decision-making life within the target area? What is the link between the work of our project and these changes?
• Has there been any change over the project period to women and men with disabilities participating in community decision-making processes? Any change in the power distribution?
• In what ways are females and/or women-headed households empowered via the VSLAs and how can this be measured?
• How does the project lead to a stronger participation of females and/or women-headed households within community-based decision-making, political envolvement?
• In case women are empowered – how are they perceived and accepted by male village authorities? 

• Child Safeguarding

The evaluation team must keep in mind to adhere to child safeguarding ethics during the entire evaluation process if children are involved. The evaluation must consider child safeguarding aspects and report on this. 

• How has a safe environment for children been established and maintained throughout the stages of planning and implementation?
• Assess the awareness of involved stakeholders about the rights of children and about child protection/safeguarding and what prevention and protection activities were undertaken.
• What measures have been put into place to ensure the partner organization and any other actors involved respect the safeguarding principles? What kind of control mechanism is in place?
• Assess whether there have been any project activities that have created unforeseen negative impacts on children’s safety.
• What feedback mechanisms are in place to report the violation of child safeguarding? 

• Disability Inclusion

• Did women and men with a disability equally gain a better understanding of their rights and entitlements?
• Were women, men, girls and boys with a disability equally able to access the full range of services offered?
• Has the community formed a deeper respect and understanding for people with a disability and their families including decreased stigma and discrimination and increased appreciation of capacities and contribution? Have project personnel received knowledge and training on the specific needs of women, girls, men and boys with a disability?
• Have organisational policies, procedures and project decisions been reviewed and updated to be disability inclusive? 

• Environment

• In term of ecology, how was the environmental project impact? If not, explain what is needed.
• How is the climate change affecting the project beneficiaries living conditions? 

• Scalability

• Were any unique activities (good practice) identified?
• What aspects of the program are suitable for replication?
• Do the necessary conditions and capacity for scale-up exist within the partner, target region, and relevant agencies & government?

4.  Methodology

The evaluation shall be conducted using rights-based principles, i.e. participatory, inclusive and following a do-no-harm approach. Data security and privacy issues must be adhered to during data collection, management as well as reporting.

Ideally, the evaluation team shall include a mixed gender team and be inclusive of a person with disability. The project team can provide support in this. 

The project team will share a matrix of key partners and stakeholder with the evaluator so that s/he is aware on who to include in the sampling. 

This evaluation will be carried out using mixed methods to review documents and conduct interviews with key project staffs, CBM CO staff, project beneficiaries from the various target groups and other stakeholders involved in the project planning and implementation. The review will collect quantitative and qualitative data from baseline information and monitoring systems including statistics, narrative reports, financial reports, and other relevant documentation, as well as stakeholder/target group interviews and focus group discussions. 

The evaluator will take into consideration the implementation status of the project and the resource disbursements made from 1 December 2021  to 30^th June 2023. 

The methodology must consider:

• Employ the following relevant evaluation criteria: Sustainability, Coherence, coverage, Relevance, Effectiveness, Efficiency, Impact, Gender, Child Safeguarding, Environment, Disability Inclusion,
• Demonstrate impartiality and lack of biases by relying on a cross-section of information sources (stakeholder groups, beneficiaries, key informants etc).
• Using mixed methods (quantitative and qualitative) to ensure
  triangulation of information through a variety of means.
• Contain a sampling strategy, including the sampling method and sample size calculations.
• Ensure comparability to the baseline indicators. 
• Apply an evaluation matrix geared towards addressing the key evaluation
  questions taking into account the data availability challenges, the budget and timing constraints.

5.  Limitations

The following limitations are identified: 

• Accessibility: According to the target groups, the focus group discussions facilitated by sign language interpreters and an on-site visit component will be included on accessible places. 

• Number of visits/interviews: it is envisaged that the team will not be able to visit all targeted groups or interview all institutions as well. Efforts will be made to ensure that institutions and stakeholders visited include officials and that stakeholders interviewed are as official representatives as possible.

 6.  Evaluation Team and Management Responsibilities

6.1. Commissioning responsibility 

The evaluation will be under the responsibility of the CBM Rwanda Country Office. However, as CBM Christoffel-Blindenmission Christian Blind Mission e.V. holds the contract with BMZ, respective approaches in terms of methodology, final reports etc. will be aligned with CBM-IO for their input and alignment. The local implementing partner NUDOR will be involved in drafting the Terms of Reference, reviewing and providing their input to the final report in close communication with CBM in form of a Management Response.

6.2. Evaluation Team 

A team or individual consultants consisting of participants from a variety of experience in evaluating complex projects is required. 

A suitably qualified evaluation team member or consultant with a disability, experience of OPD works, ideally with experience of the national financial context and development context, high experience of VSLA programme and methodology, understanding of the Community Based Inclusive Development perspectives would be highly desirable. 

In line with best practice for disability inclusive development, a Disabled People’s Organisation representative or person with a disability should participate in the evaluation as an evaluation team member or as a specific evaluation advisory group. 

To maintain objectivity, no member of the partner board of directors and partner staff should be part of the evaluation team, although of course coordinating closely with the team itself. However, in order to maximise learning, there should be sufficient time for debrief and discussion between team members, the partner and the CBM country office. 

The CBM CO and the project manager at NUDOR will accompany where possible and inform the evaluation team, and clarify issues, but at the same time ensure the team has enough space to interview partners, key staff, key stakeholders on their own, discuss among themselves and develop their findings independently. 

The evaluation team is expected to be medically insured during the time of service under this consultancy. The contracting agencies, CBM and NUDOR, will not take responsibility for any damage, injury, or loss in the framework of this evaluation. 

Child Safeguarding Policy and code of conduct: As a condition of entering into a consultancy agreement the evaluators must sign the CBM Child Safeguarding Policy and Code of Conduct Declaration abide by the terms and conditions thereof.

6.2 Management of the evaluation and logistics

NUDOR (the partner) will provide relevant project documents (as but not limited to Project Contract including budget, feasibility study, monitoring reports, semi-annual and annual reports) as necessary to the evaluation team and arrange meetings and local logistics in close alignment with the Evaluation Team. The partner must make sure that venues are accessible for persons with disabilities.

The consultant shall ensure that appropriate translation or interpretation is covered as needed. This also includes potential provision of sign language interpretation. Again, CBM or the partner organisation can support to locate suitable services as needed.

The CBM Country Office Rwanda will facilitate the contact between the Evaluation Team and the local partner. It has the responsibility to make sure that relevant CBM stakeholders are aligned and available for discussions as well as provide their feedback to the draft evaluation report. Furthermore, the evaluation will be kicked-off with a briefing meeting between the partner organisation, the CBM CO to the Evaluation Team. A final wrap-up meeting shall be conducted by the same parties including CBM-IO and initiative. The CO will facilitate discussion/review of the draft evaluation report with the partner and disseminate the final report to all concerned and CBM-IO for donor (BMZ)-reporting.

CBM-IO will make sure to be available for a briefing discussion as well as will provide feedback to the draft evaluation report. CBM-IO will make sure to share the evaluation report with BMZ for their information and further follow up.

7.  Expected Results 

• Share and present the inception Report: 16 September 2023 
• Share the draft Evaluation Report and present key findings: 15 October 2023
• Submit Final Evaluation Report: 30 October 2023 2023. 

The Evaluation Team leader is expected to submit a report complemented with attachments as necessary. The report should be written in English. After receiving the comments, the Team Leader will finalize the report and submit to CBM Rwanda Country Office. The final version of the report will be shared with CBM-IO and BMZ, and partners as deemed necessary by CO management. 

The assignment will start with a briefing with partner and CBM and will include a debriefing session with the local partner at the end of the project site visit summarising the findings and recommendations. 

The evaluation report needs to consider requirements according to the CBM Evaluation Report template and needs to consider the chapters included in the template. Good practices need to be documented for learning of internal and external stakeholders.

These following expected result documents will be developed as a result of the process:

Inception Report (due by: 16 September 2023) – The Inception report will include the proposed methodologies, data collection and reporting plans with draft data collection tools such as interview guides, a timeframe with firm dates for deliverables, and the travel and logistical arrangements for the team.

First Draft Report (due by: 15 October 2023)

A draft report, consolidating findings from the evaluation, identifying key findings, conclusions, recommendations and lessons learnt for the current and future similar projects – considering the topics and guiding questions as outlined above – will be submitted to CBM CO. The suggested table of contents for the evaluation report is indicated in the annex.

Final Report (due by: 30 October 2023) – The final report will contain a short executive summary (no more than 3 pages) and a main body of the report. The main body of the report covers the background of the intervention reviewed, a description of the review methods and limitations, findings (to be presented by evaluation criteria), conclusions, lessons learned, and clear recommendations – considering the topics and guiding questions as outlined above.

Recommendations should also outline recommendations that the project staff and the evaluators have in common or different views based on the feedback sessions to discuss the findings. The report should also contain appropriate appendices, including a copy of the ToR, cited resources or bibliography, a list of those interviewed and any other relevant materials (e.g., tools, questionnaires).  The final report will be submitted in 5 days after reception of the consolidated feedback from CBM.

8.  Duration and Phasing

The service must take place starting from 15 September 2023 until 30 October 2023.The following delivery dates should be set by mutual agreement between CBM and the evaluators, based on the following timetable:

Timeframe: The timeframe of the evaluation is expected to use 45 days consisting of the following steps:

9.  Costs and payments 

The following batches of payment will be reimbursed upon receipt of proper invoices:

• 40 % of the consultancy cost to be paid at the contract signature and submission of inception report agreed upon.
• 40% of the consultancy cost to be paid when the draft report is submitted.
• 20% of the consultancy cost to be paid once the final report is approved.

The transport and accommodation costs will be covered by the consultant

CBM CO reserves the right to terminate the contract in case that the agreed team members are not available at the agreed commencement of the assignment and if no adequate replacement can be provided.

10.              Application

Interested consultants shall submit their offer to the following address:  info.rwanda@cbm.org by 10th of September  2023 . The offer shall also include the following: 

• Letter of motivation;
• Detailed Curriculum Vitae (CV) of the suggested team members with three references from third parties;
• Licences;
• Copy of degrees;
• Technical proposal describing how the consultant intends to undertake this assignment including suggested methodology, implementation structures and schedule;
• Separate financial proposal including all applicable taxes;
• At least three certificates of good completion of similar assignments;
• RDB Registration Certificate and Tax Clearance including TIN Number from RRA for companies;
• Confirmation of availability during the time frame given above;
• Incomplete offers or applications received after the deadline will not be considered. 

CBM contact phone number: +250 78 831 6544