Terms of Reference for Evaluation

Introductory Note

The following is a guidance for drafting the Terms of Reference for any kind of evaluation. A TOR should cover sufficiently what and why you want to evaluate, and how this is going to be organised. It shall guide you through the evaluation process.

TORs can vary in their level of detail depending on your goal and the objective of the evaluation. An evaluation should be consistent with the size, complexity and strategic importance of the evaluated project/programme.

The following is a rather comprehensive outline of a TOR that needs adjustment according to the needs of the individual evaluation and donor. The scope of the evaluation questions and evaluation criteria to be covered can vary and shall be decided by the stakeholders. The green boxes include some examples which shall be deleted once the project’s very own TOR draft is finalized.

Similar Jobs in Rwanda
Learn more about Christian Blind Mission
Christian Blind Mission jobs in Rwanda

Evaluation Summary

1.  Background of Project

CBM started working with the National Union of the Disability Organizations of Rwanda (NUDOR) since 2015 until now. NUDOR exists to strengthen the voice of the disability movement in Rwanda. It is an umbrella organisation established in 2010 and composed by 15 Organisations of persons with disabilities. Together NUDOR and its fifteen members are working so that persons with disabilities can enjoy the equal rights to which they are entitled.

CBM’s and NUDOR partnership was established and formalized through contracts and Memorandum of Understanding (MoU) on Promoting the Livelihood of Persons with Disabilities in 18 districts and the project entitled “Promotion of anti-discrimination practices for Persons with Disabilities in 5 districts in Rwanda”.

The project was targeting the following direct and indirect target group:

The direct target group of the project are OPDs, which are predominantly nationally active, two organizations for other marginalized groups (Rwanda Network of People Living with HIV / AIDS "and the" Communauté des Potiers du Rwanda "(COPORWA), 10 human rights organizations and 180 community members, 600 persons with disabilities from the target districts, 300 young people and 100 social service providers. Also, include 226 government officials (36 at national level, 180 at the local level; 10 police officers) to the direct target group.

The indirect target group includes all persons with disabilities in Rwanda because of the achievement of the milestones in the project all persons with disabilities - including future generations - will benefit. According to the results of the National Census of 2012, at least 446,000 people live in Rwanda with a disability.

The overall objective of the project “Promotion of anti-discrimination practices for Persons with Disabilities in Rwanda” is that by 2022, the rights of people with disabilities are implemented in Rwanda in accordance with the UN Convention on the Rights of Persons with Disabilities.

Under the overall objective, the following outputs will be reached through a combination of the following specific objectives:

• Objective 1: The community members, social service providers and officials are sensitized and their capacities are strengthened in the field of human rights and the rights of people with disabilities.
• Objective 2: Close cooperation has been established between DPOs and Rwandan government institutions in promoting the human rights of people with disabilities, inclusion and anti-discrimination.
• Objective 3: The capacities of DPOs and HROs are strengthened to monitor and promote the integration of international human rights instruments into national and local policy and planning.
• Objective 4: The capacity of at least 600 people with disabilities in the 5 target districts to claim their rights has been strengthened and they are actively working for these rights.

In order to achieve those objectives, the project`s action plan considers the following interventions:

Objective 1: The community members, social service providers and officials are sensitized and their capacities are strengthened in the field of human rights and the rights of people with disabilities.

1.1 Documentation and Evaluation of the current understanding of the target groups in terms of international instruments on human rights and the rights of persons with disabilities.

1.2 Training and awareness of national and local authorities to international human rights, rights for people with disabilities and their integration into the national legal framework.

3.3 Training and awareness of social service providers to justice, human rights and anti-discrimination.

4.4 Training of community members on the rights of persons with disabilities, gender equality and anti-discrimination establishment of an annual national forum for dialogue on human rights with regard to people with disabilities and other marginalized groups preparation, translation and dissemination of international and national human rights instruments into accessible and easily understandable formats.

Objective 2: Close cooperation has been established between DPOs and Rwandan government institutions in promoting the human rights of people with disabilities, inclusion and anti-discrimination.

2.1. Analysis of "inclusiveness" of existing. Policies and programs towards

 international.

2.2. Meetings with government agencies to advocate for and protect the rights of people with disabilities. Human rights conventions and treaties

2.3. Advocacy meetings of DPOs for creating participatory political. Reviews and

follow-up measures with government institutions

2.4. Discussions with parliamentarians and other policy makers on legislation,

position papers of DPOs amongst others.

2.5. Annual meetings with local authorities to ensure that recommendations from public debates are taken into account.

Objective 3: The capacities of DPOs and HROs are strengthened to monitor and promote the integration of international human rights instruments into national and local policy and planning.

3.1 Consultation with HROs and cooperation regarding the monitoring and reporting of discrimination cases.

3.2 Training of DPOs and HROs for Human Rights Monitoring, reporting, prevention of discrimination cases.

3.4 Carrying out situation analyses and collecting reports on discrimination cases of marginalised groups.

3.5 Public Forums in the communities on issues such as discrimination and the rights of persons with disabilities.

3.6 Conducting radio and TV talk shows on human rights, rights of persons with disabilities and anti-discrimination.

3.7 Training and ongoing support of DPOs and HRO on leadership, governance and advocacy / lobbying.

3.8 Documentation of the impact of the project on the promotion of human rights and anti-discrimination.

3.9 Ten workshops to raise awareness and mobilize young people for the issue of disability.

Objective 4: The capacity of at least 600 people with disabilities in the 5 target districts to claim their rights has been strengthened and they are actively working for these rights.

4.1 Training of 600 people with disabilities to UNCRPD and other human rights instruments

4.2 awareness through annual awareness campaigns on the International Day of People with Disabilities

4.3 Establishment of local platforms as channels for reporting discrimination and rights violations and supporting annual general meetings

2. Evaluation Objective, Scope and Intended Use

The final evaluation will focus on the impact of the project ”Promotion of anti-discrimination practices for Persons with Disabilities in Rwanda” during the implementation period of 15 May 2019 to 31 January 2023.

The evaluation will assess the extent to which partnerships have been established, capacities have been built, and that implemented activities have achieved the results intended for the project.
It will also assess the implementation strategy itself and recommend areas for improvement and learning towards achievement of objectives aligned with NUDOR and CBM missions.

The final evaluation will intend to provide evidence of the project impact at final evaluation with a specific focus on what worked and what didn’t in order to allow for maximisation of impact.

2.1. Scope of the evaluation

The objective of this final evaluation is review and analyse the overall project implementation according to the project plan from the project start up to the moment of final evaluation takes place. It will reflect on the process of developing the project, examine how far the project is progressing towards its overall and specific objectives, understand problems that need to be addressed and what lessons can be drawn.

It is also intended to be used primarily by NUDOR, CBM and BMZ to inform necessary and/or desirable adjustments and improvements.

The scope of the evaluation is the entirety of activities as per Project Contract.

The evaluator will be required to evaluate the following:

• The relevance, coherence, effectiveness, efficiency, impact, and sustainability of the project.
• The achievement of outputs (results) and specific objective (outcome) and related delivery of inputs as per Project Contract/Logframe to measure if the project is meeting its targets.
• Evaluate critical issues relating to results achievement and the factors affecting successful implementation of the project.
• Generate lessons learned from the implementation of the project’s activities and the outcomes achieved that will be useful for similar projects in future for same context; Identify the good practices that can be reached to scale in other projects.
• Develop specific recommendations for major stakeholders to promote sustainability and long-term impact to the beneficiary communities.
• The strategic positioning and partnerships.
• Sustainability of results and exit strategy.

The Final evaluation is expected to cover the following project components:

• The entirety of activities as per Project Contract and other project document such as final project proposal including the logframe.
• Partner project Management, implementation and collaboration with other stakeholders at national and local level.

• NUDOR Engagement with government authorities on key policy issues on behalf of Persons with Disabilities at all levels and Advocacy for anti-discrimination and mainstreaming agenda in both government and civil society.
• OPDs involvement : Representation of people with disabilities in different structures, Community mobilisation (disability activists), advocacy and lobbying, Promotion of  the rights of persons with disabilities, promotion of socio-economic inclusion of persons with disabilities, identification of challenges faced by people with disability and advocacy for change, Mainstreaming disability into government and NGOs programmes, influencing national legal framework to ensure if it is disability inclusive, Training on disability and how they worked with the national human rights institutions.
• National Authorities (Gov.t and Ministries; member of Parliament): analyse how they are playing a central role in ensuring that new policies and programmes are not discriminatory of Persons with Disabilities and other marginalised groups, ensuring that existing policies and programmes are adapted to include Persons with Disabilities , establishing specific policies and laws protecting the rights of persons with disabilities and how planning and budgeting are inclusive for persons with disabilities.

• Local Authorities collaboration: crucial role in district planning and service delivery in the district as well as ensuring rights’ infringements are acted upon and how they are contributing in community mobilization on issues regarding persons with disabilities.
• Social Service Providers: they are the entry points of Persons with Disabilities in accessing public services, ensuring if their services are disability inclusive, sensitize their staff and beneficiaries on the rights of persons with disabilities.
• HROs: Analyze how they are promoting protection of the human rights at national and local level, supporting communities and vulnerable groups in cases of rights’ abuse and infringements, discrimination and violations and education and awareness raising on human rights international and national frameworks.
• Communities involvement to support and participate in Local Platforms in monitoring human rights and in reporting case infringements, community development.
• Media: on campaigns and critical support to advocacy/lobbying efforts general public, and amplify the points of view of a traditionally marginalised group.

The Final evaluation has the following principal tasks:

• Evaluate the relevance, coherence, effectiveness, efficiency, impact, and sustainability of project;
• Assess the project design in terms of its relevance to the overall development situation at the national level; relevance to national strategies, relevance to NUDOR mandate and relevance to beneficiaries;
• Assess the cost-effectiveness and efficiency of the project interventions;
• Assess the socio-inclusion situation impact on improving the socio-economic participation of persons with disabilities in the community;
• Assess the relevance and effectiveness of the project’s strategies and approaches for achievement of the project objectives;
• Assess the performance of the project in terms of effectiveness, efficiency, and timeliness of producing the expected outputs;
• Assess the quality and timeliness of inputs, the reporting and monitoring system and the extent to which these have been effective;
• Assess the relevance of the project’s management arrangements, identify strengths, weaknesses, and lessons learnt with regards to the project Analyse underlying factors beyond CBM control that affected the achievement of the project’s results;
• Provide recommendations to key project partners and stakeholders for the next period of the project;
• Assess whether the project is meeting the results (outputs) and overall objective (outcome).

 2.2.  Target audience and intended use

The evaluation shall provide a detailed account of the achievements and challenges of the project for the partner organisation, for CBM Country Office and

other stakeholders.

The final report shall be used as a demonstration of accountability to the donor/BMZ. An Executive Summary and graphical depictions of findings together with a limited number of useful recommendations shall allow stakeholders to use it for adjustment and learning.

Moreover, it shall be used by partners as a feedback by the target group and how well intended results and positive (or negative) results have been achieved. A specific format such as a slide presentation or poster in accessible format shall be used.

The review team should conduct the process in a participatory manner as much as possible and enable all stakeholders, including CBM CO staff, to reflect on their role in the design and implementation of the project itself. Their perspectives on the process and results, identifying strengths and weaknesses, successes and failures, enabling and inhibiting factors as well as challenges should be sought and incorporated.

The above audience will also be including NUDOR board member’s representative and individual key people with disabilities that were involved in the project implementation in supporting for the organisation of awareness-raising campaigns, technical support in the development of training modules and implementation of training measures and participation in project-related meetings in a supportive capacity.

3.  Evaluation Questions

The Evaluation shall measure the progress of the project and assess the relevance, coherence, efficiency, effectiveness, impact, and sustainability of the interventions in line with project outcome and output deliverables. Hereby, the final evaluation shall refer to timelines, budget and quality of implementation against project targets as per Project Contract.

Moreover, the evaluation needs to consider Gender, Child Safeguarding, Disability Inclusion and environment aspects as mandatory areas of enquiry. The evaluation will identify and analyse good practices and success stories for sharing and learning.

The following key questions should be used as a guidance for the review team to address core aspects of the enquiry.  

• Relevance, quality of design and appropriateness:

• Are the activities and outputs of the project consistent with the intended impacts and effects?
• Is the project aligned with national government’s policies and strategies?

• Coherence

• How well did the interventions fit?
• To what extent other interventions (particularly policies) support or undermine the intervention, and vice versa
• How relevant is the project to future interventions or plan in other parts of the region?
• How do you assess the synergies and interlinkages between the intervention and other interventions carried-out by the same institution/government, as well as the consistency of the intervention with the relevant international norms and standards to which that institutions/government adheres (internal)?
• How do you assess the consistency of the intervention with other actors’ interventions in the same context, including complementarity, harmonization and coordination with others, and the extent to which the intervention is adding value while avoiding duplication of efforts (external)?

• Effectiveness:

• To what extent were the planned objectives achieved? Specifically, were the annual monitoring indicators (outputs) and final impact indicators (effects and impacts) successfully reached?
• Have beneficiaries been satisfied with the services received and do they note a noticeable gain?
• To what extent did the project contribute to strengthening the capacities in the project area?
• What are the measurable success indicators since the start of project implementation?
• What lessons learned can you draw from the project to increase the effectiveness of future projects?
  • Efficiency or cost-effectiveness (of planning and of implementation):

• Have project funds been spent appropriately? Was the achievement of project objectives on time?
• How does the project co-ordinate with other similar interventions to encourage synergy and avoid overlaps?
• How could the efficiency of the project be increased? What recommendations do you have for future projects?

3.4   Impact - Contribution to change:

• What real difference has the project made to the beneficiaries?
• Which changes took place based on the project´s interventions in terms of attitude, practice or effects on individual / family / community / institutional level?
• How is the project perceived by the public? Does it receive visitors, requests etc.? Does it receive media attention and coverage?
• What contribution has the project made in terms of establishing an inclusive society?
• What should be considered to increase the impact of future projects?

3.5.   Sustainability:

• To what extent are the project’s positive actions likely to continue after the end of the project?
• Have the projects awareness raising activities and advocacy work led to a change in the behaviour of the target groups?
• What are lessons learned from this project? What specific future needs can the review team recommend to be prioritized for future institutional support activities?
• What were the most significant constraints and/or difficulties in implementing the project and, where appropriate, how did the project overcome them? What lessons learned does the consultant team identify that have implications for future program design?
• How could activities have reached an even higher level of sustainability?

3.6 Gender:

• Has there been any change in the participation of women and men in community, social, economic or political/decision-making life within the target area? What is the link between the work of our project and these changes?
• Has there been any change over the project period to women and men with disabilities participating in community decision-making processes? Any change in the power distribution?

3.7 Child Safeguarding

• What measures have been put into place to ensure the partner organization and any other actors involved respect the safeguarding principles? What kind of control mechanism is in place?
• Assess whether there have been any project activities that have created unforeseen negative impacts on children’s safety.
• What feedback mechanisms are in place to report the violation of child safeguarding?

3.8 Disability Inclusion

• Did women and men with a disability equally gain a better understanding of their rights and entitlements?
• Were women, men, girls and boys with a disability equally able to access the full range of services offered?
• Has the community formed a deeper respect and understanding for people with a disability and their families including decreased stigma and discrimination and increased appreciation of capacities and contribution?
• Have organisational policies, procedures and project decisions been reviewed and updated to be disability inclusive?

• Environment
• In term of ecology, how was the environmental project impact? If not, explain what is needed.
• How is the climate change affecting the project beneficiaries living conditions?
  • Scalability
• Were any unique activities (good practice) identified?
• What aspects of the program are suitable for replication?
• Do the necessary conditions and capacity for scale-up exist within the partner, target region, and relevant agencies & government?

4.Methodology

This evaluation will be carried out using mainly mixed methods to review documents and conduct interviews with key project staffs, CBM CO staff and other stakeholders involved in the project planning and implementation. The review will collect quantitative and qualitative data from baseline information and monitoring systems including statistics, narrative reports, financial reports, and other relevant documentation.

The evaluator will take into consideration the implementation status of the project and the resource disbursements made from May 2019 to 15^th January 2023.

The evaluation shall be conducted using rights-based principles, i.e. participatory, inclusive and following a do-no-harm approach. Data security and privacy issues must be adhered to during data collection, management as well as reporting. Ideally, the evaluation team shall include a mixed gender team and be inclusive of a person with disability.

The methodology should consider:

• Employ the following relevant evaluation criteria: Sustainability, Coherence, Relevance, Effectiveness, Efficiency, Impact, Gender, Child Safeguarding, Environment, Disability Inclusion

• Demonstrate impartiality and lack of biases by relying on a cross-section of information sources (stakeholder groups, beneficiaries, key informants etc).
• Using mixed methods (quantitative and qualitative) to ensure
  triangulation of information through a variety of means.
• Contain a sampling strategy, including the sampling method and sample size calculations.
• Ensure comparability to the baseline indicators. 
• Apply an evaluation matrix geared towards addressing the key evaluation
  questions taking into account the data availability challenges, the budget and timing constraints.

5.Limitations

The following limitations are identified:

• Accessibility: According to the target groups, the focus group discussions facilitated by sign language interpreters and an on-site visit component will be included on accessible places.

• Number of visits/interviews: Due to the limited time for the review itself, it is envisaged that the team will not be able to visit all targeted groups or interview all institutions as well. Efforts will be made to ensure that institutions and stakeholders visited include officials and that stakeholders interviewed are as official representatives as possible.

6.Evaluation Team and Management Responsibilities

6.9. Commissioning responsibility

The evaluation will be under the responsibility of the CBM Rwanda Country Office. However, as CBM Christoffel-Blindenmission Christian Blind Mission e.V. holds the contract with BMZ, respective approaches in terms of methodology, final reports etc. will be aligned with CBM-IO for their input and alignment. The local implementing partner NUDOR will be involved in drafting the Terms of Reference, reviewing and providing their input to the final report in close communication with CBM in form of a Management Response.

6.10.  Evaluation Team

A team or individual consultants consisting of participants from a variety of experience in evaluating complex projects is required.

It is strongly desired that a suitably qualified review team member or consultant with a disability, ideally from Rwandan OPD with experience of the national disability context and development context, community development perspectives, is a member of the review team.

To maintain objectivity, no member of the partner board of directors and partner staff should be part of the review team, although of course coordinating closely with the team itself. However, in order to maximise learning, there should be sufficient time for debrief and discussion between team members, the partner and the CBM country office.

The CBM CO and the project manager at NUDOR will accompany where possible and inform the review team, inform and clarify issues, but at the same time ensure the team has enough space to interview partners, key staff, key stakeholders on their own, discuss among themselves and develop their findings independently.

The evaluation team is expected to be medically insured during the time of service under this consultancy. The contracting agencies, CBM and NUDOR, will not take responsibility for any damage, injury, or loss in the framework of this evaluation.

Child Safeguarding Policy: As a condition of entering into a consultancy agreement the evaluators must sign the CBM Child Safeguarding Policy and abide by the terms and conditions thereof.

6.11.  Management of the evaluation and logistics

NUDOR (the partner) will provide relevant project documents (as but not limited to Project Contract including budget, baseline, mid-term evaluation, monitoring and annual reports) as necessary to the evaluation team and arrange meetings and local logistics in close alignment with the Evaluation Team. The partner must make sure that venues are accessible for persons with disabilities.

The consultant shall ensure that appropriate translation or interpretation is covered as needed. This also includes potential provision of sign language interpretation. Again, CBM or the partner organisation can support to locate suitable services as needed.

The CBM Country Office Rwanda will facilitate the contact between the Evaluation Team and the local partner. It has the responsibility to make sure that relevant CBM stakeholders are aligned and available for discussions as well as provide their feedback to the draft evaluation report. Furthermore, the evaluation will be kicked-off with a briefing meeting between the partner organisation, the CBM CO to the Evaluation Team. A final wrap-up meeting shall be conducted by the same parties. The CO will facilitate discussion/review of the draft evaluation report with the partner and disseminate the final report to all concerned and the donor (BMZ).

CBM-IO will make sure to be available for a briefing discussion as well as will provide feedback to the draft evaluation report. CBM-IO will make sure to share the evaluation report with BMZ for their information and further follow up.

7.Expected Results

• Share and present the inception Report: 27 December 2022
• Share the draft Evaluation Report and present key findings: 14^th January 2023
• Submit Final Evaluation Report: 24^th January 2023.

The Review Team leader is expected to submit a report complemented with attachments as necessary. The report should be written in English. After receiving the comments, the Team Leader will finalize the report and submit to CBM. The final version of the report will be shared with CBM IO and MA, and partners as deemed necessary by CO management.

These following expected result documents will be developed as a result of the process:

Inception Report (due by: 27 December 2022) – The Inception report will include the proposed methodologies, data collection and reporting plans with draft data collection tools such as interview guides, a timeframe with firm dates for deliverables, and the travel and logistical arrangements for the team.

First Draft Report (due by: 14^th January 2023) – A draft report, consolidating findings from the review, identifying key findings, conclusions, recommendations and lessons learnt for the current and future similar projects, will be submitted to CBM CO. The suggested table of contents for the evaluation report is indicated in the annex.

Final Report (due by: 24^th January 2023) – The final report will contain a short executive summary (no more than 3 pages) and a main body of the report. The main body of the report covers the background of the intervention reviewed, a description of the review methods and limitations, findings (to be presented by review criteria), conclusions, lessons learned, and clear recommendations.

Recommendations should also outline recommendations that the project staff and the reviewers have in common or different views based on the feedback sessions to discuss the findings.  The report should also contain appropriate appendices, including a copy of the ToR, cited resources or bibliography, a list of those interviewed and any other relevant materials (e.g., tools, questionnaires).  The final report will be submitted in 3 days after reception of the consolidated feedback from CBM

8. Duration and Phasing

The service must take place starting from 26 December 2022 to 24^th January 2023. The following delivery dates should be set by mutual agreement between CBM and the evaluators, based on the following timetable:

Timeframe: The timeframe of the review is expected to use 30 days consisting of the following steps:

 9.  Costs and payments

The following batches of payment will be reimbursed upon receipt of proper invoices:

• 30 % of the consultancy cost to be paid at the contract signature and submission of inception report agreed upon.
• 40% of the consultancy cost to be paid when the draft report is submitted.
• 30% of the consultancy cost to be paid once the final report is approved.

The transport and accommodation costs will be covered by the evaluator.

CBM CO reserves the right to terminate the contract in case that the agreed team members are not available at the agreed commencement of the assignment and if no adequate replacement can be provided.

10. Application

Interested consultants shall submit their offer to the following address:  info.rwanda@cbm.org by 19^th of December 2022. The offer shall also include the following:

• Letter of motivation;
• Detailed Curriculum Vitae (CV) of the suggested team members with three references from third parties;
• Licences;
• Copy of degrees;
• Technical proposal describing how the consultant intends to undertake this assignment including suggested methodology, implementation structures and schedule;
• Separate financial proposal including all applicable taxes;
• At least three certificates of good completion of similar assignments;
• RDB Registration Certificate and Tax Clearance including TIN Number from RRA for companies;
• Confirmation of availability during the time frame given above;
• Incomplete offers or applications received after the deadline will not be considered.

CBM contact phone number: +250 78 831 6544

Appendix A: Suggested overall structure / table of contents for evaluation report

Executive Summary

• Introduction
• Background (Program description and purpose of evaluation)
• Methodology and Implementation
• Results and Discussion
• Conclusions
• Recommendations
• Lessons Learned
• Annexes
  1. Performance indicators with updated values in comparison to baseline values
  2. List of meetings
  3. Survey instruments ToR
  4. List of Key informant interviewee
  5. Other relevant for report

Appendix B: Stakeholder Analysis for Evaluation.

NUDOR will work in close collaboration with a range of stakeholders in order to gain maximum impact on the expected results of this action. This action will create impact at both National and district level. Kigali 9^th December 2022

Eugenie Mukantagwera

CBM Country Director, Rwanda