Supporting adolescents to complete secondary (SACS) Project
Terms of Reference for Consultancy to Conduct Endline evaluation
1.0 Background
In response to the diverse effects of COVID-19 on adolescents, the Government of Rwanda (GoR) and its development partners implemented a number of COVID-19 control measures. The Ministry of Education (MINEDUC) has developed and adapted the COVID-19 response plan titled “Keeping the Doors Open for Learning: Response Plan to COVID-19 Outbreak.” The plan has five objective areas: (i) Ensure continuity of learning for all students; (ii) Ensure that schools reopen with appropriate services and measures in place for re-entering the formal education system; (iii) Ensure the health and safety of students, teachers, and other education personnel; (iv) Protect and provide for vulnerable populations, including children with disabilities, girls, and children from lower wealth-quintiles; and (v) Prepare the resilience of the education system against future shocks.

Through the COVID-19 crisis response campaign, CARE Rwanda secured resources to respond to the pandemic’s effects in education through the Supporting Adolescents to Complete Secondary (SACS). SACS is a two (2) and a half-year project running from January 2021 through June 2023, which aims to ensure adolescents in three districts of the Southern Province (Nyanza, Ruhango, Huye) and three districts of the Western Province (Nyabihu, Ngororero and Karongi) reach their full potential by limiting the number of adolescents, particularly girls, at risk of dropping out of school, and supporting those who are out-of-school to re-enroll. The project is increasing access to education and improving acceptance of parents and community members on adolescent education, with a target of 71,994 adolescents in Nyanza, Ruhango, Huye, Nyabihu, Ngororero and Karongi District reached. SACS is implemented by African Evangelistic Enterprise Rwanda (AEE) and led by CARE Rwanda, which plays an overall leadership role and implements field activities.

1.1 Scope of the project
The purpose of SACS is to ensure adolescent girls and boys reach their full potential in Nyanza, Ruhango, Huye Districts and Nyabihu, Ngororero and Karongi Districts by 2023 through the objective and outcomes outlined below.
Objective: Limit the number of adolescents, particularly girls, at risk of dropping out of school.
Project Outcomes:
1. Increased and sustained retention rates of lower secondary students.
2. Increased parents’ resilience, accountability, and responsiveness in supporting children education.
3. Strengthened decentralized education government systems.
To achieve the above outcomes and objective, SACS has been implementing the following activities to limit the number of adolescents at risk of dropping out of school.
• Mentorship by trained teachers to be mentors of students
• Adolescent Sexual and Reproductive Health training
• Financial literacy sessions
• Leadership skills coaching
• Mental health and psychosocial support through guidance and counselling sessions by Association Rwandaise des Con- seillers en Traumatisme ( ARCT-Ruhuka)
• Supporting young girls who dropped out of school to enroll in TVET schools
• Supporting poor Household(students and parents) with a scholarship package and help them start income generating activities to support students education needs
• Linking youth and adults/parents to existing Voluntary Savings and Loan Associations (VSLA)
• School community score card
A detailed workplan and MEAL framework will be given to the consultant.
2.0 Purpose, Objectives, and Rationale
The primary purpose for the end line evaluation is to assess the performance of SACS against its targets as documented in the project’s PMF and measure the impact of the program on participants’ lives. In addition to these objectives, the following are also objectives of this endline assessment:
• To identify factors that affected outcomes
• To identify key project achievements, challenges, best practices, and lessons learned for accountability purposes
• To demonstrate accountability for the funding received by the donor
• To support the development and implementation of the project’s sustainability and succession strategies.
• To inform key stakeholders including project partners, participants, and the government to learn lessons from the project for the purpose of informing education programming in country.
• To inform the wider policy debate amongst donors, academic institutions, and education networks
• To support future similar programming
The period of time the evaluation will cover is the duration of the current project (January 2021 to March 2023). The geographical focus will be the six districts SACS operates in, with sample selection done from 245 schools.
The key indicators, as outlined in the project’s PMF, to be measured through the endline evaluation include the following:
• % adolescents returned to school who stay in school through the end of the school year
• % of adolescents who report anxiety
• % of adolescents who report depression
• % of adolescent girls and boys who report to have increased their knowledge and skills on SRH, GBV and leadership
• % of adolescents who report participating in adolescent-led activities in their communities
• % of adolescents who believe that they can report cases of GBV
• % of adolescent girls and boys in school who report to have increased business and entrepreneurship skills
• % of parents who report increased trust and improved relationships between them and their children/adolescents
• # parents / adolescents who report increased amount of savings spent on education at the end of a savings cycle (Dec 2022) vs at the beginning of a savings cycle (Jan 2022)
• % of women and girls who are engaged in entrepreneurial activities
• % of parents who report enhanced skills on open dialogues between parents and adolescents
• % of students who report increased space both at home and within communities for them to apply leadership competencies and voice their issues/needs
• % of targeted schools which meet Covid-19 government prevention measures at schools
• % of targeted schools with adequate students, teachers and school staff's protection mechanisms including Covid-19 response


Similar Jobs in Rwanda
Learn more about CARE International Rwanda
CARE International Rwanda jobs in Rwanda

2.1 Evaluation Questions
The evaluation will consider the following aspects and ascertain to what extent the changes can be attributed to Supporting adolescents to complete secondary (SACS) Project.
1. To what extent have the project interventions caused reductions in barriers to access and retention in schools after the COVID-19 pandemic?
2. To what extent have project interventions led to a decrease of anxiety and depression among adolescents, and what is the effect of these changes on adolescents’ access to education [transition to next grade and/or completion of education]?
3. Are there changes in adolescents’ level of knowledge on ASRH and GBV and changes in social acceptance and reporting gender-based violence (GBV) and school related GBV (SRGBV)?
4. To what extent have adolescents improved their financial literacy skills?
5. To what extent has community engagement improved as a result of community scorecards? How can it be made more meaningful? How has it impacted accountability? How has this impacted education outcomes?
6. To what extent are adolescents and parents using their money from VSLAs on education related expenses? How have VSLAs impacted students learning, retention, dropout, transition and promotion?
7. How have student clubs impacted adolescent leadership and life skills?
8. How effective were radio campaigns in positively engaging adolescents?
9. What are the most effective methods for challenging agender and social norms to reduce barriers to education?
10. To what extent are parents and community members committed to supporting adolescents’ education? How have the small grants helped parents with setting up income generating activities and has this had an impact on household incomes?
11. How have VSLAs impacted the ability of parents to have a more sustainable stream of income?
12. How have adolescents’ self-perception of leadership competencies changed through the project?
13. To what extent has SACS contributed to strengthening decentralized education systems?
14. Are there any positive/negative, direct/indirect, intended/unintended impacts of the project?
15. To what extent are the benefits likely to be sustained after completion of the project?
16. To what extent have the outcomes been achieved in the most efficient way?
3.0 Evaluation approach and methodology

3.1 Methodology
The evaluation will be completed within two months, starting February 2023 and not later than May 2023. The endline is intended to be a mixed-methods, pre-post design which includes longitudinal tracking of a student cohort to identify changes over time within the same individual. As such, the evaluation team will attempt to re-contact the same individuals that were part of the baseline and midline to follow up with them at the endline using the same tools. A longitudinal study is important in this context for two reasons: i) Measuring retention rates of students, one of the key outcomes for the project, has to be done for the same individual at two points in time; and ii) Aspects of the project, such as leadership skills, may improve naturally over time for an adolescent, as older adolescents tend to have a higher degree of leadership skills; thus, to understand the project’s impact on these aspects, values will be compared against adolescents one year older in age at baseline.
In addition to recontacting the same individuals at baseline, the endline evaluation will also collect data on the following respondents: parents of adolescents/students, teachers/mentors, master trainers, peer mentors, adolescents and parents linked to VSLAs, headteachers, local leaders, government officials (Ministry of Education and Health), CARE and implementing partner project staff, ARCT Ruhuka staff.
3.2 Primary data

Quantitative data
Quantitative data will be collected from 48 schools, equally distributed across the six districts of the project. The sample size for the quantitative arm of the evaluation is given below:
Respondent type Sample size Notes
In-school adolescents at the start of the project 517 This is based on the actual sample achieved during the baseline and midline. Note that of this sample, all18 students at baseline should be re-contacted, as well as the students at midline.
Parents of adolescents 259 Half of the parents of adolescents surveyed will be contacted at endline.
Mentors 142 This represents 10% of the total number of mentors in the project.
CARE has several tools / modules to be incorporated into the adolescent quantitative surveys as follows:
• Mental health questions using the Washington Group set of questions
• Youth Leadership Index
• Financial Literacy tool
• Child and Youth Resilience Measure
• SRH knowledge and perception questions

Qualitative data
The sample size and respondents for the qualitative arm of the evaluation is given below:
Respondent Total
Students enrolled / graduated from TVET 9 FGDs (1 in each TVET center)
Parents of scholarship recipients receiving funds for income generating activities 6 KIIs (1 in each district)
Head teachers 6 KIIs (1 in each district)
VSLA members 6 FGDs (1 in each district)
YSLA members 6 FGDs (1 in each district)
Ministry of Education officers 6 KIIs (1 in each district)
CARE project staff 1 FGD
Implementing partner project staff 1 FGD
ARCT Ruhuka PSS Professionals 1 FGD

3.3 Secondary data
Secondary data will be collected from review of existing documents and will include: a desk review of the project design documents, project log frame, baseline and midline reports, semi-annual and annual reports, project implementation plans, M&E data, formal policy documents, official public statistics, and other relevant quantitative and qualitative secondary data that will support the evaluation implementation strategies. The project baseline and midline data collection tools will guide this data collection exercise.
Required guidance on indicators, sampling, and methodology from the baseline report will be sought so that data is comparable and consistent from the baseline to end-line measurements.
3.3 Data collection and management
The collection of primary data will involve data collection using interviews to collect both quantitative and qualitative data. Data collection is to be done using mobile data collection kit (Kobo Collect preferred), key informant interviews (KIIs) and focus group discussions (FGDs). The tools will be piloted to determine the responsiveness and flow of the set questions. The consultants are responsible for the setup of electronic data collection forms, following the structure of the tools provided by CARE.

4.0 Ethical approaches
The approach to the endline must consider the safety of participants and especially children at all stages of the evaluation. All consultants staff and data collectors who will participate in the study will receive training on CARE’s policies on Child Protection and on Prevention of Sexual Harassment and Abuse (PSHEA); all consultants staff and data collectors will be required to sign their compliance with both policies before deployment. The bidder will need to demonstrate how they have considered the protection of children through the different data collection stages, including recruitment and training of research staff, data collection and data analysis and report writing.
Bidders are required to set out their approach to ensuring complete compliance with international good practice with regards to research ethics and protocols particularly with regards to safeguarding children, vulnerable groups (including people with disabilities) and those in fragile and conflict affected states. Consideration should be given to:
• administrative, technical and physical safeguards to protect the confidentiality of those participating in research.
• physical safeguards for those conducting research.
• data protection and secure maintenance procedures for personal information.
• parental consent concerning data collection from children or collation of data about children.
• age- and ability-appropriate assent processes based on reasonable assumptions about comprehension for the ages of children and the disabilities they intend to involve in the research; and
• age-appropriate participation of children, including in the development of data collection tools. the assumptions underpinning the successful completion of the proposals submitted and the anticipated challenges that might be faced.
• estimates of the level of risk for each risk identified.
• proposed contingency plans that the bidder will put in place to mitigate against any occurrence of each of the identified risk.
• specific child protection risks and mitigating strategies, including reference to the child protection policy and procedures that will be in place; and
• health and safety issues that may require significant duty of care precautions including COVID adaptations and safety measured during the study

5.0 Data analysis
It is paramount that the evaluation triangulates results and interprets them before formulating (a) findings, (b) conclusions, and (c) programmatic recommendations, each of which should show how they relate to each other. Data analysis must ensure it provides demographics of population sampled, and also give statistical relationships (i.e. tests of significance, predictive analyses, etc.) between variables.
The consultant(s) is expected to share an objective data analysis plan, taking into account data quality, coding and analysis procedures (both qualitative and quantitative).
6. Expected Tasks and roles
The consultant(s) will be responsible for setting up procedures and guidelines to:
• Train enumerators for the data collection exercise.
• Conduct evaluation data collection in the six districts and schools (provided by CARE) covered in the project, as well as specific individuals that CARE provides
• Ensure that datasets are clean, complete, and consistent.
• Analyze data and write endline evaluation report.
• The consultant will be liable to secure any prior visa or approvals that might be required to conduct the evaluation
• The consultant shall be liable for ethical procedures including getting consent from respondents
• All documents and data collected will be treated as confidential and used solely to facilitate analysis.
• The production of the end of project evaluation report will be the liability of the consultant covering all the aspects as outlined in this ToR.
• All training and logistics cost for the enumerators including transport and data collection materials during data collection will be covered by the consultant
• Seek NISR Visa to conduct the endline if required.
Tasks and roles of CARE
• Assume all the responsibilities pertaining to the consultant hiring process
• Ensure that all necessary documentation is availed to the consultant including datasets for longitudinal tracking
• Facilitate initial connections of the consultant with different stakeholders including Local authorities and respondents
• Overall data quality control and technical review of the report and final signing off (including participation in the consultation session with the consultant about the draft research findings)
• Provide any other technical or operational support to the consultant as needed, for example participating in the selection and training of enumerators
The Task Manager for this work will be the Quality Assurance Specialist. The Consultant is also expected to liaise closely at the design stage and subsequently with other key personnel in CARE.
7. Expectations and Deliverables
The consultant (s) is expected to provide the following:
1. Inception report including:
1. Research methodology including the sampling methodology and the analysis framework,
2. Draft data collection tools and protocols
• Detailed work plan outlining all tasks to be completed by each of the members of the consultant team for the duration of the baseline.
1. Ethical approach, quality control plan, limitations, enumerator training plan,
2. Upload data collection tools Kobo collect and share with CARE for approval before commencement of actual fieldwork.
3. Training guidelines for enumerators including training tool kits, and training session plans and train local enumerators on the data collection tools and lead data collection activities while ensuring COVID-19 safety precautions are in-place, as well as data-quality monitoring assessments.
4. The development of clear protocols for data quality control and data management during mixed methods data collection, entry and cleaning processes.
5. Draft end term evaluation report with preliminary findings (with infographics), likely recommendations and conclusions which will be presented to program team for comments.
6. Learning briefs on project 4 learning themes (3-4 page each)
7. A complete set of raw and cleaned datasets, including, where applicable, consolidated datasets (i.e. combining parent and adolescent surveys); complete codebooks for quantitative files generated and analyzed for the report. For the qualitative data, this includes the audio recording files, original transcripts, and translated transcripts of the full verbatim. Note that summary transcriptions or translations will not be acceptable.
8. SPSS / Stata syntax and output files reflecting the analysis conducted.
9. Validation of endline results and documentation of the validation – power point presentation and meeting notes with questions raised.
10. Final endline evaluation report. The consultants will submit a complete final report in English after incorporating comments and feedback. The report will be in Hard copy and Soft copy. CARE Will provide the evaluation report template
11. Updated PMF with all the endline value for the indicators

8. Evaluation Timelines
The following tables delineates the evaluation timelines and milestones during the evaluation process.
Table 4. Evaluation timeline and milestones.
Evaluation Activities Description Indicative Timeline
Review of secondary data and background documents Review all secondary data and background documents 2 days
Inception meeting with selected consultant(s). Meeting to ground break the exercise and set timelines and expectations. 2 hours
Inception report Inception report demonstrating understanding of the TOR and the proposed methodology, including sampling methodology, sample size, analysis framework, draft data collection tools, detailed workplan, ethical approach, quality control, enumerator training plan 1 week
Updating tools and inception report Updating the tools and inception report based on comments from CARE 3 days
Training enumerators Train enumerators in all tools 3 days
Piloting of data collection tools Test the practicability and usage of the tools and methodology of data collection in context 1 day
Updating the data collection tools based on the pilot Review and update the tools with input from the pilot. 1 day
Data collection Carry out the actual data collection exercise including quantitative surveys, KIIs, FGDs in Huye, Nyanza, Ruhango and Karongi, Nyabihu, Ngororero. 3 weeks
First debrief meeting. Another debrief meeting shall be called if necessary. Hold debrief meeting to discuss reflections of the data collection process and key highlights (may be more than one meeting) 1 hour
Data cleaning, triangulation, analysis & interpretation. Consultant to conduct data analysis after collection and present preliminary findings of the analysis. 1 week
Development of first draft report. The consultant(s) to develop the first draft of the end-line report and share with CARE team. 1 week
Review of first draft of report. CARE team to review the draft report and share feedback with the consultant. 5 days (CARE)
Second draft with comments The consultant(s) to incorporate comments and feedback from CARE team and produce a revised copy of the report for CARE team to review. 1 week
Validation workshop Consultant to share findings with key stakeholders and receive questions. Comments, input 1 day
Final soft copy report and hard copies in prescribed format. The consultant(s) to produce the final copy of end-line report, incorporating any final feedback from CARE, and share with the team. Hard copy reports shall be produced as required. 1 day
10. Professional Skills and Qualifications
Qualifications: bidders are required to clearly identify and provide CVs for all those proposed in the Evaluation Team, clearly stating their roles and responsibilities for this endline. The lead consultant should have a master’s degree (PhD preferred) demonstrating thematic and/or research qualifications and a minimum of five years of experience in delivering rigorous program evaluations. The consultants’ proposed evaluation team should include the technical expertise and practical experience required to deliver the scope of work and endline evaluation outputs, in particular, with regards to:
• Study design: the team should include skills and expertise required to design, plan and conduct mixed-methods impact evaluation, using quasi-experimental techniques;
• Skills in quantitative and qualitative data collection and analysis (including merging datasets), drawing findings from multiple sources and handling potential contradictions between data sets; using justified approaches for analysis of qualitative data.
• Relevant subject matter knowledge and experience: knowledge and experience required on conducting research with children, the education sector, disability, gender and gender equality to ensure that the evaluation design and research methods are as relevant and meaningful as possible given the aims and objectives of the project and the context in which it is being delivered;
• Evaluation management: manage a medium-scale and complex research process from end-to-end including endline studies;
• Primary research: gender-sensitive design, management and implementation of primary quantitative and qualitative research – this could include the design of longitudinal household panel surveys, in-depth interviews, focus groups, participatory qualitative exercises with children, etc.;
• Country experience: it is particularly important that the team has the appropriate country knowledge /experience and ability to interpret findings from a contextual perspective, as required to conduct the research;
• Information management: design and manage disaggregated data and information systems capable of handling large datasets for MEL purposes, including cross-referencing different datasets;
• Statistical analysis: a range of statistical modelling and analysis of impact data; highly proficient user of: SPSS or STATA; and qualitative data analysis techniques.
• Data management and data cleaning. Ability to supervise the collection, entry (if required), cleaning and management of large data sets. Digital data collection processes are preferred.
• Safety considerations: Ensuring the whole evaluation process adheres to best practice for research with children including the implementation of child protection policy and procedures to ensure safety of participants. Note that all bidders are expected to be able to show that they have a child protection policy in place to safeguard children that the research team would come into contact with through the research activities.
II. Application procedures
The application file should contain the following documents:
• A technical proposal, with a clear timeframe and a description of the proposed methodology detailing how the deliverables will be achieved, the team suggested, the credentials for similar assignments.
• Detailed Curriculum Vitae of the proposed team to carry out the assignment with clear roles and functions
• A financial offer detailing the various costs associated with the delivery of the above services, in PDF format and must be a separate document from the technical offer.
• Evidence of the consultant’s experience in doing similar assignments: Copies of similar assignments (with evidence for good completion of the previous similar assignments);
• Submit at least 3 references with their contacts and addresses
• Consultant firm profile; VAT registration certificate; RRA tax clearance certificate; RSSB tax clearance certificate (when applicable)

Submission Procedure
Interested consultants or consultancy firms are requested to submit their offers not later than January 31st,2023 at 5:30 pm local time to the following e-mail address: rwa.procurement@care.org, with mention of “Consultancy to conduct an end-line evaluation for SACS project” in the subject line to email by date.
Kigali, January 16th,2023
Procurement Unit
Attachment
attachment_file_b69a36a46521fdb0edbf