ME&A – USAID/Rwanda Collaboration, Learning, and Adapting Activity (CLAA)

Request for Proposals (RFP)For External midline evaluation of Uburezi Iwacu Activity

Request for Proposal name: Collaboration, Learning, and Adapting (CLA) Activity

Request for Proposal Number CLAA-RW-RFP-2024-02

USAID/Rwanda Collaboration, Learning, and Adapting (CLA) Activity is funded by the U.S. Agency for International Development (USAID), Contract No. 72062320D00005/72069622F00001. CLA is a Task Order under USAID Kenya and East Africa Evaluation, Assessments, and Analyses (EAA) IDIQ. USAID Terms and Conditions of Kenya and East Africa EAA IDIQ and the CLA Task Order are applicable to the anticipated subcontract.

The objective of the CLA Activity is to improve USAID/Rwanda’s performance monitoring, evaluation, collaboration, learning, and adapting processes to ensure the Mission effectively utilizes data in all decision-making processes. The services provided under this Activity will enable USAID/Rwanda to better deliver the sustainable development outcomes articulated in its new Country Development Cooperation Strategy (CDCS) 2020–2025 and enhance the United States (U.S.) Rwanda partnership. CLA is implemented by ME&A, Inc. (ME&A), a small businesswoman-owned international development firm based in the Washington, D.C., area, providing innovative solutions for economic development and social progress in more than 80 countries.

CLA is soliciting Offers to conduct an External Midline Evaluation of the Uburezi Iwacu Activity from local Rwandan Monitoring, Evaluation, and Learning (MEL) firms that demonstrate that they are technically qualified, possess the relevant experience in conducting evaluations, especially for Education activities in Rwanda, have a good record of performance and meet all the request for proposal requirements.

The MEL firm described here will collect data to answer the evaluation questions, make conclusions based on the findings, and identify opportunities and challenges.

Uburezi Iwacu (“Homes and Communities”) activity is a five-year cooperative agreement implemented by World Vision, Humanity & Inclusion and Imbuto Foundation. to provide all Rwandan children with literacy-supportive, stimulating and safe home and community environments. The activity contributes to improved literacy outcomes for all children by the end of Primary 3 by improving home literacy environments, increasing community engagement in promoting children’s literacy, and improving literacy learning opportunities for children with disabilities. Uburezi Iwacu focuses on mother tongue literacy (Kinyarwanda) as an essential building block to developing reading and writing skills in any language.

The activity also addresses social and emotional development, given its importance to forming the foundation of a child’s future learning and holistic positive development. Uburezi Iwacu collaborates with the Government of Rwanda to develop the human and institutional capacity necessary to sustain improved learning outcomes. The activity fosters self-reliance, ownership, and sustainability by focusing on community-based solutions.

Purpose

The purpose of this external midline evaluation of Uburezi Iwacu is to assess the relevance, effectiveness, and efficiency of the activity’s interventions. (1) It will examine the extent to which the activity is relevant to the beneficiaries in the targeted homes and communities. (2) It will determine whether the activity is meeting its intended results and targets.  The evaluation exercise will not only assess project performance, but also seek to  identify factors contributing to (or inhibiting) successful implementation and achievement of results. Offerors are invited to submit their proposals in accordance with the Instructions to Offerors provided below.

Issuance of this RFP does not in any way obligate ME&A to award a subcontract or a purchase order, nor does it commit ME&A to pay for costs incurred in the preparation and submission of the Offer.

Instructions to Offerors

Offer Deadline, Submission Instructions, and Offering Procedures:  Offers must be submitted no later than 5:00 pm Central Africa Time, July 8th 2024. Each Offeror must submit an electronic copy to the ME&A email address clarwanda@engl.com, and place in the subject line of the email the name of the Offering company and the words “External midline evaluation of Uburezi Iwacu Activity”. Proposals received after the closing time will be rejected.

Offers shall include the Offer Price and the information indicated in Clauses 3 and 5 of this section. Each Offeror may submit only one Offer. Offerors submitting more than one Offer will be disqualified, and their Offers will be rejected.

1. Signature of Offers:  The Offer shall be signed by the Offer’s authorized representative. The first page of the Offer shall indicate the Request for Proposal number CLAA-RW-RFP-2024-02. 
2. Eligibility: Local MEL Rwandan companies that demonstrate that they are technically qualified, possess the relevant experience in conducting evaluations, especially for Education activities, have a good record of performance, meet all the request for proposal requirements, and are not included in the USAID list of companies that are debarred, suspended, or proposed for debarment are eligible for submitting Bids in response to this RFP.
3. Offer Contents: The offer will include the following information:
4. Offeror’s Technical Qualifications and Experience: Interested Offerors shall submit the following:

• Technical qualifications and past experiences of the Offeror relevant to this assignment (not to exceed 3 pages).
• Offeror’s contact information (telephone number, e-mail address, and work address).
• Copies of all licenses and permits required under the Rwanda legislation for the performance of the work.
• Evidence and description of either/or at least two projects about conducting evaluations especially for Education activities during the last five years.
• Contacts (telephone number, E-mail address, and address) of at least three previous clients to provide references related to the quality of services and goods delivered by the Offeror.
• Firm Fixed Offer Price: The Offeror Firm Fixed Price must be expressed in Rwandan Francs (RWF). Offerors must ensure that all elements of the Offer Price are accurate and reasonable. The Offer Price should include VAT.

All firms must submit only one Offer Price. Offerors to submit more than one Offer Price will be rejected.

The Offer Price shall be the cost proposal to be submitted in a separate email from the technical proposal.

1. Key Personnel and Qualifications: For this assignment the offeror should include the CVs of key personnel with the following qualifications and experience.

• Experience in evaluation design, methods, management, and implementation;
• Technical subject matter expertise in education interventions, particularly early grade literacy;
• Qualitative research and analysis skills;
• Experience implementing EGRA assessments; experience with IDELA also preferred;
• Rwanda experience; and
• Kinyarwanda skills.

The contractor must provide information about evaluation team members, including their curricula vitae, and explain how they meet the requirements in the evaluation SOW. Per ADS 201.3.6.8, all team members must provide to USAID a signed statement attesting to a lack of conflict of interest or describing an existing conflict of interest relative to the project or activity being evaluated (i.e., a conflict of interest form). 

Proposed key personnel are expected to be the people who execute the work of this contract. Any substitutes to the proposed key personnel must be vetted and approved by the AOR/COR before they begin work.

Selection Criteria and Corresponding Points: Proposals will be evaluated using a 100-point system allocated as follows:

a) Technical proposal (30 points)

b) Qualifications of key personnel performing the assignment (20 Points)

c) Experience of the firm/team (20 points)

d) Financial Proposal (30 points)

Additional information required:

A description of the understanding of Education programming and evaluations is a requirement.

• Questions/Clarifications: All additional questions and/or requests for clarifications regarding this procurement must be made via email and submitted to ME&A at clarwanda@engl.com before June 25, 2024. No phone calls, oral questions, or requests for meetings will be accepted in response to this request for proposals. Responses to questions will be provided by June 27 2024.
• Specifications: All the minimum technical specifications of the requested products and services are provided in the Midline Evaluation of USAID Uburezi Iwacu Activity SOW Annex A of this request for proposals.
• Time for Performance: The period of performance shall be six months starting from July 22, 2024, to January 22 2025.
• Contract Type: ME&A will issue a Firm Fixed Price subcontract to the successful Offeror. The subcontract format will follow USAID requirements and will include USAID mandatory contract clauses.
• Payment Schedule: The Offer price will be paid as follows: 

Payment(s) for invoices will be made to the Subcontractor within five (5) business days of ME&A receipt of payment from the Client for Subcontractor’s services accepted by the Client.

CLAA reserves the right to refuse acceptance of the work if the quality or specifications of implemented work does not correspond to the requirements of the Subcontract.

• Offer Validity Period: Offers must remain valid for a period of 30 (thirty) calendar days from the date of Offer submission.
• Evaluation and Award: It is anticipated that one award will be made to the Offer who demonstrates that it is technically qualified, possesses the relevant experience, has a good record of performance, meets all requests for proposal requirements, and provides the best value. However, ME&A reserves the right to request clarifications prior to awarding a subcontract and to reject any or all Offers.
• Unique Entity ID: There is a mandatory requirement for your organization to provide a Unique Entity ID to ME&A. (https://sam.gov/content/home). Without a Unique Entity ID number, ME&A cannot deem an Offer "responsible" to conduct business with, and therefore, ME&A will not enter into a subcontract/purchase order or monetary agreement with any organization that does not meet this requirement. The determination of a Successful Offer resulting from this request for proposal is contingent upon the winner providing a Unique Entity ID number to ME&A. Offers who fail to provide a Unique Entity ID number will not receive an award and ME&A will select an alternate Offer.

All U.S. and foreign organizations that receive first-tier subcontracts/ purchase orders with a value of $25,000 and above are required to obtain a Unique Entity ID number prior to the signing of the agreement. Organizations are exempt from this requirement if the gross income received from all sources in the previous tax year was under $300,000. ME&A requires that Offers sign the self-certification statement if the Offer claims an exemption for this reason.

Information on how to register an international entity in SAM.gov and obtain a Unique Entity ID number is provided in the following link:

https://sam.gov/content/entity-registration

Annex A:

STATEMENT OF WORK

External Midline Performance Evaluation
of Uburezi Iwacu Activity

I. Purpose of the Evaluation

Uburezi Iwacu (UI) is a five-year USAID funded literacy project. The purpose of the activity is to ensure that all Rwandan children have literacy-supportive, stimulating and safe home and community environments by improving home literacy environments, increasing community engagement in promoting children’s literacy, and improving literacy learning opportunities for children with disabilities.

The purpose of this external midline evaluation of Uburezi Iwacu is to assess the relevance, effectiveness, and efficiency of the activity’s interventions. (1) It will examine the extent to which the activity is relevant to the beneficiaries in the targeted homes and communities. (2) It will determine whether the activity is meeting its intended results and targets.  The evaluation exercise will not only assess project performance, but also seek to  identify factors contributing to (or inhibiting) successful implementation and achievement of results.

The primary audience for this report is USAID in both Rwanda and Washington, the Rwanda Ministry of Education, and World Vision, our Implementing Partner. The results will  inform any necessary adaptations and improvements to the activity in its final two years of implementation.

II. Summary Information 

III. Background

A.     Description of the Problem and Context

Rwanda is committed to improving human capital, with a focus on building strong foundational skills. This is clearly stated in the Education Sector Strategic Plan III (ESSP) (2018/19-2023/24), Priority 1: All learners achieve basic levels of literacy and numeracy in early years and beyond.

Over the last decade, in addition to school based literacy interventions, the Government of Rwanda has increasingly recognised support for learning at home and in the community as a necessary condition to improve student learning outcomes. USAID Rwanda has been supporting this initiative, as a partner of choice, by investing in 3 major activities, L3, Mureke Dusome and current Uburezi Iwacu (concerned by this Midline Evaluation).

Through this partnership, the Ministry of Education has drafted the National Literacy Policy to purposefully integrate literacy-promotion activities and opportunities into everyday life, in a way that builds cross-sectoral ownership and elevates literacy development to a national priority. The Soma Rwanda platform has been created to spur and coordinate literacy promotion activities throughout the country. The book industry has been supported to produce more high quality children’s books and low-cost reading materials. Rwanda Children Book Organization was established to produce gender sensitive, age appropriate children books and book banks were established in each catchment village with a public and government-aided primary school (over 2,500), and are being used by community volunteers to facilitate reading clubs and other community literacy activities. Head Teachers and School General Assembly Committees have been trained to increase community support for reading. Rwanda now also has 67 community libraries with books and trained librarians.  These interventions have been effective to strengthen the partnerships between schools and the broader community, but continued investment was necessary to sustain and further these gains.

The Mureke Dusome knowledge, attitudes, and practices (KAP) survey^[1] revealed that while the proportion of children participating in community reading activities increased from 17% in 2016 to 31% in 2018, the majority of children still have inadequate opportunity to practice reading outside of school. While the number of parents/caregivers or other household members who encourage their children to read and write increased from 59% in 2016 to 78% in 2018, 62% of parents and 74% of children still believed that only literate parents could help children read. And while more children have access to storybooks outside of school (10% in 2018 compared with 6% in 2016), the vast majority still do not.

To bridge the above mentioned bottlenecks, USAID Rwanda designed and awarded  Uburezi Iwacu Activity to World Vision in 2021. It's been 3 years since it was implemented. The current evaluation is meant to provide performance measurements against its intended results and objectives. Being a data driven organization, USAID will use the results to adjust where necessary the implementation of the remaining 2 years.    

B. Description of the Intervention to be Evaluated

Uburezi Iwacu is a five-year cooperative agreement. The purpose of Uburezi Iwacu (UI) is to ensure that all Rwandan children have literacy-supportive, stimulating, and safe home and community environments. While the primary activity outcomes focus on Kinyarwanda literacy, Uburezi IwacuI includes addressing the language and social and emotional skills development of children, recognizing that these skills are part of children’s holistic positive development that form the foundation for future learning. Uburezi Iwacu also fosters self-reliance, ownership, and sustainability by focusing on community-based solutions, as well as relevant institutions and systems. More specifically, Uburezi Iwacu I aims to achieve the following three Intermediate Results:

IR1: Improved home literacy environments

IR2: Increased community engagement in promoting children’s literacy support

IR3: Improved literacy learning opportunities for children with disabilities 

Project or Activity Monitoring, Evaluation, and Learning (MEL) Plan

During this exercise, activity Annual Work Plans and Annual MEL plan will be available for the evaluators. Additionally, quarterly and annual reports and technical products (such as research reports and educational materials) will be available. Baseline report, GIS mapping report, Early Grade Reading Assessment (EGRA) reports, previous  Mureke Dusome and L3 KAP reports,  will also be made available. Evaluators are encouraged to consult the Development Experience Clearinghouse (DEC) in particular for all activity related reports.

IV. Evaluation Questions

SECTION 1:  Outcomes.

1.  Is the activity achieving  its objectives and reaching the intended outcomes?

What factors contributed to the achievements of the observed change in the targeted homes and communities ?  What are the major barriers /factors that hinder the achievement of the intended results? Is there any difference between high touch and low touch districts of operations.

SECTION 2:  Effectiveness/Fidelity Of Implementation

• Was the activity implemented as planned? Did the Activity reach the intended geographic coverage (targeted villages),  and the intended target population (children and parents)? Were all activity interventions implemented as planned?
• Did the activity provide targeted children with literacy-supportive, stimulating, and safe home and community environments?
• Did the home literacy environments improve?  Are Parent Peer Support Networks operational and of good quality (i.e did meetings regularly take place across targeted locations, were parent learning sessions of good quality, etc). Did positive parenting practices to support children’s language, literacy, and social emotional development improve? What drivers contributed to that change? What were the challenges if these were not improving? Were children able to access an increased quantity of accessible and high-quality educational content and materials at home?
• Did  community engagement in promoting children’s language, literacy, safety, and social-emotional development increase? Were community literacy structures and activities of good quality? Did the intended number of reading clubs regularly take place in the intended number of locations? Was the home-community-school partnerships to promote children’s language, literacy and social emotional development strengthened? Did children access these community literacy structures?
• Have  literacy learning opportunities for children with disabilities increased. Has participation of children with disabilities in community’s children’s language, literacy, safety, and social emotional development interventions increased? Did the caregiver, and community members' knowledge and skills to support safety, and language, literacy, safety, and social emotional development of children with disabilities increase? Did the intended identification and referrals process take place in the intended number of locations? Was home-community-school partnership to promote enrollment and retention of children with disabilities in school strengthened?

NB: Disability is integrated in the Activity, the evaluators are required to collect disabilities data across all questions.         

SECTION 3: Sustainability.

How are the strategies/activities for sustaining activity outcomes beyond the life of the activity being implemented? Are the activity outcomes likely to be sustained?

V. Evaluation Design and Methodology

The evaluation team, in collaboration with USAID, will finalize the evaluation methods before fieldwork begins. The evaluation team must have experience in evaluating Knowledge Attitudes and Practices (KAP) related activities implemented in communities, conducting EGRA assessment particularly in Rwanda and measuring Early Childhood Development and Education learning outcomes. 

In order to evaluate reading and social emotional skills, and to avoid duplication and cost, the evaluation team will liaise with Tunoze Gusoma and analyze EGRA, IDELA and SEL  midline evaluation  data- as secondary data- pertaining to children participating homes and community reading interventions and try to get an indication on how these are performing.

USAID expects that, at a minimum, the evaluation team will:

• Upon award, familiarize themselves with documentation about the project and USAID’s current assistance in the Education area in the region. USAID will ensure that this documentation is available to the team prior to their arrival in the region;
• Conduct a comprehensive review of Activity and Activity-related documents and secondary data.
• Review existing performance data and baseline studies;
• Meet and interview USAID activity beneficiaries, partners, and host government counterparts at appropriate levels (when applicable);
• Interview USAID staff and a representative number of experts working in the sector; and
• Spend at least two months in the region carrying out this SOW.

The desk review includes at a minimum:

• USAID activity SOW
• Annual and Quarterly Reports
• Annual Work Plan,
• MEL Plans, sector assessments
• Trip reports
• Baseline studies
• Performance reports
• Project Appraisal Document
• Miscellaneous thematic reports from other sources.

The contractor will submit the preliminary evaluation design for review by USAID. The evaluation design matrix should include the data sources, data collection methods, and analysis plan for each evaluation question. Requests of the offeror must include an explicit description of major limitations in data collection and analysis.

VI. Deliverables and Reporting Requirements

1. Evaluation Work Plan:

Within two weeks of the award of the contract, the lead evaluator shall complete and present a draft work plan for the evaluation to the AOR/COR.

The work plan will include:

1. Draft schedule and logistical arrangements;
2. Members of the evaluation team, delineated by roles and responsibilities;
3. Evaluation milestones;
4. Anticipated schedule of evaluation team data collection efforts;
5. Locations and dates for piloting data collection efforts, if applicable;
6. Proposed evaluation methodology including sampling methodology and selection criteria for comparison groups, if applicable; and
7. Evaluation Report outline (if different from the attached template), including an overall report and province or district-level infographics/one-pager summaries.

The contractor will update the evaluation work plan (the lists of interviewees, survey participants, the schedule) and submit the updated version to the AOR/COR on a biweekly basis.

2. Evaluation Design:

Within two weeks of approval of the work plan, the evaluation team must submit an evaluation design to the AOR/COR. The design will become an annex to the evaluation report.

The evaluation design will include:

• Detailed evaluation design matrix that links the Evaluation Questions from the SOW (in their finalized form) to data sources, methods, and the data collection and analysis plan, including who will be involved in data collection and analysis;
• Draft questionnaires and other data collection instruments or their main features, in both English and Kinyarwanda;
• List of potential interviewees and sites to be visited and proposed selection criteria and/or sampling plan (must include sampling methodology and methods, including a justification of sample size and any applicable calculations);
• Limitations to the evaluation design; and
• Dissemination plan (designed in collaboration with USAID).

Unless exempted from doing so by the AOR/COR, the evaluation design will be shared with partner country stakeholders as well as with the implementing partners for comment before being finalized.

The data analysis plan should clearly describe the evaluation team’s approach for analyzing quantitative and qualitative data, including proposed sample sizes, specific data analysis tools, and any software proposed to be used, with an explanation of how/why these selections will be useful in answering the evaluation questions for this task. Qualitative data should be coded as part of the analysis approach, and the coding used should be included in the appendix of the final report. Gender, geographic, and role (beneficiary, implementer, government official, NGO, etc.) disaggregation must be included in the data analysis where applicable.

All dissemination plans should be developed with USAID and include information on audiences, activities, and deliverables, including any data visualizations, multimedia products, or events to help communicate evaluation findings and recommendations. See the Evaluation Toolkit for guidance on Developing an Evaluation Dissemination Plan.

If applicable based on the Disclosure of Conflict of Interests Forms submitted with the awardee’s proposal, the evaluation design will include a conflict of interest mitigation plan.

USAID offices and relevant stakeholders are asked to take up to 10 working days to review and consolidate comments through the AOR/COR. Once the evaluation team receives the consolidated comments on the initial evaluation design and work plan, they are expected to return with a revised evaluation design and work plan within five working days.

3. Draft Evaluation Report:

The draft evaluation report should be consistent with the guidance provided in Section VIII, Final Report Format. The report will address each of the questions identified in the SOW and any other issues the team considers to have a bearing on the objectives of the evaluation. Any such issues can be included in the report only after consultation with USAID. The submission date for the draft evaluation report will be determined in the evaluation work plan, and will include a presentation of key findings to gather feedback (a validation workshop; the participant list will be agreed upon in advance with the AOR).

Once the initial draft evaluation report is submitted, the Education Office will have 10 working days in which to review and comment on the initial draft, after which point the AOR/COR will submit the consolidated comments to the evaluation team. The evaluation team will then be asked to submit a revised final draft report within 10 working days, and again the Education Office will review and send comments on this final draft report within 10 of its submission.

4. Final Evaluation Report:

The evaluation team will be asked to take no more than 10 working days (or as agreed upon in the work plan) to respond to and incorporate final draft evaluation report comments from the Education Office. The evaluation team lead will then submit the final report to the AOR/COR, and present it in a workshop; the workshop participant list will be agreed upon in advance with the AOR.

5. Submission of Dataset(s) to the Development Data Library:

Per USAID’s Open Data policy (see ADS 579, USAID Development Data) the contractor must also submit to the AOR/COR and the Development Data Library (DDL), at www.usaid.gov/data, in a machine-readable, non-proprietary format, a copy of any dataset created or obtained in performance of this award, if applicable. The dataset should be organized and documented for use by those not fully familiar with the intervention or evaluation.

Please review ADS 579.3.3.2 Types of Data To Be Submitted to the DDL to determine applicability.

6. Submission of Final Evaluation Report to the Development Experience Clearinghouse:

Per USAID policy (ADS 201.3.6.9) the contractor must submit the evaluation final report and its summary or summaries to the Development Experience Clearinghouse (DEC) within three months of final approval by USAID.

VII. Evaluation Team Composition

The contractor must provide information about evaluation team members, including their curricula vitae, and explain how they meet the requirements in the evaluation SOW. Submissions of writing samples or links to past evaluation reports and related deliverables composed by proposed team members are highly desirable. Per ADS 201.3.6.8, all team members must provide to USAID a signed statement attesting to a lack of conflict of interest or describing an existing conflict of interest relative to the project or activity being evaluated (i.e., a conflict of interest form).

Required qualifications and skills:

• Experience in evaluation design, methods, management, and implementation;
• Technical subject matter expertise in education interventions, particularly early grade literacy;
• Qualitative research and analysis skills;
• Experience implementing EGRA assessments; experience with IDELA also preferred;
• Rwanda experience; and
• Kinyarwanda skills.

Proposed key personnel are expected to be the people who execute the work of this contract. Any substitutes to the proposed key personnel must be vetted and approved by the AOR/COR before they begin work. USAID may request an interview with any of the proposed evaluation team members via conference call, video conference, or other means.

The evaluation AOR/COR may observe the data collection efforts. USAID may also delegate one or more staff members to work full-time with the evaluation team or to participate in selected evaluation activities. The evaluation AOR/COR will inform the contractor in writing about any full-time or part-time USAID delegates no later than seven working days after the submission of a draft evaluation work plan. USAID will pre-define any staff’s level of involvement by indicating the purpose of their inclusion, their role on the team and in which components of the evaluation they will participate, their expertise in the topic or sector, and their expertise in evaluation design or implementation. USAID maintains primary responsibility for management of its own staff. USAID will outline collaboration, delivery, and performance expectations for its staff as well as reporting lines and how staff management roles and responsibilities will be coordinated between USAID, the contractor, and the evaluation team lead. This plan will be finalized in consultation with the contractor and the evaluation team lead, with final approval by the AOR/COR, to ensure it is feasible and appropriate to the evaluation objectives and Rwanda’s needs and that it addresses mitigation of risk of impeding evaluation implementation or biasing findings. All costs associated with the participation of full-time or part-time USAID delegates in the evaluation will be the responsibility of USAID.

VIII. Final Report Format

It is recommended that the final report be approximately 40-60 pages long, plus annexes. The main body of the report should include:

1. Abstract
2. Executive Summary
3. Evaluation Purpose
4. Background on the Context and the Interventions being Evaluated
5. Evaluation Questions
6. Methodology (to include in the body of the report or as annexes: evaluation design, sampling methods, data collection methods, evaluation tools, recruitment and training of enumerators, data monitoring and quality assurance, and data analysis methodology including explanation of any sampling weights applied)
7. Limitations to the Evaluation
8. Findings for each evaluation question
9. Conclusions, and (If Applicable) Recommendations
10. One-pager or infographic summaries of key findings and recommendations at a province or district-level
11. Annexes

See the Evaluation Toolkit for the How-To Note on Preparing Evaluation Reports and ADS 201mah, USAID Evaluation Report Requirements. An optional Evaluation Report Template is also available in the Evaluation Toolkit.

The evaluation abstract of no more than 250 words should describe what was evaluated, evaluation questions, methods, and key findings or conclusions. The executive summary should be 2–5 pages and summarize the purpose, background of the project being evaluated, main evaluation questions, methods, findings, and conclusions (plus recommendations and lessons learned, if applicable). The evaluation methodology shall be explained in the report in detail. Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methods (e.g., in sampling; data availability; measurement; analysis; any potential bias such as sampling/selection, measurement, interviewer, response, etc.) and their implications for conclusions drawn from the evaluation findings.

Annexes to the report must include:

• Evaluation SOW and Evaluation Questions (updated, not the original, if there were any modifications);
• All data collection and analysis tools used in conducting the evaluation, such as questionnaires, checklists, and discussion guides;
• All sources of information or data, identified and listed;
• More detailed information regarding data collected, following norms established in prior reports to allow for data comparisons over time, including statistical output tables showing the effect size with standard errors reported (see ADS 201mah for related requirements);
• Statements of difference regarding significant unresolved differences of opinion by funders, implementers, and/or members of the evaluation team, if applicable;
• Signed disclosure of conflict of interest forms for all evaluation team members, either attesting to a lack of or describing existing conflicts of interest; and
• Summary information about evaluation team members, including qualifications, experience, and role on the team.
• Evaluation data or link to data.

IX. Criteria to Ensure the Quality of the Evaluation

Per ADS 201maa, Criteria to Ensure the Quality of the Evaluation Report, draft and final evaluation reports will be evaluated against the following criteria to ensure quality.

• Evaluations must have the required sections outlined in ADS 201mah, USAID Evaluation Report Requirements.
• Evaluation reports should represent a thoughtful, well-researched, and well-organized effort to objectively evaluate the subject of the evaluation (e.g., strategy, project, activity).
• Evaluation reports should use clear language per the USAID Style Guide.
• Evaluation reports should adequately address all evaluation questions included in the statement of work, or the evaluation questions subsequently revised and documented in consultation and agreement with USAID.
• According to the ADS 201.3.6.2 principle that evaluations should be based on the best methods of appropriate rigor, evaluations must produce well documented findings that are verifiable, reproducible, and on which stakeholders can confidently rely, while providing clear explanations of limitations. Evaluation methodology should be explained in detail and sources of information properly identified. Sufficient information on methodology and data collection should be included to allow for stakeholders to make informed judgements about the quality and accuracy the findings, and to allow other evaluators to replicate the protocol.
• In support of ADS 201.3.6.2 principle that evaluations should be independent, objective, and unbiased in measuring and reporting, limitations to the evaluation should be adequately disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). Evaluators should strive for objectivity in the planning and conduct of evaluations and in the interpretation and dissemination of findings, avoiding conflicts of interest, bias, and other partiality.
• Evaluation reports should adequately capture the situations and experiences of both males and females. If evaluation findings or data include people-level indicators they must be disaggregated by sex. For more information, see How-To Note: Engendering Evaluation at USAID.
• Findings, conclusions, and recommendations (if any) should be specific, concise, and supported by strong quantitative and/or qualitative evidence.
• Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or simply the compilation of people’s opinions.
• Conclusions should clearly be based on the evaluation findings.

If recommendations are included, they should be supported by a specific set of referenced findings, and should be prioritized, action-oriented, practical, and specific. To support the ADS 201.3.6.2 evaluation principle that evaluations should be oriented to reinforcing local ownership, when possible, evaluators should include relevant local stakeholders in joint development of recommendations. See ADS 201mah, USAID Evaluation Report Requirements and the Evaluation Report Checklist and Review Template from the Evaluation Toolkit for additional guidance.

Annex B: USAID Contract Clauses

(See the document attached

Save the Children Rwanda. Mureke Dusome Impact Evaluation Endline Report, 2018.