Consultant (Final External Project Evaluation-Learn Consortium)

Project : LEARN Consortium Phase II – BHA Funded Reference: AMM/IM/002/AM/21 This Expression of Interest solicitation is open to all eligible consultants, in accordance with SOLIDARITES INTERNATIONAL recruitment procedures.

Description of Project

The LEARN Consortium is a partnership between four (4) International Non-Governmental Organizations (INGOs). Following the successful completion of the initial phase of LEARN in September 2020, LEARN is implementing the Phase II which is a one-year award (10/2020 – 9/2021) funded through the Bureau of Humanitarian Assistance (BHA), wherein it seeks to leverage the expertise from the four implementing partners and their two sub-awardees to deliver Health, Shelter and WASH services to Internally Displaced Persons and other vulnerable people in North East Syria (NES). The main activities being implemented are related to the rehabilitation and running of Primary Health Centers (PHCs), health posts (HP), hospitals and mobile medical units (MMUs), provision of WASH services, private shelter repairs and distribution of essential kits (new arrivals, hygiene, shelter, winterization; in-kind or through cash-for-kits), provision of multi-purpose cash assistance (MPCA) and protection services. The program incorporates a Rapid Response Mechanism and supports Humanitarian Coordination groups for the NES Response while also mainstreaming protection across the different sectors. LEARN’s goal is achieved through an integrated and harmonized, multi-sectoral approach which addresses both the emergency and early recovery needs of 550,000 of the most vulnerable conflict-affected men, women and children across NES. The four implementing partners and their sub-awardees are based in NES, Erbil and Amman, and implement throughout NES. The Consortium Management Unit (CMU) is the oversight structure for the program that aims to facilitate smooth coordination, oversee program delivery, lead donor communications, ensure compliance and support risk management. LEARN’s Response Strategy The LEARN response strategy (LEARN’s detailed response strategy will be provided on requests made prior to the deadline of the submission of evaluation proposal) is grounded in in-depth needs analysis, evidence-based approaches and learning gained from extensive operational experience in NES. The consortium has recognized and demonstrated technical capacity in health, shelter, WASH and protection. In Phase II, two new BHA sectors, multi-purpose cash assistance and protection, are added based on the lessons learnt gathered during the Phase I of implementation and pre-existing capacities of partners to work in the new sectors. LEARN has widespread access to affected populations across NES and hosts coordinators for three working groups (Health, WASH and S/NFI) of the NES Forum. LEARN operates under a coordinated structure building on established and strong relationships with local authorities and civil councils across NES. It is designed to be agile and responsive to the rapidly changing context, using its flexibility to constantly adapt programs and reallocate resources where required. Rooted in the communities, the consortium network ensures a proactive operational approach, based on early warning and sound contextual analysis. The consortium uses a joint integrated data management and M&E system, drawing on extensive experience and strong FCRM capacity. The consortium focuses on ensuring monitoring data, learning and beneficiary feedback is integrated into design" style="border-bottom: 1px dotted #007bff !important;">program design throughout the program cycle. To ensure accountability and effective management, LEARN management and governance structure has developed robust processes and controls, which guides operations, decision-making, risk management, communication, grant management and compliance. The sector-wise project target and location of intervention is given below: WASH SECTOR

• Targeted Beneficiaries : 97 000
• Location of intervention/Governorates : Ar-Raqqa, Al-Hasakeh, Aleppo
• Activity details : water trucking, water station rehabilitation, WASH NFI, Sanitation, Environmental Health, Cash for Work, and Hygiene Promotion

SHELTER SECTOR

• Targeted Beneficiaries: 60 000
• Location of intervention/Governorates: Ar-Raqqa, Al-Hasakeh, Aleppo
• Activity details: Private Shelter Repair, Institutional Repair and Distribution of SNFI

HEALTH SECTOR

• Targeted Beneficiaries: 550 000
• Location of intervention/Governorates: Ar-Raqqa, Al-Hasakeh, Aleppo, Deir-Ez-Zor
• Activity details: Support to Health services2 in hospital, PHC and Mobile Medical Units, Community Health operations through human resource, pharma and equipment support

MPCA SECTOR

• Targeted Beneficiaries: 19 800
• Location of intervention/Governorates: Ar-Raqqa, Al-Hasakeh, Aleppo
• Activity details: One-off and multi-rounds of cash transfer

PROTECTION SECTOR

• Targeted Beneficiaries: 3068
• Location of intervention/Governorates: Ar-Raqqa, Al-Hasakeh, Aleppo, Deir-Ez-Zor
• Activity details: Psycho-social support, GBV mitigation and response activities

HUMANITARIAN COORDINATION AND INFORMATION MANAGEMENT SECTOR

• Targeted Beneficiaries: Wash working group, SNFI working group and health working group
• Location of intervention/Governorates: NES
• Activity details: Coordination among various actors in the NES Forum, support to Information Management services

BACKGROUND AND CONTEXT

LEARN’s operational context in NES is dynamic and evolving since its inception in September 2018. In early 2019, Northeast Syria (NES) witnessed a steady decline in conflict activity since the group known as Islamic State of Iraq and the Levant (ISIL) was expelled from the region. This decline in conflict activity led residents who had previously been displaced to return to their communities of origin. At the same time, the governorates of Ar-Raqqa, eastern Aleppo, Deir-ez-Zor and Al-Hasakeh continued to host a large number of IDP populations. The displaced population mainly live in camps, informal settlements, collective centres or private shelters. Residents of Northeast Syria faced a new humanitarian crisis after the escalation of conflict in October 2019; this resulted in significant displacement from the region, both internally and towards the Kurdish Region of Iraq (KR-I). More than 190,000 women, children and men had been displaced, mostly from Al-Hasakeh and Ar-Raqqa. Critical infrastructure, including health and water treatment facilities, was damaged or closed. Since mid-December 2019 conflict has continued at a reduced level in comparison to October/early November. The onset of the COVID-19 pandemic resulted in public health measures being taken in NES to control the spread of infection causing disruption in program implementation towards the end of the Phase I award. To date, in the Phase II implementation period between October 2020 and May 2021, whilst the security" style="border-bottom: 1px dotted #007bff !important;">security context has remained relatively calm there have been two defining contextual factors: COVID-19 and the worsening economic situation. As of the 25 May 2021, 17599 cases and 717 deaths due to COVID-19 have been registered according to the Self-Administration (SA). However, concerns remain about the validity of this data with under-reporting still a major issue. Recent weeks have seen an increase in cases as the ‘Second Wave’ takes hold and intermittent lockdowns are being enforced. Such restrictions are disrupting not only the access to basic services but also the economy which had already been negatively impacted by external factors including sanctions, the Lebanese economic and banking crisis, SYP devaluation and the protracted nature of the humanitarian and political crisis in Syria. The relative calm of the period combined to the deterioration of socio-economic situation have contributed to the small to medium scale population movements across NES, especially as people search for jobs in the area. Since end of May 2021, the water levels in the main water sources are also depleting which may pose a health and hygiene risk on the population. Partners’ ability to deliver programming may be strongly impacted by the evolving context, including possible supply chain and money transfer system ruptures, plus curfew and other movement limitations which are likely to become increasingly restrictive. LEARN activities include preparedness and response to COVID-19, and LEARN’s operational model and activities continue to adapt as the global pandemic continues. It is anticipated that in the following 3-6 months, areas of control in NES remain somewhat similar. The program monitoring is based on a set of BHA sector output and outcome level standard indicators (WASH, Shelter, Health/Nutrition, MPCA, Protection and Humanitarian Coordination) and incorporates measurement of beneficiaries’ safety and dignity when accessing assistance. Monitoring data are routinely collected by implementing partners and analyzed in semi-annual reports. Feedback and complaint response mechanisms are established and managed by partners in all program locations. Third-party monitoring activities, commissioned by the donor were conducted on multi-purpose cash transfers in Al-Hasakeh and Ar-Raqqa governorates. Contingency measures at partners’ level include resorting to their own TPM should access be loss.

EVALUATION PURPOSE

The purpose of the evaluation is to assess the overall success of project outcomes of LEARN Phase II (implementation period 10/2020 to 9/2021), measuring the effectiveness of partner-specific as well as integrated interventions. The evaluation findings will be disseminated for use by the LEARN partner organizations, the donor and other relevant stakeholders such as working groups, local authorities and communities. The objectives of the External Evaluation are to: 1. to assess program’s effects on direct beneficiaries and effectiveness of the integrated approach 2. to identify key aspects of consortium effectiveness

1. to draw broader programmatic lessons that are informed by internal learning exercises, document good practices and provide actionable recommendations for follow-on LEARN intervention Scope: The evaluation will assess all components (Health, WASH, Shelter, Multi-purpose Cash transfer, Protection and Humanitarian Coordination) of the project in all areas of intervention (Deir-ez-Zor, Aleppo, Ar-Raqqa and Al-Hasakeh governorates). Specific geographic coverage and locations will however be identified based on security and access considerations. Differentiation between results in different implementation areas for similar activities and modalities should be captured. The evaluation study will look at the period of program performance, i.e. October 2020 – September 2021. Special attention should be paid to protection mainstreaming and gender sensitive approaches (specifically inclusion and access of women and people with special needs to all project activities). It will explore the extent to which newly added sectors (multi-purpose cash assistance and protection) are integrated within the LEARN program

EVALUATION TYPE

Primary approach is to choose methods that suit conditions on the ground and provide flexibility for evaluators to change methods based on changing context and access. In the absence of a rigorously defined counterfactual that would be needed for evaluation on impact, the final evaluation will be conducted as performance evaluation using qualitative methods: document review, key informant interviews, focus group discussions, baseline-endline comparison and direct observations. Quantitative data collected in the course of performance monitoring, baseline and endline data collected by the partners should be incorporated into the analysis. The evaluation will make use of before-after comparisons especially for assessing the improvement of practices related hygiene promotion, reproductive and maternal health, nutrition practices and living conditions.

EVALUATION QUESTIONS

The following evaluation questions represent the broad scope of the final evaluation. The questions will be further discussed and reviewed in cooperation with the selected evaluation consultant. • Has the project strategy and approaches adapted well to the changing context and needs of target beneficiaries and stakeholders? How well has the consortium worked with local institutions and stakeholders (partners, market actors, etc.) to address the needs in a coordinated manner? • What needs of the affected population have been best or least met? Did the different groups of beneficiaries feel safe and treated with dignity when accessing services/assistance provided or supported by LEARN? What are the most significant changes, positive or negative, identified by beneficiaries and local stakeholders? • How effectively has LEARN handed over or implemented the exit strategies related to institutional repairs, WASH infrastructure and health services to local actors identified in the mapping exercise? • To what extent has the multi-sectoral integrated approach helped improve beneficiaries’ dignity, quality of life and the ability to withstand future shocks as well (resilience)? To what extent do the newly added sectors in Phase II enhance the intended outcomes of the integrated approach of the LEARN Program? • How did LEARN leverage consortium for improved effectiveness, efficiency and learning (e.g. in logistics, procurement, BOQs, information management, accountability, RRM, CMU) in the current phase of the award. • To what extent have the learnings of Phase I been incorporated in Phase II programming and reflected upon by the consortium?

EVALUATION METHODS AND LIMITATIONS

The methodological approach will be outlined by the evaluator in the evaluation proposal – indicating expected scope regarding locations and interviews – and refined in detail in the inception report. While proposing the methods for data collection, consultant(s) needs to consider the COVID-19 related situation. The proposed data collection methods should be to a maximum extent statistically representative, but the safety of both enumerators and respondents is a priority consideration. Given the uncertainties while accessing the project areas for non-locals, consulting team must include local personnel and design the tools for remote data collection where required. Document Review • Relevant secondary literature, such as working groups or other agency reports • Project documents – such as proposal, budget, workplan, monthly and semi-annual report, BoQs, etc. • Project MEAL documents and data – baseline, endline, performance monitoring, third-party monitoring and progress reports, MEAL Plan and Indicator Tracking Tables Primary Data Collection • Key informant interviews with CMU and project staff of each partner • Key informant interviews with representatives of different beneficiary groups in the targeted population, local officials, medical staff, technical departments, NES Forum and coordination actors and other local stakeholders. • Focus group discussions with project beneficiaries (identified distinctly for each result), separately capturing the perspectives of different demographic groups (especially IDPs/returnees, male/female, older/younger) • Observations of field locations, services provided and infrastructure rehabilitated with LEARN funding • Case studies – one case study mandatory, more than one preferred. Topics of interest for case studies will be provided to evaluation team during the planning stage. At the same time LEARN will appreciate ideas brought by evaluation consultant/team • The team shall adhere to the data protection guidelines of SI Limitations Considering the situation of COVID-19, movement to project sites might be limited, planning for remote data collection of primary data should be considered while proposing different data collection methods.

DELIVERABLES AND TIMETABLES

Inception Report (max 10 pages) containing proposed methodology incl. proposed adjustments of the evaluation questions, a brief evaluability assessment, sampling strategy, tools and work schedule. Evaluability assessment will provide review of the degree to which the proposed evaluation questions can be answered and measured effectively and findings attributed to project. This should be done based on document review and consultation of the following aspects with project teams: • Available data: what relevant data is currently available to track progress and measure outcomes? What implications does availability or lack of data have for evaluation design? • Considerations for data collection and limitations: what limitations and challenges exist for data collection and how could these be mitigated or overcome? • Use of findings and lessons learned (including a proposition for relevant dissemination formats for internal and external use): how will project management team and stakeholders use findings? What are the implications of such use for evaluation design? Validation Presentation conducted by the evaluator at the end of the field mission to introduce, discuss and validate the initial findings, lessons and recommendations with LEARN CMU and partners Draft Report (Max. 30 pages excluding annexures) allowing for feedback from Consortium, within two weeks from the end of field assignment. Case study(ies) will be annexed to the draft report. Final Report (including annexures like assessment tool, list of referenced documents, timeline of evaluation, list of interviewees) incorporating feedback from Consortium to the Draft Report and final recommendations, within two weeks from receiving the response. Evaluation data including interview transcripts/summaries/recordings, photo documentation, site observation records, databases if produced etc. These deliverables are to be: • prepared in English • submitted to LEARN electronically via e-mail: meal.spe@learnconsortium.org and dcop@lernconsortium.org. Inception report

• Content: Evaluator provides clarifications on methodology (incl. list of documents to be reviewed, sampling, tools, work schedule, evaluability assessment)
• Responsibilities: Evaluator submits to LEARN electronically. LEARN provides all necessary inputs such as project documents, timing requirements etc.)

Validation Presentation

• Content: Initial Findings, lessons and recommendations
• Responsibilities: Evaluator carries out a validation session for LEARN ideally at field level LEARN organizes venue (physical or virtual) and invites participants

Draft of the final report

• Content: Full report (max 5 pages of executive summary, maximum of 25 pages of main chapters excluding annexes and forefront pages, font Arial 10 with 1.15 line spacing)
• Responsibilities: Evaluator submits the draft of the final report and presentation on key findings and recommendations to LEARN for the field teams. LEARN provides feedback within 5 working days of the draft report receipt

Final report

• Content: Revised report
• Responsibilities: Evaluator submits the draft of the final report to LEARN Presentation of the final report with LEARN CMU and Partner Focal Points

Evaluation data

• Content: Interview transcripts/summaries/recordings, photo documentation, site observation records, databases if produced etc.
• Responsibilities: Evaluator submits the data to LEARN in one package, all documents must be in either English (preferred) or Arabic

Desired structure of the evaluation report: Executive summary: Summary of the evaluation, with particular emphasis on the main methodology, findings, conclusions, lessons learned and recommendations. • Introduction: Description of the evaluated intervention, its logic, history, organisation and stakeholders. Presentation of the evaluation’s purpose and questions. • Methodology: Description of the sampling strategy and methods used for data collection; evaluation questions and description of the limitations. • Findings: Factual evidence relevant to the questions asked by the evaluation and interpretations of such evidence (answered evaluation questions). • Conclusions: Assessments of intervention results and performance against given evaluation criteria and standards of performance. o Problems and needs (Relevance) o Coordination and Integration (Coherence) o Achievement of purpose (Effectiveness) o Consortium management (Efficiency) o (Likely) achievement of wider effects (Impact) o (Likely) continuation of achieved results (Sustainability) • Lessons Learnt: General conclusions with a potential for wider application and use • Recommendations: Specific and actionable proposals regarding improvements of the project or management addressed to the client of the evaluation or other intended users. • Annexes: Terms of Reference, timeline of evaluation, tools, list of respondents, references, etc. The evaluation report will be reviewed against the Evaluation Policy’s “Criteria to Ensure the Quality of the Evaluation Report” as described in Appendix 1 of the USAID Evaluation Policy. Duration of the evaluation The expected period of the evaluation is to not exceed 5 weeks.

• Preparation (Inception Report) : Beginning September 2021
• Field Work : September 2021
• Draft Evaluation Report : September/October 2021

Management and Reporting lines • Position under the hierarchical supervision of: LEARN Deputy Chief of Party (DCOP) • Position under the functional supervision of: LEARN MEAL Specialist • The Consultant, CMU and Prime Recipient will discuss appropriate working relationships between them. • The consultancy deliverables be reviewed and inputs provided by the Review Group comprising of the MEAL focal points of all partners and CMU MEAL Specialist/DCOP. • Key interlocutors: All CMU Specialists; Consortium Technical Advisory Group members; Key program and/or technical focal points at partner agencies; SI HQ Desk team. Logistics The consortium partners will work with the selected consultant and team to identify any potential areas of concern related to security and logistics. The evaluation team is expected to be able to make all logistics arrangements independently, including the transport and accommodation inside NES. While LEARN will share necessary security and access information of which LEARN is aware with the evaluation team, LEARN will not assume responsibility for evaluation team safety or duty of care. If agreed selection of sites to visit requires the evaluation team to make trips to some distant or difficult locations, LEARN and evaluation team will discuss possible logistics support to be provided by LEARN. Consultant should keep in account the possibility of remote data collection due to movement restrictions. Final schedules will be adjusted during the initial evaluation planning period. The logistics costs must be included in the Financial Offer.

EVALUATOR PROFILE: TEAM COMPOSITION

The evaluation study consultant or team is expected to demonstrate the following experience and qualifications: • At least a Master’s degree in Social Science, Political Sciences, Statistics or related field (mandatory for lead evaluator) • Minimum of 5 years of experience in conducting evaluations for multi-sectoral/integrated program approach-based consortium projects/programs, collecting data and producing quality baseline/midterm/end line study reports, preferably for international non-governmental organizations or multilateral agencies (mandatory for lead evaluator) • Sectoral expertise and experience in emergency projects (mandatory) • Experience working in the conflict or post-conflict environments (preferred, not mandatory) • Good knowledge of the context and the conflict in Northeast Syria, where the project is being implemented (preferred, not mandatory) • Experience of effective interaction with local non-government organizations, government departments, and population (preferred, not mandatory) • Excellent spoken and written communication skills in English (mandatory) and Arabic/Kurdish (mandatory for at least one evaluation team member). Translation and Enumerator costs will not be covered by LEARN. • At least one field team member must be a woman and one field team member must be a man (mandatory) • Strong established field-based structure to enable autonomous data collection (Preferable). • At least two field team members must be able to access LEARN project sites in Northeast Syria, including one woman and one man (mandatory) • Evidence of similar work in the recent past (mandatory) • Knowledge of USAID-BHA donor requirements • Ability to deliver report within the deadline set out in the TOR

APPLICATION PROCESS

SOLIDARITES INTERNATIONAL invites consultants to apply for the provision of services as described above. Interested agencies must submit a Cover Letter outlining their financial offer, full technical proposal to meet the ToR (including the analysis and understanding of the present ToRs), scheduling, CVs of key evaluation team members (Lead evaluator, Sector experts) providing information on their capacity and experience showing that they are qualified for the services and 2 latest samples of Evaluations conducted previously by the Lead evaluator and/or Evaluation team. The Financial offer must be submitted in USD, with a TOTAL budget, with a breakdown of following chapters: I. Daily Rates/ Consultant Fees II. Logistics Costs (travel, accommodation, visa, etc. expenses) III. Taxes (all taxes that are payable by SI to the consultant according to the tax regulations in the consultant’s residence country Interested consultants may express their interest by sending an email to the following address: dcop@learnconsortium.org and meal.spe@learnconsortium.org by end of 23 July 2021 (UTC+3) and expressly mentioning « LEARN Final Evaluation Consultancy» The proposal will be evaluated in line with the service procurement guidelines of SOLIDARITES INTERNATIONAL for the use of Consultants. It is noteworthy that the interest shown by a Consultant does not imply any obligation on the part of SOLIDARITES INTERNATIONAL for inclusion in the short list. Applications without the Cover letter, Financial Offer and Technical Proposal, CVs of evaluation member(s), and 2 latest sample reports of past experience in evaluation will not be considered.

BUDGET AND PAYMENT SCHEDULE

The renumeration for the services offered will follow the below schedule. Milestones :

• 40% Following the signing of the Contract on Evaluation Services
• 30% After the draft evaluation report has been submitted and approved
• 30% After the final version of the final evaluation report has been submitted and approved

LEARN will require specific and data-supported answers to each of the agreed evaluation questions or strong justification why the data could not be obtained. Justification of data unavailability must be communicated by the evaluator without delay as soon as it becomes apparent and approved by LEARN. Failure to do so can result in decreasing the consultant’s remuneration. Document Review • Relevant secondary literature, such as working groups or other agency reports • Project documents – such as proposal, budget, workplan, monthly and semi-annual report, BoQs, etc. • Project MEAL documents and data – baseline, endline, performance monitoring, Primary Data Collection • Key informant interviews with CMU and project staff of each partner • Key informant interviews with beneficiary representatives, local officials, medical staff, technical departments, NES Forum and coordination actors and other local stakeholders • Focus group discussions with project beneficiaries (identified distinctly for each result), separately capturing the perspectives of different demographic groups (especially IDPs/returnees, male/female, older/younger) • Observations of field locations, services provided and infrastructure rehabilitated with LEARN funding • Case studies – one case study mandatory, more than one preferred. Topics of interest for case studies will be provided to evaluation team during the planning stage. At the same time LEARN will appreciate ideas brought by evaluation consultant/team Limitations Considering the situation of COVID-19, movement to project sites might be limited, planning for remote data collection of primary data should be considered while proposing different data collection methods.