Concern Worldwide

Consultant (End of Project Evaluation)

Concern Worldwide

Job Description

Emergency Education, Protection Support and Resilience Building for Refugees in South Eastern Turkey Phase III (ECHO/TUR/BUD/2020/91000) 1. INTRODUCTION

Concern Worldwide is an international, non-governmental, humanitarian organization dedicated to the reduction of suffering and working towards the ultimate elimination of extreme poverty in the world’s poorest countries. Concern began working in Turkey in 2013 due to the high number of refugees coming to Turkey. Today, it is estimated there are over 3.6 million refugees, with over 97% of refugee population residing outside of camps in challenging circumstances.

Concern is funded by ECHO to implement an ‘Emergency Education, Protection Support and Resilience Building for Refugees in South Eastern Turkey, Phase III’,** which started on 01/09/2020 and ends on 30/11/2021, for a total of 15 months. To document and assess impact of the programme, Concern intends to undertake an external evaluation of the programme. The external evaluation will evaluate the programme achievements against the results and indicators specified in the logical framework of the programme. The final evaluation will be implemented in the two months after the end date of the project on November 30th 2021 in the 4 provinces targeted by this action. (Sanliurfa, Adiyaman, Malatya, and Gaziantep).

The evaluation will involve a review of end line data against the baseline to assess programme progress against the Principle and Specific Objectives and targets including results of the programme, a review / Analysis of risks and assumptions stated in the proposal.

The principle objective of the programme is ‘*To contribute to the establishment of a protective environment for refugees in Turkey by addressing education and protection needs of the most vulnerable individuals through dignified and appropriate support*.

The Specific Objective of this Action is measured with indicators below[1]: · % of children who report improved resilience, coping, and adaptation mechanisms as a result of participation in psychosocial activities (80%) · % of parents and caregivers reporting an improvement in their children’s overall wellbeing (70%) · % of school-age refugee children out of school enrolled in the formal education system (80%) · % of surveyed refugees satisfied with the protection services received through intervention (80%) · % of beneficiaries who report improved wellbeing as a result of Concern Case Management (60%) The programme is structured into two results focusing on a different aspect of protection programming, which is measured by the indicators listed: Result-1: At-risk children are enrolled in formal education and receive PSS that contributes to their resilience and supports their success in school[2] · # of students provided with psycho-social support services (1980) · # of caregivers completing the Caregiver Engagement programme (990) · # of targeted girls and boys enrolled in formal education services (1100) · # of education technical Capacity Building trainings provided to implementing partners (4) Result-2: Refugees are aware of their rights and obligations and receive protection services appropriate to their needs · # of refugees participating in group activities to provide information and raise awareness (4800) · % of refugees who are aware of their rights and obligations relating to accessing protection and social services (75%) · # of inventories of functioning services (including corresponding referral pathways) conducted and updated regularly (4) · # of newly-identified and assessed individuals (4800) · # of (unique) individuals referred to relevant external (specialized) services (from all sources) (2650) · # of external referrals made (4500) · # of protection services/sessions provided/conducted to refugees (3840) · # of refugees (unique individuals) provided with protection services (3840) · # of persons who obtain appropriate documentation/legal status (500) 2. PURPOSE OF THE EVALUATION

The purpose of the evaluation is to assess the degree in which the project ‘**Emergency Education, Protection Support and Resilience Building for Refugees’,** has met the objectives and results as outlined in the programme proposal’s narrative and logical framework and to assess evidence of outcomes and impact on people’s lives over the programme. The evaluation also aims to assess the adaptive management response to Covid-19 by the Concern Turkey team. The evaluation will consider the quality of implementation of the programme and identify lessons learnt, unexpected (good or otherwise) results and best practices to be documented for future programming.

The overall objective of the evaluation is to assess the effectiveness of ECHO funded Protection programme and to assess the degree to which the anticipated improvements in people’s lives have been achieved. The specific objectives of the evaluation are set according to the Development Assistance Committee (DAC) criteria and should answer relevant questions listed below against the DAC criteria: Please note the main question should be answered, with the following sub-heading questions as potential areas to explore in the evaluation: Relevance/Appropriateness [3]** v Is it still the right thing to do?[4] · Was the programme concept, design, and implementation arrangements relevant in current context and complementary to the services already provided by governmental and non-governmental service providers? · What role has Concern’s engagement with Mukhtars, government and community leaders played to address the needs of refugees? · Whether or not beneficiaries were assisted in proportion to their needs and vulnerabilities? · What adaptations were made to fit with the realities in an evolving context and respond to changing needs and priorities of vulnerable refugees? Effectiveness

v To what extent have objectives been met? · How well did the team and partners apply adaptive management principles in monitoring and evaluation to improve programme quality? · Were the targets set in the logical framework achieved? (A comparison between baseline and endline data for each indicator and if any change is statistically significant).

· How effective is the information management system for CM and referral? § Can this system be made more efficient? § How timely were cases responded to? · How effectively are the information management tools used? § To what extent did they support the risk analysis and implementation? § How effectively is the data in each tool used? · Assess how effectively the SDQ and CYRM tools were used? § How did the data contribute to improving the children’s wellbeing through PSS activities? · Assess the effectiveness of Concern’s management of the programme § Has the programme targeted the most vulnerable refugees effectively and has met their priority needs as per commitments made in the proposal, and § Were issues of equality, gender, age, ethnicity, disability, legal status or any other social identities appropriately addressed in order to ensure the most vulnerable participate and benefit equally in the project. Such as, how are gender issues considered in caregiver sessions? Efficiency

v Were resources used efficiently? · How cost effective was the project in accounting for inflation, delays, and insecurity? · How the funding inputs led to outputs ? · How efficient CM and IPA processes were? · Is the documentation of IPA and CM cases appropriate? Is there any important information missing or is there opportunity to simplify the documentation? Impact

v What indications are there of significant changes taking place? What indications are there of wider impact on project participants, partners and the community? · This question reviews stakeholder’s participation, including gender balances in participation and influence as well as any external factors such as other programmes, social and economic factors have had any influence in contributing to bringing about change or not; and conflict sensitive programming and how this was accounted for throughout the programme.

· Assess the quality of programme implementation in terms of satisfaction of programme participants.

· Was support to refugees and host communities within the intervention area provided equitably? · Assess the degree to which changes have been brought about in the lives of programme participants and if these have been negative or positive · Identify if there were any unexpected outcomes / results of the Action (Negative and positive) Sustainability

v To what extent might the achievements of the programme last in the life of people benefited.

o To what extent did Coordination between Concern and Mukhtars, Government bodies and partners support the sustainability of the programming.

In addition to the DAC criteria, the evaluation will also try to answer below areas: Connectedness

v How did the protection programme take account of other sector responses, namely education, livelihoods and programme adaptions? · How was the programme supporting the broader protection environment for refugees in Turkey? Coherence

v Did the programme adhere to core humanitarian principles and policies, as well as to Concern's Approach to Protection and Concerning PSS policies? Coverage

v To what extent were protection needs addressed? · Examining the reasons why the programme provided or failed to provide affected populations with assistance and protection proportionate to their needs.

· What role did partnerships with LNGOs provide to address the needs of refugees? Coordination

v Assess the added value of links and coordination with other key protection actors, including participation in protection cluster / working group and sub-clusters / sub-working groups 3. METHODOLOGY

External evaluators will independently assess the project’s relevance, effectiveness, efficiency, impact, and sustainability in line with extended DAC evaluation criteria. This final evaluation will be participatory and seek input from all parties involved including partner organizations, programme participants, beneficiaries, PDMM, Municipalities, Education and Protection Working Groups’ Coordinators and the ECHO Technical Advisor. The evaluation team will be required to meet with programme participants in all 4 projects targeted by this action (Sanliurfa, Adiyamen, Malatya, Gaziantep) The final evaluation will contribute to lessons learned and knowledge on resilience in protection and education in protracted crisis, response in urban settings and the findings will be disseminated to ECHO and published on Concern’s website.

Concern will make sure to contract external evaluators according to Concern’s Evaluation Guideline to undertake the evaluation independently and to meet the donor’s requirements. The final evaluation will be a combination of primary and secondary Data Collection and review process, with all endline quantitative data completed by the Concern Turkey team in advance of the evaluation beginning and triangulating the data. For primary qualitative data collection, tools like Focus Group Discussions (FGDs), Key Informant Interviews (KII) or interviews with beneficiaries, interviews / visits of partners and above-mentioned government stakeholders and relevant programme staff will be used. Concerning the secondary data, a process of desk review of programme documents, reports, and monitoring data will be included.

The Programme Director (PD) and the Education and Protection Coordinators will brief the evaluators on the programme, subsequently, introduce the focal point for the external evaluation; nominated by the PD. Concern’s focal point will provide all required documents, information and local contextual knowledge throughout the evaluation. The evaluators and the focal persons will agree on the design and process of the evaluation in detail, finalising dates, methods, tools, and sampling for the evaluation. Cooperation between the Concern focal persons and the evaluators is expected to increase the quality and relevance of the evaluation. However, Concern’s focal person will not influence the independence of evaluation by their level of involvement. The Evaluators will endure personal data protection of all involved in accordance with Turkish Personal Data Protection regulation as well as European General Data Protection Regulation DPR.

4. TASKS AND RESPONSIBILITIES OF THE EVALUATION TEAM · Inception report (maximum 4 pages) consisting of the methodology and consolidated Work Plan for the evaluation illustrating timelines, roles and responsibilities.

· Review programme documents · Prepare an evaluation plan, tools and methodologies in consultation with the focal person and in consultation with the PD for approval · Conduct interviews, questionnaires and other necessary processes for information collection and analysis.

· Carry out the review with participation of beneficiaries where possible hold FGDs (considering COVID-19 restrictions).

· Travel to partner locations (Adiyaman, Malatya and Gaziantep) and conduct interviews with their relevant staff including management and implementation staff (considering COVID-19 restrictions) · Conduct workshop / meetings to discuss and agree upon findings and recommendations · Provide an in country debrief to country management · Prepare a comprehensive evaluation report in line with above-mentioned objectives of the evaluation and recommendations for programme improvements in design and implementation · Adhere to Concern’s Code of Conduct and Associated policies including compliance with Concern policies on security and any other relevant policies 5. EXPECTED OUTPUTS

The expected outputs include: · Data collection tools and desk review methodologies.

· Presentation of initial findings to the PD, Coordinator" id="link" class="link">Meal Coordinator and relevant programme staff before departure for input, feedback and recommendations.

· A draft report will be submitted to the PD for review with CD, DO and RD and feedback within three weeks after the field work has been completed · A final report incorporating comments on the draft report and presenting final key findings and recommendations as explained above, is expected within one week after receiving feedback on the first draft.

6. FINAL EVALUATION REPORT STRUCTURE

The Evaluation report should consist of the following (not exceeding 20 pages): · Executive summary (1page) · Introduction (1page) · Methodology (1.5 pages) · Limitations (0.5pages) · Findings to the DAC criteria and overall assessment (15 pages) The evaluators will score the project against each of the DAC criteria according to the below table: 4 Outstanding performance 3 Performance in line with what would be expected of a well-functioning organisation 2 Generally acceptable performance but with some clear, and documented, shortcomings 1 Barely acceptable performance with some major shortcomings and reservations Totally unacceptable performance or insufficient data to make an assessment · Detailed conclusions and recommendations as per the objectives of the Evaluation (1page)[5] · Appendices of (a) Terms of reference, (b) Summary of activities, meetings, and discussions during the evaluation, (c) Summary results of field work, (d) Brief description of any workshops / training provided, and (e) Bibliography.

7. COMPOSITION, SKILLS, AND EXPERIENCE OF THE EVALUATION TEAM

The evaluation team should consist of two or more consultants to undertake the evaluation process and they should be selected as per the below essential criteria: **
  • More than 5 years experiences in undertaking independent evaluations of humanitarian responses programmes using DAC criteria, preferably in protection and education sectors **
  • Technical experience in the Education and Protection sector. **
  • Strong understanding of the Turkey context, GDPR and Turkish data protection regulations **
  • Understands key concepts of economy, efficiency, effectiveness, impact, design, relevance**
  • Has strong analytical, facilitation, listening and writing skills in English **
  • The evaluation team should include fluent Arabic and Turkish speakers so that the evaluation team will be able to conduct meetings independently with project participants in Arabic and Turkish**
  • Proven ability to produce high quality report e.g. evaluation reports from the past **
8. TENTATIVE TIMEFRAME

· The evaluation is expected to be undertaken over a period of 2 -3 weeks between December 15th 2021 and January 31st 2022 a 9. ROLES AND RESPONSIBILITIES

Concern Turkey will be responsible for:

· Approval of the final evaluation plan, tools and methodology · Providing relevant documentation to the evaluation team on the programme activities and an initial briefing on the response as well as expectations of this evaluation · Provision of timely feedback to the evaluation team on first draft of report · Communication of results of the evaluation to key stakeholder groups in the appropriate languages · Providing in-country (Sanliurfa) airport pick-up, transportation for traveling to partner offices and programme locations · Providing in-country working place External evaluation team will be responsible for:

· Inception report (maximum 4 pages) consisting of the methodology and consolidated Work Plan for the evaluation illustrating timelines, roles and responsibilities.

· Developing methodology and evaluation plan for approval by Concern Turkey outlining expected sources of data · Conducting interviews with beneficiaries, stakeholders and Concern staff · Interpretation/translation of interviews with project stakeholders from Arabic and Turkish to English · Data compilation and analysis · Report writing including finalisation according to feedback provided.

· Presentation of draft findings to senior Concern staff (CD, PD, CFC, Programme Coordinators, Programme Managers, MEAL Coordinator, MEAL Manager, IM Manager) · Hotel accommodation for evaluation team in Sanliurfa and to visit the project partners in Adiyaman, Gaziantep and Malatya.

· Flights to Turkey (if needed) · Visa and travel documents The evaluation information and report will be the intellectual property of Concern. Its content can only be shared with third parties only with Concern’s consent. Sensitivities to confidentiality must be respected. Names and identifiable data associated with any particular quotes or experiences should not be cited. The evaluation team, if collecting any personal data will anonymize the information to avoid any unintended harm.

10. DOCUMENTS REQUIRED TO ASSESS CONSULTANTS EXPRESSION OF INTEREST

Applicants should submit the following documents: · Proposed methodologies and workplan · Profile of the evaluation team · Samples of previous work and list of evaluations done · Cost of proposal (all inclusive) · Confirm ability to provide formal invoice (providing registration and other related documents)  
Apply Now