Caritas Austria
TERMS OF REFERENCE (ToR)
FINAL EVALUATION OF THE ACTION
MIRA (Migrants’ Rights and Advocacy)
Project Title: MIRA – Migrants’ Rights and Advocacy Supporting and advocating for the protection of migrants’ human rights in Lebanon.
Project country/locations: Lebanon
(Beirut, Mount Lebanon, Bekaa and South of Lebanon)
Budget of the Action to be evaluated:
Total project costs: EUR 735.000
EU Contribution: EUR 700.000
Caritas Austria Contribution: EUR 35.000
Date of the Action to be evaluated:
Start: 01.01.2021
End: 31.12.2022
Deadline for submission
10th March 2023
- INTRODUCTION AND BACKGROUND
Although Lebanon is strongly affected by compounding crises, the country still hosts a large number of migrant and refugee populations. In addition to refugees, Lebanon has traditionally been a country of destination for many Asian and African Migrant Domestic Workers (MDW). Apart from official figures many more migrant workers are in the country with an irregular status.
In general, MDW are not protected by the country’s labor law, provision 7 under the Lebanese labor law even explicitly excludes domestic work. Instead they work under a set of laws, policies and customs called Kafala, a sponsorship system which is the legal basis for residency and employment for migrant workers. This system requires the employer to assume full economic and legal responsibility for the employee giving him/her considerable power over the worker. NGOs working in the field have denounced this system to create structural dependence between an employer and a migrant worker, which enables serious fundamental human rights violations.
MIRA intervention combined and continued previous advocacy and awareness effects aiming at improving migrant workers’ rights in Lebanon and pursuing the abolishment of the Kafala system with providing crucial protection and direct services for affected migrant workers. The action responded to immediate needs (direct support), especially focusing on social protection services for MDW (Beirut and remote areas) as well as long-term improvements of their situation through policy reforms.
In total MIRA targeted over 5.000 Vulnerable Female Migrant Domestic Workers and migrant communities in addition to state authorities, recruitment agencies, diplomatic missions, CSO’s, NGO’s and networks and the general public.
Implementing partners in Lebanon: AMEL Association, Caritas Lebanon and KAFA.
Overall Objective:
To contribute to the protection and defense of human rights of migrants in Lebanon and / by working towards the abolishment of the Kafala system
Specific Objective 1:
Abilities of migrant workers to access their rights and social protection services to prevent negative impacts resulting from migration are enhanced.
Specific Objective 2:
Advocacy for policies aiming at the protection of rights of migrant workers and monitoring the implementation of respective policies is increased. This includes concrete actions to enhance conditions for Migrant Domestic Workers (MDW).
Specific Objective 3:
The capacity of and collaboration between Lebanese CSOs working on achieving common human rights objectives for migrant workers and toward the abolishment of the Kafala system is strengthened.
Actions in brief:
- Provision of basic and humanitarian services in shelters for victims of GBV and human trafficking
- Provision of legal, medical and psychological assistance for MDW and migrant workers
- Carry out awareness sessions and social counselling
- Carry out events for and with migrant communities
- Provision of medical/health assistance/Covid19 awareness/ health advocacy
- Capacity building and empowerment for MDW and migrant workers
- Organising and implementing a media campaign
- Conduct a training for social workers of the MoL and recruitment agencies staff
- 10 Advocacy coordination meetings with CSO’s, human rights actors, community leaders and embassies.
- 3 Round Table Meetings with state and non-state actors on migrants’ rights topics
- Finalisation of a manifesto outlining demands/legislative objectives to be reached
- Drafting a domestic law to ensure fundamental labour rights to MDW and validation by coalition core group members
- Presentation of draft law to decision makers, CSO’s and service providers aiming at adoption of a new Labor Chapter on Domestic Work
For more detailed information, please refer to the logframe of this Action and to the project documents that will be made available upon assignment.
-
OBJECTIVES OF THE EVALUATION
Evaluations should provide an understanding of the cause and effect links between: inputs and activities, and outputs, outcomes and impacts. Evaluations should serve accountability, decision-making, learning and management purposes.
The main objectives of this evaluation are to provide the relevant services of the European Union and the implementing partners with:
- an overall independent assessment of the past performance of the Action to be evaluated, paying particular attention to its results measured against its expected objectives; and the reasons underpinning such results;
- key lessons learned, conclusions and related recommendations in order to improve future Actions.
The main users of this evaluation will be the Delegation of the European Union to Lebanon, Caritas Austria, Caritas Lebanon, Amel Association and Kafa; in other words those that are involved in the implementation and the financing of the Action to be evaluated.
- APPROACH AND METHODOLOGY
The evaluation will be undertaken in line with OECD/DAC standards for evaluations (Applying Evaluation Criteria Thoughtfully, 2021). Furthermore, the evaluation needs to include key informant interviews with partner staff and an examination of partnership principles. Due to vulnerability of target groups safeguards for confidentiality, privacy, and respect of dignity need to be respected. In addition, certain groups may not be interviewed directly but questions may be transmitted by the staff or case worker due to their safeguarding needs.
It is up to the evaluation team to design a final methodology and detailed evaluation questions (based on the guiding questions below) during the inception phase, so the team feels well placed to respond to the guiding questions outlined below in the most appropriate and credible manner.
The evaluation/review consists of several phases:
- Briefing
- Contract and Kick-off meeting: Contract is signed and a discussion of the assignment takes place. First documents, including available data, are provided to the evaluation team.
- Desk Study: The evaluation/review team studies all necessary project/programme documents; re-construct and analyse the intervention logic/programme theory and theory of change and its assumptions. Existing data needs to be analysed and interpreted.
- Inception-Phase: In the inception report the evaluators will describe the design of the evaluation and will elaborate on how data will be obtained and analysed. The use of a data collection planning worksheet or a similar tool is required. First interviews take place.
- Field-phase: Data and information needs to be gathered, analysed and interpreted. It is expected that the evaluation will include quantitative and qualitative data disaggregated by sex.
- Presentation: Presentation of key findings (feedback workshop) at the end of the field trip.
- Final Draft Report: Submission and presentation of final draft report, inclusion of comments from partners and contractor.
- Final Report: Submission of final report, see reporting requirements under point 4).
- All data collected needs to be disaggregated by sex.
For the different phases it is expected that data and information will be obtained through different methods such as: analysis of documents, structured interviews, semi-structured interviews face-to face or by phone, group discussions, others.
All data collected needs to be disaggregated by gender. It is expected that the evaluation/review team will present concrete recommendations which are addressed to the specific stakeholders.
- EVALUATION QUESTIONS
Criteria
Guiding evaluation questions
Relevance
1. To what extent where the activities tailored to respond to the needs of vulnerable migrant domestic workers and migrant workers?
2. To what extent was the project design appropriate to reach policy reforms?
3. To what extent are the objectives of the Action still valid for the partner organization and the beneficiaries?
4. Are the activities and outputs of the programme consistent with the overall goal and the attainment of its objectives?
Effectiveness
5. To what extent were the objectives of the programme achieved?
6. What are the (concrete) contributions of interventions of the programme for achieving the objectives?
7. What factors were crucial for the achievement or failure to the project objectives?
8. To what extent were the target groups reached?
9. To what extent did participants participate in and assume ownership of the project’s activities?
10. To what extent did the project encourage positive involvement or support from stakeholders
Impact
15. What are the positive and negative, primary and secondary long-term effects produced by the programme, directly or indirectly, intended or unintended?
16. To what extent has the project interventions contributed to changes in the lives of migrant domestic workers and migrant workers?
17. To what extent did the project enhance abilities of MDW and MW to access justice, rights and social protection services?
18. To what extent did other stakeholders assume responsibility for key aspects of the projects?
19. To what extent did coordinated and reinforced advocacy actions lay the ground for long-term change?
20. To what extent was capacity of and collaboration between Lebanese CSOs working on achieving common human rights objectives for migrant workers and toward the abolishment of the Kafala system strengthened?
21. To what extent the activities aiming at legal reform of the Kafala (Labour Law) have achieved to promote among stakeholders and public the need and merits of this (ambitious) reform?
22. Was there enough flexibility in the project to respond to changes in needs or increased vulnerability?
Sustainability
22. To what extent are the target groups capable and prepared to receive the positive effects of the programme intervention without support in the long term?
23. To what extent has the local capacity of partners been strengthened at organisational levels?
24. What elements of the project activities are likely to continue after funding ends?
- DELIVERABLES
The consultants will submit the following reports:
- an inception report (10 pages max. without annexes), clearly outlining the details of the evaluation methodology and approach based on preliminary findings from the document review. The report should include a tentative outline, a detailed work plan with key milestones as well as an evaluation matrix with key and specific evaluation questions, their expected data collection source and method of data collection. Interview guides and/or questionnaires should also be annexed to the draft inception report. The report should be 10 pages long excluding annexes. The report should be shared with Caritas Austria for consultation and approval before finalisation.
- a final draft evaluation report (about 25-35 pages without annexes), including a draft executive summary and the results-assessment form (part of the reporting requirement)
- and the final evaluation report (25-35 pages without annexes), the final executive summary and the results-assessment form (part of the reporting requirement).
- A dissemination meeting (proposed online or in Beirut), whereby key findings, lessons learned, and recommendations are shared with partners including a PowerPoint presentation of key findings and recommendations.
- A minimum of 5 case stories of best practices and/or most significant change stories of maximum 1 page each.
All reports need to be written in English.
The executive summary should summarize key findings and recommendations (three to five pages) and needs to be submitted as part of the final draft report.
The findings and recommendations of the draft final report and final report have to be structured according to the evaluation questions. An outline of the report’s structure needs to be agreed upon during the inception phase.
- RESPONSIBILITIES AND CONDITIONS
Confidentiality of information: All documents provided to the evaluator(s) and all data collected during field research must be treated as confidential and used solely for the purpose of this evaluation. Respondents shall not be identified in reports without their explicit written permission. Photo, video, and other research data that can be traced back to individual research participants shall be anonymized unless agreed otherwise, in writing, with the person concerned.
Ownership of data, findings, products: Caritas Austria retains full ownership of all data, findings, and products produced as part of this assignment.
Support by Caritas Austria and partners: All relevant documents including programme proposal, reports, etc. will be provided for the consultant(s). Field research can be supported and facilitated by Caritas Lebanon/Amel Association/Kafa based upon the methodology and approach defined. Relevant contacts of other stakeholders will be shared upon the consultant(s)’ request.
Safeguarding and protection: For the time of the assignment, the evaluator commits to safeguarding and protection policies and procedures that are shared and reiterated during the preparatory phase.
Distribution Policy: Intended users of deliverables are Caritas Austria, Caritas Lebanon, Amel, Kafa and potentially other migrant sector partners of implementing partners.
All deliverables generated through this evaluation will be subject to approval by Caritas Austria before public dissemination.
- TIMELINE
Phases/ Timeline
Timeline
Actions
Procurement Phase
Dec
2nd Jan
1st and 2nd week Jan
Jan
- Publication of ToR
- Submission of proposals by applicants
- Assessment of proposals and selection by Caritas Austria
- Contracting
Preparatory Phase
Jan
- Kick-off meeting with implementing partners
- Desk review of existing project documents and data
Inception Phase
End of Jan
- Submission of inception report
Feedback Inception Phase
1st week Feb
- 1 week after inception report has been shared with Caritas Austria
- Inclusion of comments in the inception report by Caritas Austria, Caritas Lebanon, Amel, Kafa
Evaluation Phase: Lebanon
2nd week and 3rd week Feb
- Research and data collection – Remote and/or physical to be determined
Report Writing and Feedback Phase
4th week Feb
- Preparation and submission of draft report
- Feedback by Caritas Austria
- Incorporation of feedback
Presentation of final draft report
28th Feb
- Presentation of the final draft report; meeting including Caritas Austria, Caritas Lebanon, Amel, Kafa
Submission of final evaluation report
10th March
- Submission of final report (hard copy and electronic copy) to contractor
The consultant is requested to include a suggested timeline in their technical offer.
How to apply
Interested applicants should submit their application via e-mail to [email protected] with the subject line ‘Final Evaluation MIRA”. Deadline for submission of applications is 02.01.2023. Consultants who do not submit their quotation by this deadline will not be considered.
The application should include as a minimum:
- CVs of the consultants
- Cover letter which clearly summarizes relevant experience
- Brief technical proposal including:
- Methodology and tools
- Detailed work plan incl. timeline
- Executive summary of a previous evaluation (name of client can be deleted)
Submission of tender application/offer:
Applications can be submitted in English or German.
Please note that only signed offers can be accepted. A scanned copy of the signature will be accepted (via e-mail).
Financial proposal (in EUR) including a clear breakdown of costs including fees and other expenses such as travel, accommodation and transportation
Key Qualifications in the team should be:
- Relevant academic degree (master level)
- A minimum of three years’ experience and expertise
- Team leader has conducted at least three evaluations in the last five years ideally in the relevant field
- Team member has participated in at least three evaluations ideally in the relevant field
- Knowledge of Lebanon
- Experience in project cycle management
- Experience preparing and analysing a theory of change
- Experience and expertise in evaluating cross-cutting issues
- Experience in social science methods
- Excellent oral and written English
- Sound MS Office and IT skills
Closing date: 2 Jan 2023