lbania, Armenia, Belarus, the Republic of North Macedonia, Ukraine: Call for Proposals for Final External Evaluation of the Regional Project: A Right to Family – De-institutionalization to Reform Child Protection Systems

INTRODUCTION: 

These Terms of Reference (ToR) serve as a request for proposals from individual consultants or consortiums, who are interested in conducting an external evaluation of the regional project: A right to Family – De-institutionalisation to reform child protection systems in Albania, Armenia, Belarus, North Macedonia and Ukraine (De-I project)”.

Details regarding contents of proposals and submission procedures are explained herein.

1. BACKGROUND AND RATIONALE: 

SOS-Kinderdörfer weltweit Hermann-Gmeiner Fonds Deutschland (HGFD) and the 5 locally registered SOS Children's Villages National Associations Albania, Armenia, Belarus, North Macedonia and Ukraine, since October 2021, are successfully implementing the regional project entitled “A right to Family – Deinstitutionalization to reform child protection systems”, funded by the German Federal Ministry for Economic Cooperation and Development (BMZ). The project aims to improve children´s rights with a specific focus on supporting the reform of alternative childcare systems and the deinstitutionalization process in the 5 project countries.

SOS Children's Villages in Albania, Armenia, Belarus, North Macedonia and Ukraine are non-profit non-governmental organizations and active members of SOS Children’s Villages International, which comprises more than 130 SOS Children's Villages national associations. We work together with a single vision: every child belongs to a family and grows up with love, respect and security.

Albania, Armenia, Belarus, North Macedonia and Ukraine are confronted with numerous challenges at different levels that make successful de-institutionalisation of their care systems massively difficult. In general, children in the target countries are placed in state care due to various circumstances related to family circumstances (e.g. poverty), individual characteristics (e.g. special needs/ disability) and specific risks (e.g. abuse and violence). Poverty is one of the main risk factors for family separation and institutionalisation of children. The placement of a child in state care has less to do with the fact that the child has lost parental care (orphans, children of parents who have been deprived of parental rights) but is a consequence of the widespread poverty and vulnerability of families and the unavailability of preventive support and alternative care services. Only if these challenges are met will countries be able to effectively address the causes of children's separation from their families through preventive services and, where necessary, provide quality care for children.

Poor planning and implementation of De-I strategies largely stem from a mismatch between stated policy objectives and actual capacities, resources, knowledge to implement these objectives. Implementation gaps and thus non-compliance with stated policy objectives stem from systemic deficiencies at macro, meso and micro levels.

Since the start of the regional project in October 2021, significant results are achieved so far in advocating for improved legal framework in childcare system, in strengthening the social service workforce to support the better protection and care of children, in raising awareness at all levels to promote family-based care and in developing of a range of community-based services and familybased care services.

The impact matrix of the project consists of regional impact matrix and impact matrixes of each country.

The regional impact matrix is a summary of country impact matrixes and includes the following:

Impact: 

Children who have to grow up without parents or are at risk of losing parental care (collectively referred to here as vulnerable children) are given special legal protection in Albania, Armenia, Belarus, North Macedonia and Ukraine and can exercise their rights to protection and individual care. They are thus in the long term enabled to lead socially active, participatory and independent lives.

Programme Objective:

Vulnerable children benefit from preventive services, family-based care and de-institutionalisation efforts in line with the UN Guidelines for the Alternative Care of Children in Albania, Armenia, Belarus, North Macedonia and Ukraine.

Expected results of the project are:

SUB-GOAL 1 (Macro -level): 

In all five target countries, the existing legal and policy frameworks for the childcare system are aligned with the UN Guidelines for Alternative Care for Children (supporting family- and communitybased alternative care rather than institutionalisation).

SUB GOAL 2 (Meso -level): 

Decision-makers at local, regional and national level and the wider society are aware of the need for De-I and actively support the core contents of De-I, i.e. preventing separation of children from the family of origin and promoting community-based care services.

SUB GOAL 3 (Micro -level): 

In all five programme countries, the relevant professionals are trained and use their expertise to provide high-quality and child rights-compliant support services for vulnerable children and their families.

SUB GOAL 4 (Micro -level): 

In all five programme countries, community-based prevention and alternative care services enable vulnerable children to grow up in their families of origin or to be placed in family-based care.

The regional project affects different target groups (TG) at all 3 programme levels, i.e.:

• MACRO: 1.060 representatives of the legislative decision-making bodies at national, regional and local levels, as well as representatives of the implementing government agencies, i.e. heads of relevant government agencies and departments, and mid-level professionals involved in the implementation of the adopted policies, laws and regulations.
• MESO: general society, especially about 5 million residents of the participating countries who will increase their awareness and understanding of alternative care, family care environment, positive parenting and the need for reforms resulting in DE-I, and at least 45 journalists and media representatives who will be trained on the alternative care system and are expected to support public awareness raising on the need for DE-I during and after the programme implementation; - MICRO:
• about 4.550 children & adolescents, mainly from programme sites, who are at risk of losing or have already lost parental care,
• min 4.232 parents/caregivers who demonstrate positive changes in parenting and protecting their children by adequately addressing their needs and rights.
• min 4.847 child welfare professionals and allied professionals (e.g. health workers, judges, police officers, etc.) involved in the protection, care and well-being of children who improve their professional skills and better protect and support vulnerable children and their families.

Each project country has its own impact matrix with planned sub-goals at each of the levels.

As the implementation of the project is coming to an end by 30th of April 2025, an external independent participatory evaluation is planned that will cover all 5 project countries. Contracting authority is SOS Children's Villages, Albania.

2. PURPOSE AND SPECIFIC OBJECTIVES: 

The main purpose of the external evaluation is to assess the relevance, coherence, effectiveness, efficiency, impact and sustainability of the regional De-I project, which is implemented in 5 countries.

The specific objectives of the evaluation are:

• To assess the impact, relevance and effectiveness of the activities implemented in De-I project.
• To assess how project interventions are compatible with the other interventions in the countries and how they are coordinated with other local stakeholders.
• To evaluate the efficiency of the project in relation to beneficiaries, cost and timeframe of the project.
• To assess the current figures of the Objectively Verifiable Indicators as found in the regional impact matrix and countries´ impact matrixes.
• To assess the sustainability of the project (institutional, social, financial, etc.) - To make recommendations for improving future interventions.

The desired results of the evaluation are:

• To document the impact of the project with special emphasis on the impact the project has had children without or at risk of losing parental care.
• To critically examine the impact matrix and verifiable indicators found in the original proposal and provide post-project figures along with a narrative explaining the reasons for under/over performance achievement.
• To provide commentary on the current political, economic, social and cultural factors influencing the implementation of the project.
• To provide commentary on the overall project design, the intervention logic and an analysis of the strategy and methodology used in regional De-I project.
• To document the communities’ attitude towards the project.
• To draw conclusions, state lessons learnt and make recommendations for future strategy and improvements in the implementation of the project.

Results of the project will also contribute to improvements in the implementation of activities in the next project cycle.

Key stakeholders to be targeted during the external evaluation are:

• Direct target groups (children and youth without or at risk of losing parental care, parents and caregivers, general society),
• Direct institutional target groups (representatives of state bodies at national, regional and local levels, child protection and social workers working for governmental and nongovernmental institutions, journalists and media professionals),
• Indirect target groups (those who were not directly participating in the activities, but are affected by the project),
• Project team on national and regional level
• Management on national level and other national or programme members in SOS National Associations
• SOS-Kinderdörfer weltweit Hermann-Gmeiner Fonds Deutschland (HGFD)

Key users of evaluations results are:

• Project level: Project team and implementing partners
• Management level: National Director (ND), National Departments (programme development, communication, finance etc.),
• Regional level: SOS Regional Office CEE/CIS/ME
• Global level: SOS Children´s Villages International
• Donor: German Federal Ministry for Economic Cooperation and Development (BMZ) and SOS-Kinderdörfer weltweit Hermann-Gmeiner Fonds Deutschland (HGFD)

3. SCOPE OF WORK: 

The external evaluators will conduct the field evaluation in the five project countries: Albania, Armenia, Belarus, North Macedonia, and Ukraine. The regional De-I project is implemented at national, regional and local levels in the 5 countries. At regional/local levels, the project is implemented in the following locations:

• Albania: Tirana, Shkodra, Vlora and Korca municipalities
• Armenia: Yerevan and Shirak regions
• Belarus: Minsk and Mogilev regions
• North Macedonia: Skopje and Pelagonija regions
• Ukraine: Ivano-Frankivs'k, Kolomyia and Horodenka in Ivano-Frankins'k region and Fastiv, Boyarka, Irpin and Borodianka in Kyiv region.

Considering that each country has its specific project course and its own project results, due to various contexts determined by the socio-economic development of the country, the policies and awareness at the national level, the legal framework development, emergency context, etc., it is important that the approach of the external evaluation is country specific, and conclusions are provided for each country individually.

The evaluation questions should be revolved around the following criteria and areas: Relevance, Coherence, Effectiveness, Efficiency, Impact, Sustainability, and Project Management and Coordination. On the basis of the above-mentioned evaluation criteria/area, instruments /questionnaires for the field mission should be prepared for targeted stakeholders. An illustrative set of questions is provided below, which would have to be elaborated further during the external evaluation process.

In addition, evaluators should also assess the extent to which gender aspects were considered in planning and implementation of the activities.

Evaluation Issue / Key Guiding Questions / Relevance: 

The extent to which the aid activity is suited to the priorities and policies of the target group, recipient and donor.

• To what extent was the project focused on the intended target group?
• What were the specific criteria for the selection of beneficiaries?
• To what extent did project participants meet the selection criteria?
• To what extent did the project respond to the needs of the community?
• To what extent did the project interventions respond to the needs and priorities of the project participants?
• To what extent have the project adjustments made so far been relevant?

Coherence: 

How well does the intervention fit? 

• How coherent are the planned activities with human rights principles (inclusion, participation), conventions and relevant standards/guidelines (national and international ones)?
• To what extent did the project comply with the overall national/regional development objectives specified in the laws/bylaws and government programs on these issues?
• To what extent were there synergies and links between the planned project and other interventions by the same actor (organisation) and other actors?
• To what extent did the intervention create added value and was duplication avoided?
• To what extent is the project intervention implemented in coordination with other stakeholders at various levels?

Effectiveness: 

A measure of the extent to which an aid activity attains its objectives.

• To what extent have the project objectives been achieved?
• To what extent have the project strategies, methodologies, tools and processes contributed to the achievement of the planned results? Is replication or removal of certain approaches required to increase effectiveness?
• To what extent were the project objectives and activities in compliance with the target group needs?
• To what extent were the beneficiaries aware of the project and the services it provided? Did all the targeted beneficiaries receive services by the project?
• To what extent were beneficiaries satisfied with the project interventions?
• Does the support system built in the target communities effectively respond to the situation of the target group?
• To what extent were the state authorities at national, regional and local levels involved and provided support to the project?

Efficiency: 

An economic term, which signifies that the aid uses the least costly resources possible in order to achieve the desired results.

• Were activities cost-efficient?
• Were objectives achieved on time?
• Was the project implemented in the most efficient way compared to alternatives?
• Were the investments, operating expenses and personnel in proportion to the intended goals? Could the results have been achieved at lower cost through applying a different approach?

Impact: 

The positive and negative changes produced by a development intervention, directly or indirectly, intended or unintended.

• What contribution did the specific objective and planned sub-goals made to the overall objective (impact)?
• What was the impact against the planned results of the project, in terms of changes brought about in the living circumstances of beneficiaries and communities?
• To what extent has the quality of support improved the lives of the beneficiaries?
• What was the impact beyond the planned results of the project, in terms of changes brought about in the living circumstances of beneficiaries and communities? (positive and/or negative)
• Is the intervention transformative: does it create enduring changes in norms, and systems whether intended or not?
• Is the intervention leading to other changes, including “scalable” and “replicable” results?

Sustainability: 

Concept concerned with measuring whether the benefits of an activity are likely to continue after donor funding has been withdrawn.

• To what extent can activities, results and effects be expected to continue after the project financing (BMZ/HGFD) has ended? How was sustainability ensured (structurally, economically, socially, ecologically etc.)?
• What long-term capacities (and in which areas) have been built up among the target group to be able to continue the implement measures on their own? Do relevant stakeholders demonstrate commitment and capacity to continue project activities or replicate those?
• How many direct and indirect beneficiaries (children without or at risk of losing parental care) benefit from the services of the project? To what extent are the results which they have reached sustainable and are the results effective after the direct beneficiaries left the project?
• What positive changes (role, behaviour, mechanisms, networks and others) will favour civil society in the long term?

Project Management and Coordination: 

Evaluation of the role of the project management and coordination in ensuring quality implementation.

• To what extent did the project have appropriate management and coordination structures? Were these structures aimed at the quality of the project implementation? How effective were the project’s planning, monitoring, evaluation, reporting and learning mechanisms?
• To what extent was the project delivered in close cooperation with partners? Which other partners were involved in the process of management and coordination and how did this affect the quality of implementation?
• Were there any challenges or new/changed risks encountered during implementation? If so, what were they and how were they addressed?

Gender Mainstreaming: 

Extent to which gender was considered in planning and implementation of the activities

• Is the project responsive to gender needs, social and cultural values, conditions and practises?
• Has gender been considered in the implementation of relevant project activities?

4. METHODOLOGY AND APPROACH: 

The external evaluation should be based on a participatory approach involving and engaging a wide and diverse range of stakeholders. Stakeholders’ participation is necessary for accountability, promoting ownership and sustainability, facilitating buy in, and further use of the evaluation recommendations.

The evaluation should imply the inclusion of various 'rights holders' who benefit from the project, as well as the 'duty bearers' or those responsible and accountable for providing services. This is necessary to assess whether benefits and contributions are fairly distributed by the interventions being evaluated.

The evaluation approach should be results-oriented to provide evidence of both quantitative and qualitative achievements as per planned project results. Hence, both primary and secondary data should be used in the evaluation and be collected from a wide and diverse range of primary and secondary sources.

Methods and tools may include, but are not limited to:

• Desk research (analysis of internal key reference documents and of external data sources)
• Interviews (structured and/or semi-structured; in person)
• Focus groups
• Quantitative data collection (e.g. through surveys)
• Case study of randomly selected beneficiaries (present and those who exited the project)
• Other methods relevant to evaluation objectives and scope

The external consultant is asked to come up with a suitable methodology by developing all required data collection instruments, sampling and data analysis methods as per requirement to answer all evaluation questions. The evaluation is expected to follow ethical evidence generation and ensure ethical involvement of children in evaluation. A gender-sensitive approach to evaluation process is also expected.

The external consultant is expected to act in line with the SOS Child Safeguarding Policy and the Code of Conduct, and to take required pre-cautions for handling all data collected in a save and respectful manner. A participative approach should be ensured at any stage.

4.1 WORK PLAN: 

The evaluation will be carried out in 3 stages. All steps that lead to deliverables are marked in bold.

Stage 1: Prepare

• Undertake all required preparations for the field evaluation: propose detailed methodology, conduct desk research, identify key informants, plan data collection process, organization & logistics, and prepare field visit schedule. Submit and secure approval for the refined evaluation design and methodology in an inception report: document (i) proposed methodological set-up, (ii) proposed data collection tools (iii) proposed time schedule for data collection.

Stage 2: Conduct

• Collection of data as per the agreed methodology and tools
• Facilitate learning during the data collection exercise (regular exchange with SOS staff)
• Analysis of the data

Stage 3: Submit findings

• Preparation of a draft report in English Language.
• Presentation of the findings to relevant stakeholders at national and regional levels.
• After having received feedback from the various stakeholders – finalize the report and submit it in English language by 11th of April 2025. The final report should follow the structure and content as outlined in the terms of references.

The evaluation will be conducted starting from 13th of January 2025 and the final report should be submitted by 11th of April 2025.

Interested consultants must present during their application the anticipated work plan and the time schedule required for: 1) preparation of the evaluation and development of data collection tools, ii) desk research, iii) data collection, iv) data analysis, v) preparation of draft report and presentation of the findings, vi) incorporation of feedback and preparation of final report.

4.2 DELIVERABLES: 

The external consultant should prepare the following key deliverables:

1. Refinement of evaluation study design and methodology in an inception report
2. Draft evaluation report in English
3. Presentation of preliminary results to relevant stakeholders on national and regional levels 4. Final report in English

4.3 REPORT FORMAT: 

The evaluation will result in the conciseevaluation report, in English language with a maximum length of 50pages including an Executive Summary and excluding annexes. The final report should be provided electronically to the regional contact person responsible for coordinating the evaluation process.

Please see appendix 1 in this ToR about the structure of the evaluation report that must be followed.

5. PROFILE OF THE EVALUATION TEAM: 

The service provider should consist of a team of experts with national consultants in each country.

It is preferable that this team is coordinated by one team leader, who has international experience. Roles and responsibilities should be distributed among team members, in accordance with their competence and level of expertise and meet the requirements of this Terms of Reference. The leader of the evaluation team would be responsible for quality and timely completion of the external evaluation in line with the requirements of the Terms of Reference.

External evaluators should not be biased and have any reason for conflict of interests.

The external consultants should have the following core competencies:

• A social science background
• Proven competency in monitoring and evaluation, including impact assessment or project evaluation
• Proven experience in participatory processes and data collection methods
• A good understanding of development work
• Experience/knowledge of children rights and particularly related to child welfare systems, alternative, community based social services and de-institutionalization processes and preferably for countries of concern,
• Good facilitation and interpersonal skills
• Strong analytical and conceptual skills
• Excellent written communication skills
• Ability to transfer complex concepts and ideas into practical and simple language
• Excellent English language skills and national consultants should have excellent language skills of the five project countries (Albanian, Armenian, Belarus, Macedonian and Ukrainian); and
• Ideally experience in organising research processes with/for SOS Children’s Villages
• Ideally experience and credibility in providing evaluating services to BMZ financed projects.

6.1. GENERAL INFORMATION FOR BIDDERS: 

• This Request for Proposals is open to all national and international suppliers (independent consultants or companies) who are legally constituted and can provide the requested services.
• Bidders shall bear all costs of the bid preparation; costs of proposal development cannot be included as a direct cost within the assignment.
• Proposals and all supplementary documents must be submitted in English; financial bids to be stated in Euro.

6.2. MODIFICATION AND WITHDRAWL OF BIDS: 

Bids may be withdrawn on bidder’s written request to SOS Children's Villages, Albania prior to the closing date of this Request for Proposals. Any corrections or modifications must also be received in writing prior to the closing date.

Changes must be clearly stated in comparison with the original proposal. Failure to do so will be at bidder’s own risk and disadvantage.

6.3. RIGHTS OF CONTRACTING AUTHORITY: 

Throughout the bidding and evaluation under this Terms of Reference, contracting authority maintains the rights to:

• Contact any or all references supplied by the bidder(s).
• Request additional supporting or supplementary data from the bidder(s).
• Arrange interviews with the bidder(s).
• Reject any or all proposals submitted.
• Accept any proposals in whole or in part.
• Negotiate with the bidder(s) who will be considered for contracting; and
• Contract any number of candidates as required to achieve the purpose of this assignment.

HOW TO APPLY: 

1. Application Documents: 

Interested consultants should submit a detailed technical and financial proposal on the basis of following points:

• A letter of interest stating eligibility for the assignment.
• CVs of the consultant(s) to undertake the assignment, including a summary of similar assignments undertaken previously.
• A technical proposal, which should include information about the evaluation team, a description of the proposed methodology, a schedule of planned activities, proposal how the project team will be involved in the study, description of main deliverables .
• Financial proposal statement of the requested amount per consultant/day and the overall gross amount for execution of the contract (all expenses and taxes included). In case there is no bilateral agreement between Albania and resident country of the service provider, then 15% of income tax should be included in addition in the financial proposal.
• At least two references for previous experiences of consultants in evaluations, preferably in the field of child welfare and alternative care or similar.
• An example of a recent/relevant evaluation report (if available for public use).

2. Submission of Application Documents and Deadline: 

Bids need to be submitted electronically to the following e-mail address: info@soskd.org.al with subject line: “Application to conduct external evaluation for BMZ De-I project” by 11th November 2024, 17:00 CET. Late submissions shall not be considered.

Technical and financial proposals are to be submitted as separate PDF files, with the expressions “technical proposal” and “financial proposal” used in the documents’ filenames respectively.

Questions related to the application process, including assistance with provision of potential contacts of national experts in 5 project countries, must be addressed at info@soskd.org.al.

2.1. Modification and Withdrawal of Bids: 

Bids may be withdrawn on bidder’s written request to SOS Children´s Villages, Albania prior to the closing date of this Request for Proposals. Any corrections or modifications must also be received in writing prior to the closing date.

Changes must be clearly stated in comparison with the original proposal. Failure to do so will be at bidder’s own risk and disadvantage.

2.2. Rights of Contracting Authority: 

Throughout the bidding and evaluation under this Terms of Reference, contracting authority maintains the rights to:

• Contact any or all references supplied by the bidder(s).
• Request additional supporting or supplementary data from the bidder(s).
• Arrange interviews with the bidder(s).
• Reject any or all proposals submitted.
• Accept any proposals in whole or in part.
• Negotiate with the bidder(s) who will be considered for contracting; and
• Contract any number of candidates as required to achieve the purpose of this assignment.