Program & Policy Evaluation
Measuring a project’s progress through key indicators and evaluating its impact on beneficiaries are crucial in making sure program budgets are spent efficiently and effectively to attain the goals set out. High-quality monitoring and evaluation of programs allow development practitioners to better gauge their impact while improving service delivery. In particular, learning from past experiences and efficiently planning and allocating resources allows them to demonstrate accurate results and remain accountable to their key stakeholders.
UNICEF Tajikistan Country Programme Evaluation 2020
Evaluation, Country Programme Evaluation
The goal of the UNICEF country programme is to accelerate progress towards the realisation of the rights of all children in Tajikistan. Country Programme Evaluations (CPEs) in UNICEF are strategic evaluations that provide an assessment of the totality of UNICEF's programme of work in a country. In this respect, the overall objectives of this evaluation are to look back and assess the relevance, effectiveness, efficiency, sustainability and resilience, coherence, and longer-term changes associated with UNICEF's portfolio, and programmatic and strategic choices made in the design and implementation of the country programme (CP) to identify good practices and draw lessons and forward-looking recommendations that can inform the 2021–25 CP planning process, which will commence during the last quarter of 2019. Development Analytics is part of the consortium led by IOD PARC in conducting the Country Programme Evaluation of UNICEF Tajikistan (2016-2020).
February 2020 - ongoing
ESSN Mid-Term Review 2018/2019
Refugees, cash transfers
Turkey currently hosts more refugees than any other country in the world. There are 4 million registered refugees in Turkey, of whom 3.6 million are Syrian refugees. The ESSN launched in December 2016 with the objective of stabilising or improving living standards of the most vulnerable out of camp refugee households. The ESSN was designed in conjunction with the Government of Turkey and is implemented through a partnership between WFP, TK, the Ministry of Family, Labour, and Social Services (MoFLSS) and Halkbank.
The World Food Programme (WFP) Turkey Country Office on behalf of the ESSN stakeholders commissioned this mid-term review of the Emergency Social Safety Net (ESSN), funded by the Directorate General for European Civil Protection and Humanitarian Aid Operations (DG ECHO). The first phase of the ESSN (ESSN 1) ran from September 2016, with roll-out starting in early 2017; while ESSN 2 commenced in January 2018 and will run until March 2020. This review covers the period from May 2018 to November 2019. In collaboration with Oxford Policy Management (OPM), Development Analytics was involved in organising fieldwork, carrying out focus group discussions and key informant interviews, and conducting quantitative and qualitative data analysis for the study.
October 2019 - February 2020
Decentralised Evaluation of the ECHO funded Emergency Social Safety Net (ESSN) in Turkey
Poverty, Cash Transfers
Turkey is the country with the largest refugee population in the world with 3.9 million refugees, 3.6 million of whom are Syrians as of April 2019. The Emergency Social Safety Net Programme (ESSN) started being implemented at the end of 2016 and aimed to provide at least 1 million refugees with an unconditional and unrestricted cash transfer of 100 TL per month (which was raised to 120 TL later on) making it the largest ever EU-supported humanitarian cash transfer programme. The programme is available to all foreigners with refugee status who live off-camp and are under temporary protection or international protection. The programme is funded by the European Commission through European Civil Protection and Humanitarian Aid Operations (ECHO) and is implemented through the Turkish government social welfare system by means of an ATM cash card. The WFP is a key stakeholder in the project and worked closely with ECHO to design and implement the programme. The interim evaluation assesses the first year of the programme’s implementation and is aimed to be used to inform decision-making in the second year. In collaboration with Oxford Policy Management, Development Analytics was involved in organising fieldwork, carrying out focus group discussions and key informant interviews, and conducting qualitative data analysis.
May 2017 - April 2018
Myanmar Dry Zone Social Protection Project Endline Assessment
Poverty, Cash Transfers
“Dry Zone Social Protection Project” has been implemented by HelpAge International in collaboration with the Mandalay YMCA to help vulnerable households in the Dry Zone to cope and manage risks. This three-year project had the objective to expand social protection by enhancing informal community-based mechanisms and practices; strengthening government and community capacity to protect the poor; and delivering cash benefits to vulnerable groups (people with disabilities and older people). The project ended in December 2018. The locations of the project were a total of 180 villages with 30 villages in each of six target townships of two regions of Myanmar. In addition to the two different baseline surveys collected at the beginning of the project, two end line surveys were collected to measure the change in vulnerability and social protection over the course of the project and also measure the impact of the cash transfers. In collaboration with ODI, Development Analytics assisted with the end line assessment's study design, data analysis, and reporting on the findings.
May 2018 - December 2018
Evaluation of Women's Human Rights Trainings
Gender, Women’s Empowerment, Human Rights
In the scope of the project “More and Better Jobs for Women: Women’s Empowerment through Decent Work in Turkey”, an Implementation Agreement with Women for Women’s Human Rights-New Ways (WWHR) was signed to deliver a women’s human rights training for 2000 women and human rights seminars for 1250 men and women in project provinces of Ankara, Bursa, Konya, and İstanbul. Women’s human rights training is provided to women participating in the vocational training courses offered by the municipalities in project provinces in line with the work-plan of the project. The monitoring and evaluation of the training aimed to measure the effectiveness, quality of the training in various aspects, as well as to receive recommendations on how to improve the training for the second phase, as planned to take place in September 2016 until the end of December 2016.
The purpose of the evaluation is to:
Determine if the training has made progress towards its stated objectives and outcomes and explain why/why not;
Provide recommendations on programme improvement and further action for the second phase of training between September-December 2016.
Where necessary, identify the possible needs to refine content, duration, methodology, trainer approach
📖 Download the handbook titled "Impact Evaluation Methods in Evaluating Development Programmes and Applications from Turkey" here.
May 2016 - September 2016
Final Evaluation of the Pilot Implementation of SCORE Methodology in Turkey
Sustaining Competitive and Responsible Enterprises (SCORE) is a practical training and in-factory counselling programme to increase the productivity of small to medium-sized enterprises (SMEs) while promoting respect for workers’ rights. The programme demonstrates the best international practice in the manufacturing and service sectors and helps SMEs to participate in global supply chains. The intervention in Turkey targeted SMEs in Turkey in a selected province; Kocaeli to improve the competitiveness and productivity levels of the selected SMEs through strengthening the workplace cooperation and working conditions to ensure scaling up SCORE programme by the relevant government agencies and industry organizations to enable a larger number of SMEs to benefit from the SCORE in Turkey. The final evaluation of the project aimed to assess the results of the work done in order to properly report on the results as well as define the steps for possible further project development in the improvement of working conditions in SMEs and SME efficiency and productivity. The evaluation applied the key criteria of relevance, effectiveness, efficiency, sustainability, and impact potential. To this end, a fieldwork composing of in-depth interviews had been conducted, and original data collected through i) these qualitative interviews, and ii) quantitative surveys from the programme participants.
December 2015 - April 2016
Final Global Evaluation of the Decent Work in the Green Economy Project
The ILO project “Decent Work in the Green Economy” has focused on the employment aspects of national efforts to transition to a green economy. The project was implemented in 3 countries Mexico, China, and Turkey. The global evaluation report was undertaken in this project after carrying out the local interviews in Turkey and receiving local country reports for Mexico and China. The report followed the guidelines indicated in the TOR and included: (i) the project’s relevance and effectiveness; project achievements against immediate objective and expected outputs; (ii) emerging outcomes and impacts of the interventions; (iii) lessons learned and recommendations for future projects of similar focus.
March 2015 - July 2015
Final Evaluation Report for "A Fair Start for Young Children in Turkish Rural Communities" Project
Early Childhood Care and Education, Gender and Women's Empowerment
“A Fair Start for Young Children in Turkish Rural Communities” project was conducted by the Mother-Child Education Foundation (AÇEV) in Tokat during 2012-2015. The purpose of the project was two-fold: (i) to improve the physical, mental, social, and emotional development of young children in rural Turkey by engaging them in an early childhood education program, and (ii) to increase community awareness in early childhood education. The mixed-methods evaluation of the project utilized quantitative data that was collected from treatment and control villages in the baseline and post-test surveys, qualitative data from 16 focus groups with beneficiary and non-beneficiary mothers and fathers in the program area, and in-depth interviews and a short quantitative survey of teachers in the final phase of the program.
November 2014-August 2015
Impact Evaluation of the Hüsnü M. Özyeğin Rural Development Program in Eastern Turkey
Rural Development, Impact Evaluation, Aid Effectiveness
The Özyeğin Rural Development Program was launched in January 2009 in project villages in Bitlis with the aim of reducing rural poverty by investing in improving the returns to the assets of the poor: mainly by means of investments in agricultural technology and human development. This final evaluation report considers the impact of the Özyeğin Rural Development Program by looking particularly at changes in economic and human development-related outcome indicators five years after the launch of the program. A household survey was collected from treatment and control villages just before the start of the programme in 2009 and and later in 2014. The survey instrument includes modules on agricultural activities, migration, household assets, income & consumption, time use, social networks, access to education and health services, and women’s empowerment. The fieldwork was funded by the Hüsnü M. Özyeğin Foundation while Development Analytics was responsible for the design of the impact evaluation study, the design of the survey instrument, training of enumerators in the field, training of data entry staff, econometric analysis, and preparing the final report.
December 2008 - March 2014
Impact Evaluation of UN-Women Father Training for Violence-Free Families Project
The project evaluated the impact of a 3-year project funded by the UN- Women Trust Fund for Reducing Gender-Based Violence and implemented in Turkey by the Mother-Child Education Foundation (AÇEV). The Father Training for Violence-Free Families Project (FTVFFP) aims to prevent violence against women and girls (VAW/G) by engaging men in a comprehensive and community-based violence prevention program. Throughout the three-year project, fathers and their wives were trained in a program that aims to foster democratic, anti-violent, and gender-sensitive attitudes and behaviours within the family. The impact evaluation study used a mixed-methods methodology using both quantitative surveys and qualitative focus groups and key informant interviews.
The evaluation aimed (i) to measure improvements and changes in outcomes as a result of the trainings, in terms of the fathers’ attitudes towards their children and wives, and the mothers’ awareness of violence and the existing legal structure in Turkey for protecting against VAW/G; (ii) to describe the external and contextual factors that may have also been at play and influenced the factors measured during the evaluation; (iii) to identify the strengths, weaknesses, challenges, and trends in the project activities that have implications for strengthening its future administrative, programmatic and strategic directions.
January 2014 - March 2014
Socioeconomic Impact of Mining on Local Communities in Africa
Welfare and Poverty Measurement, Africa
This analysis focuses on finding the socioeconomic impact of mining on local communities in 2 case countries in Africa: Tanzania and Mali. More specifically, the impact of the extractive activity on households’ welfare and children’s nutrition and education indicators was examined. Data used in the study are coming from various sources such as Demographic Health Surveys (DHS), Household Budget Surveys (HBS), and Censuses of Population of the two countries. Information on the location, opening date, and actual production of mines in Tanzania and Mali were also used. According to the data provided by the World Bank on the operation of mines in these two countries, the first mine started operating in Tanzania in 1998 and in Mali in 1997. The study follows a diffs-in-diffs methodology looking at outcome variables before the setting up of the mines and about a decade after the mines have been in existence. In order to have a good understanding of the impact of gold extraction on the outcomes different specification methods were employed throughout the analysis regarding the choice of treatment units, the treatment variable, and control units.
March 2014 - June 2014
Tools to Assess Value for Money in Agriculture: Agriculture Sector Dialogue
Impact Evaluation, Agriculture, Capacity Building
This project involved the preparation of a key-note paper and case studies describing ex-ante and ex-post quantitative evaluation methodologies in agriculture in Africa. The paper reviewed a list of about 30 studies using ex-ante and ex-post impact evaluations that have been conducted in Africa in the agricultural sector and selected studies that highlighted applications of each methodology to African case studies. In addition, two in-depth case studies were prepared on Mali and Malawi's agricultural sector program impact evaluations. The case studies included interactive sessions providing the context of the policy to the policymakers, and asking them questions in each step to solve pieces of the puzzle, thus imparting in them the necessary tools for discussion and selection among various methodologies and techniques. The keynote paper and case studies were presented in Johannesburg at the CABRI Tools to Assess Value for Money in Agriculture: Agriculture Sector Dialogue on April 14-15, 2014. The two-day workshop was attended by delegates from the Ministries of Finance and Agriculture from 13 CABRI member African countries.
October 2013 - June 2014