Impact study

We remain critical towards our own work and aim at keeping learning about the outcomes of the program and keep improving it.  Do we reach the set objectives for the outcomes as described in our Theory of Change? Where and how can we optimise the work and further increase our impact?  Our partner UBS Optimus Foundation was willing to fund an impact study to find evidence.  Via an international tender procedure UBS selected the KIT Royal Tropical Institute)  to implement this study.

Due to the nature of the evidence sought, it is expected that the impact study will use a quasi-experimental mix-methods design, in which findings from various methods will be triangulated to reach the highest level of synthesis and understanding of impact, cost-effectiveness and their influencing factors. The study proposes:

  1. a staggered implementation design combined with trend analysis of existing key program indicators captured by the project:
    • A Household survey on key Maternal, Neonatal and Child Health and Disability issues in at least two implementation districts at base-, mid- and end-line.
    • A yearly assessment of HMIS data[1] and Medic Mobile program data in all six districts, compared with non-intervention districts in Province 1, or a to be determined province.
  2. a qualitative study at province level and in 2 selected districts over time (base-, mid- and end-line) comprising of interviews with key informants, health and social workers and focus group discussions with community members at base-, mid- and end-line, to give insight into the barriers and drivers for sustainable implementation in existing Government, local health institutional and community structures as well as unexpected impact of the program.
  3. a cost and cost-effectiveness analysis is to be conducted to understand the important cost drivers of the program and its efficiency in terms of cost per DALYs averted. This analysis will use the same methodology employed in previous assessments to ensure comparability. Furthermore, based on this analysis recommendations can be made to further maximize impact and minimize investments of the Karuna Foundation.

[1] Pending quality and applicability of HMIS data assessment in Phase 1