Education has long been the poor cousin of internationally-funded social science research in international development settings. The Economic and Social Research Council and Department for International Development strategic partnership is playing an important role in correcting the imbalance. What impact does this research aim to achieve? And what type of impact is appropriate to expect from such research? Given the growing focus of education research within the ESRC-DFID programme, it is timely to ask ourselves these questions.
Currently 30 projects are funded under ESRC-DFID programmes supported by the Impact Initiative for International Development Research – 19 under the Joint Fund for Poverty Alleviation Research, and 11 under a dedicated education programme on raising learning outcomes. These projects cover a range of topics – with issues of teachers, quality and learning, and inequalities associated with poverty and gender standing out in particular.
Our analysis of ‘pathways to impact’ within these projects highlights some initial insights. We presented these ideas at the 60th Comparative and International Education Societies Conference in Vancouver on 6-10 March 2016, alongside presentations from three projects (at different stages of their project phase):
- Strengthening open, distance and flexible learning systems to increase education access and attainment for young people in high HIV prevalence SADC countries
- Engaging teachers in peace-building in post-conflict contexts
- The Literacy Laboratory Project (LLP) under the Northern Uganda Literacy Program
Mapping education research on to the Initiative’s wheel of impact
All the projects we looked at span a range of approaches. Mixed methods predominate, with ones led by quantitative approaches most apparent. Using the the framework adopted by the Impact Initiative) we identified clear patterns of impact across the projects, often relating to their methodological approach.
- Conceptual Impact on policy and practice: Most research aims to raise awareness based on evidence to influence policy and practice indirectly. Those projects that are primarily quantitative mainly target international and national policymakers (such as one providing guidance to policymakers on reforming the conditional cash transfer programme in Mexico). Predominantly qualitative research pays more attention to local policymakers and institutions, practitioners and children (such as one aiming to influence universities in the Western Cape, South Africa, by providing guidelines to promote social justice approaches in teaching).
- Instrumental Impact on policy and practice: Whether impact directly attributable to research is feasible or desirable was the most debated during the panel in Vancouver. One question raised was whether such expectations are too ambitious within a one- to three-year timeframe of the projects. In her analysis of impact achieved related to open, distance and flexible learning in Malawi, Dr Kate Jere noted that impact attributable to the research mainly occurred after the project had ended. The impact achieved was thanks to key researchers being embedded in the environment in which change was happening enabling them to take up opportunities through their informal networks. It also benefited from the combination of an RCT with participatory approaches, such that students and out-of-school children were engaged throughout the process of the research. Without such local engagement in the research process, impact is unlikely to be sustained (particularly in the absence of funding once the research has been completed).
The discussant of the panel, Christine Beggs of USAID, questioned whether attempting a direct link between research and policy change undermines the purpose of independent research. Others also raised questions about whether anticipating a linear relationship between research and policy impact is appropriate. Professor Yusuf Sayed reported for example that, in his research on engaging teachers in peace-building in South Africa, impact was more likely to be achieved where the research could link with key policy moments or issues receiving media attention. Such opportunities cannot be planned in advance.
If direct policy outcomes from research are expected, questions arise of staffing and resources on research projects. As Dr Rebecca Thornton noted, researchers might be good at running regressions, but not at creating infographics or framing policy briefs. For this reason, her research, with Victoria Brown, using an RCT to investigate the impact of a literacy project in Northern Uganda engages with communications experts to ensure the research findings are presented to parents and teachers in ways that best suits them.
- Networks and connectivity: Most research aims to engage with policymakers, NGOs, practitioners and others. For research that employs a predominantly quantitative approach, it seems that building such networks is more often a means to an end - supporting the framing of research questions and the process of the research. For qualitative research, the development of a network can be an end in itself.
- Capacity building for research uptake: Researchers are increasingly aware that dissemination is not the same as engagement or communication required for research uptake, but are perhaps less familiar with how to adopt effective approaches to engagement and communication. It is notable that very few research proposals pay attention to the need to strengthen the capacity of researchers in communicating research to strengthen its uptake.
So, given increased expectations on researchers to achieve impact, and limited experience of researchers to do this, do we need more capacity building in research uptake? This is perhaps an area that The Impact Initiative can help to support.
What type of impact is desirable?
For those attending the conference this week in Pretoria, South Africa, on ‘Lessons from a Decade’s Research on Poverty: Innovation, Engagement and Impact’ organised by the Impact Initiative, some further questions we mulled over in Vancouver may provide food for thought:
- Who defines impact? for example, is there a tension between expectations in the North and the South; or who defines what types of impact at the local level? (does this differ for a child in school compared with a local education official, for example)?
- Should impact be seen as an iterative process rather than a one-off outcome at the end?
- And does research need its own monitoring and evaluation framework if we are to judge properly whether research is achieving its intended impact?!
I look forward to hearing your thoughts on #impactlessons
- Log in to post comments