Engaging research with policy and practice
To exactly what extent evidence improves the efficacy of policy and practice remains highly contested. However, a consensus does exist amongst governments, donors and research funders, practitioners and researchers around the important contribution research can make to strengthen the evidence base that informs decision-making around development policy and practice (Newman et al. 2013; Levitt 2013).
In defining the impact they are seeking to achieve, research programmes often identify specific changes they would like to see in the behaviours and attitudes of policymakers and practitioners, as well as the policies and practices that those stakeholders are responsible for. Achieving these changes relies on effective engagement with these audiences, and there are multiple resources that provide advice and guidance on how this can be achieved (Young et al. 2014).
This Learning Guide draws on lessons from fourresearch projects funded by the ESRC-DFID's Joint Fund for Poverty Alleviation Research:
Healthy Urbanisation: Tackling child malnutrition through intervening to change the social determinants of health in informal settlements and slums, (2010-2014, Principal Investigator: Professor Pat Judith Pridmore, Institute of Education). The project focused on the issues of child obesity in Chile and child undernutrition in Kenya. During its four-year duration, the study facilitated participatory action research (PAR) to find the most effective ways to change knowledge, attitudes and practices relating to the social determinants of child malnutrition at municipal and community level. This was done by bringing together practitioners from institutions and sectors that are important for tackling malnutrition, and community members from the informal settlements on which the study focused to form Urban Nutrition Working Groups (UNWGs) – one in Mombasa and one in Valparaíso. (Pridmore et al. 2014)
Embedding Poor People’s Voices in Local Governance: participation and political empowerment in India, (2008-2010, Principal Investigator: Dr Glyn Owain Williams, University of Sheffield). The project looked at three related aspects of political empowerment (poor people's political capabilities, their political space, and their substantive citizenship) in four selected locales in West Bengal and Kerala. (Williams et al. 2011)
Pregnancy termination trajectories in Zambia: the socio-economic costs, (2011-2014, Principal Investigator: Dr Ernestina Coast, London School of Ecnoomics, LSE). The project ‘Pregnancy termination trajectories in Zambia’ was run by a team of researchers from the London School of Economics and Political Science, Health and Social Care (LSE), King’s College London and the Lusaka University Teaching Hospital. In addition to the research funding they were also awarded an impact maximisation grant (the maximisation grant was follow on funding designed to enable researchers to respond to emerging opportunities for knowledge exchange and research impact) to extend their dissemination and engagement work. In this learning guide we examine the impact this project achieved in terms of influence on policy and practice in more detail – see Spotlight section.
Mining, Social Networks and Rural Livelihoods in Bangladesh, (2008-2011, Principal Investigator: Professor Katy Gardner, London School of Economics, LSE). The research aimed to generate policy recommendations, tools and ‘best practice’ guides, aiming to reduce the negative impacts of mining and displacement on affected populations, as well as providing insights into practices of CSR. A multinational energy cooperation was a particular focus for the research due to their CSR programmes in Bangladesh in which they invest in health, education and alternative livelihoods projects as part of their Community Engagement Programme. (Gardner 2011).
Incentives and drivers of positive engagement of research with policy and practice
- Integrated targeted engagement activities into the research design, which are undertaken regularly and continuously from the outset of the project.
- A sound understanding of the policy environment, target audiences and who is best placed to engage with them, and how.
- The ability to respond to rapidly emerging policy and practice windows and opportunities.
- Existing networks that can be built upon and leveraged for engagement activities.
- Broader organisational capacity and support to underpin research engagement activities.
Policy makers and practitioners, alongside other users and beneficiaries, have:
- Been involved in a collaborative way in the research process from the outset.
- The capacity and capability to understand and utilise evidence in decision-making, or this is built as part of the research process.
- Existing appetite for policy or practice change.
- Limited funding available for effective engagement by researchers.
- Policy makers and/or practitioners do not have adequate resources to effectively engage with research, particularly single projects, and understand the benefits and implications for their own work.
- Unclear or too broadly defined policy engagement and impact goals, with limited understanding of what success looks like.
- Unexpected events and changes to the policy environment.
- Parallel or separate research and research uptake tracks within a research project.
- Diverging and competing interests of target audiences.
- Over reliance on existing networks and connections.
Top tips for researchers
Be clear about your impact goals and how you will achieve them from the outset
Having clearly defined impact goals and pathways to achieving them is critical to effective engagement, even if these need to be refined, revised and adapted as the programme progresses. This was highlighted in the impact evaluation report which assesses the impact of the first two phases of the Joint Fund for Poverty Alleviation:
'…it was found that 71% of impactful projects had good or excellent clarity to their intended impacts, 70% had a good or excellent understanding of how their impact would be achieved, and 78% had a good or excellent impact plan…'
The project, led by Professor Ernestina Coast at the London School of Economics and Political Science (LSE), on unsafe abortion in Zambia had a clear set of impact objectives from the outset for the programme. These included building the capacity of the research team, contributing to knowledge around why women seek unsafe abortions and the cost of them, influencing debates around safe abortion services, providing evidence for advocacy NGOs to use in lobbying and encouraging evidence-based decision making around abortion by policy makers.
Identifying a clear set of objectives allowed the project to be specific about the best tools and tactics to employ for engagement such as meetings, press releases and policy briefings. It also positioned them well to get the most out of their impact maximisation grant (this was a grant, or follow on funding, designed to enable researchers to respond to emerging opportunities for knowledge exchange and research impact), planning specific engagement activities with individuals and organisations in a much more detailed and targeted way.
Involve those who are at the heart of the change you wish to see from the outset.
Targeting the key individuals, or groups of individuals, who are instrumental for change, and ensuring they are a part of the process from the start, and throughout, will help to support medium to longer term research impact.
'It can be seen that the highest level of impact achievement was found in projects that identified and directly engaged with individuals or communities that were the subject of the research (i.e. the ‘ultimate beneficiaries of the research’).'Example:Healthy Urbanisation: Tackling child malnutrition through intervening to change the social determinants of health in informal settlements and slums
The project identified one of their target audiences as the community members that the research would directly benefit. They involved the Chief of Chaani and nursery school teachers in Valparaiso in the Urban Nutrition Working Groups (UNWGs), which helped ensure the relevance and feasibility of the planned interventions.
Interaction with the communities through involvement in the UNWGs and capacity building and training initiatives led to a number of positive outcomes. They built the confidence and awareness of the communities around improved nutrition and also established stronger links between the communities and other organisations and the government. Relationships that would allow for ongoing engagement and demand for change to policy and practice by the community that could be sustained beyond the life of the project.
As highlighted by the evaluation report, in Kenya “…community members were still reporting a significant improvement in the health and nutritional status of their children, of their income levels and their social status within the community as a result of their role in the […] project.”
Through the UNWGs the project also brought different ministries together, building on earlier initiatives to foster cross governmental working.
'The coordination of the different ministries is very exciting at our level as officers. We feel it is something very exciting to have a group of different ministries working together to achieve the same goal.'
(Kenyan government employee)
Collaborate and work in partnership
Researchers working on Joint Fund projects regularly cited a lack of knowledge around research engagement and/or policy influencing (and time to undertake these types of activities) as a significant barrier to achieving impact.
Collaborating and working in partnership not only helps shape and target engagement activities more effectively, it also offers a way in which to extend the sometimes limited resources and capabilities of researchers.
In fact, research partners may be better placed to lead on certain engagement activities such as media or social media engagement (which the Joint Fund impact evaluation report noted was under-used by all the research projects). Or where they have existing relationships or established credibility and reputation with target audiences.Example:Embedding poor people's voices in local governance: participation and political empowerment in India
Researchers leading a project on participation and political empowerment in India worked with a partner institution, the Kerala Centre for Development Studies, to host a series of events to engage with key stakeholders. One important stakeholder, the Principal Secretary in charge of local self-governance for the Government of Kerala, identified the involvement of the Centre and their perceived strong reputation as a key success factor.
Successful working with partners to strengthen engagement was also evident in the Zambian research project on unsafe abortion. In this project, a significant part of the communications and uptake work was led by the in-country partner who was a medical expert in the field of study who was well plugged into relevant networks and policy circles. His background and connections established a level of credibility and reputation which meant he was well positioned to engage with stakeholders such as the Zambian Medical Association, the Zambian Association of Obstetricians and Gynaecologists and the Resident Doctors Association of Zambia, which the Deputy Minister for Health attending and subsequently led the in-country partner to present at a Senior Management meeting at the Ministry of Health.
The Zambian project was also unique in the way that from the outset the researchers consulted with stakeholders around how they could help them to make use of the findings – in terms of tailoring messages and the format and presentation of the findings. The lines of communications between researchers and stakeholders were kept open and maintained throughout the project and as the impact evaluation report outlines, the “…combination of ongoing communication and flexible collaboration proved to be very successful.”
Design engagement into your research
A number of the projects referred to a lack of time and funding to undertake research uptake and engagement activities. Or that it was hard to conceive pathways to impact and related engagement activities at the beginning of the research process before researchers had a clearer idea about the findings.
However, it was evident from a number of other projects that where plans around engagement had been integrated into the research design, rather than undertaken as part of a parallel or separate track, this yielded positive results.Example:Healthy Urbanisation: Tackling child malnutrition through intervening to change the social determinants of health in informal settlements and slums
By creating the UNWGs, the project created an effective research engagement mechanism which allowed them to build relationships and networks and tailor their approaches and messages appropriately to the local context. Researchers in the project also identified an existing appetite amongst policy makers and practitioners for change in Kenya and Chile, which provided a convincing rationale for why the countries were selected as focus countries.
Researchers working on the Zambian abortion project involved the in-country partner, a medical practitioner with strong networks and understanding of the political and policy environment, closely in the research design. Again this resulted in strong levels of engagement with the research by policy makers and practitioners.
Understand the policy environment but expect the unexpected
As the Joint Fund research projects overwhelmingly demonstrated, effective engagement with research and practice needs to be shaped by an understanding of the local context. The projects also highlighted how keeping on top of often rapidly changing environments or unexpected changes can prove challenging. Yet it is often these rapidly emerging issues or changes that can offer ‘windows of opportunity’ for engagement, and the researchers’ ability to adapt and respond these is critical.Example:Embedding poor people's voices in local governance: participation and political empowerment in India
In India there is generally a five year planning process, where multi-disciplinary task forces comprising of academics, officials and civil society are established to draw together findings over the last five year period and propose recommendations for the next five year period. This process was key to researchers in India looking at political participation and empowerment as it offered a clear opportunity for research to shape policy.
However these known quantities regarding the policy and political environment are offset by the unknown, which can offer both opportunities and risks for research programmes to influence and shape the change they are seeking to achieve. A researcher in the Indian-based research project was approached by a senior member of an opposition political party who was likely to be returned to government in the forthcoming State Assembly elections. The politician requested the researcher to provide him with a paper outlining the challenges faced by the current government’s anti-poverty programme. This obviously represented a significant influencing opportunity, although there were risks attached to how the opposition might use the findings to shape the political agenda which it was important for the research team to consider in framing their messages in the paper.
In the case of the Zambian-based research project looking at unsafe abortion, the team highlighted the ‘snow-balling effect’ whereby often one opportunity would lead to another. There uptake and influencing activities were also shaped by the unexpected death of the Zambian President.
'The death of the president in September 2014 led to a period of political uncertainty and made it hard to find interested stakeholders in the ministry. Newspapers focused on political news and there was no space for the research findings to be mentioned in the media.'
A more positive unforeseen opportunity arose when the research team had the chance to influence debates in the first Youth Parliament Zambia.
Researchers’ ability to capitalise on these ‘windows of opportunities’ and become ‘policy entrepreneurs’ is vital. As is their ability to adapt and respond to politically volatile and sensitive contexts characterised by the diverging interests of stakeholders.
This was certainly a situation that researchers in Bangladesh looking at mining and rural livelihoods found themselves in, where a main stakeholder was perceived not to be fully committed to dialogue and there was a question of trust by other stakeholders that the programme was attempting to engage with. As a researcher highlighted:
'We were continually assessing what is working and what is not working. We were doing fieldwork in villages and we had to follow our noses. I was continually shaping the research according to the circumstances.'
Top tips for donors
Incentivise focus on impact through funding
A number of the joint fund projects cited lack of time and resources to effectively undertake engagement and impact work as a limiting factor to achieving impact.Example:Healthy Urbanisation: Tackling child malnutrition through intervening to change the social determinants of health in informal settlements and slums
The project highlighted difficulties arising from a lack of understanding amongst key stakeholders and partners both around the concept of impact and also in relation to the benefits of the research to their own work, lives and lives of other potential beneficiaries of the research. Researchers cited a lack of time to engage and develop this understanding as a key barrier.
Targeted funding for research impact activities is one way that these types of barriers can be overcome, as demonstrated by the impact maximisation grant that the Zambian project looking at unsafe abortion trajectories received. The grant was follow on funding designed to enable researchers to respond to emerging opportunities for knowledge exchange and research impact.
This type of grant allows research programmes to extend the reach and impact of their work with policy and practice, beyond the life of the project. As the impact evaluation report points out “It is evident that the research has greatly informed and reinvigorated the debate around abortion in Zambia…[and]…reconceptualised unsafe abortions as a public health issue”.
'For me the impact is great. Issues of abortion in Zambia are not talked about. It’s very emotive, the topic is taboo. The research and initiation of such topics to start a discussion, it goes a long way in changing the perception of people in communities.’
(Research user, Zambia)
Promote an integrated approach to research and research uptake
The challenge of limited time and resources on the part of researchers can also be addressed, as highlighted earlier in the researcher focused recommendations, by integrating uptake and engagement activities into the research design.
Many donors and research funders request detailed pathways of impact statements or uptake strategies are included in proposal documents and provide guidance and advice in these areas. For example, the ESRC’s Impact Toolkit. However the integration of research uptake into research design could be strongly promoted in funder and donor calls, through reemphasising the value of this approach and articulate clearly in awards criteria that those proposals that demonstrate this approach will be rewarded accordingly.
Promote a culture of mutual learning
Donors and research funders can also contribute to building and strengthening a cultural of learning and sharing around impact, including their own role in this process.
A number of the UK researchers also reported that there was a low-level of understanding around research impact – how it is achieved, monitored and reported – outside of the UK. Through funding programmes such as the Impact Initiative, research funders can build awareness around impact where it is low, and also create new opportunities to strengthen potential long-term and scalable impact by establishing new networks of research projects and knowledge.
In the Joint Fund impact evaluation report, ECORYS and IDS proposed that ESRC and DFID should consider developing a range of possible outcome indicators that research programmes could use to monitor their impact. They also recommended that ESRC and DFID require researchers to report on impact annually over the course of a project. Tools such as Researchfish (a web-based tool used by researchers and research funders to track impact) could aid this process and the appointment of a monitoring, evaluation, learning consultant could help ensure more effective measurement of research impact.Example:Embedding poor people's voices in local governance: participation and political empowerment in India
For researchers working in India on the project on political empowerment and participation clearer guidance around how best to work with DIFD country offices would have been helpful as they struggled to engage effectively with the DFID country office. Researchers also have a role to play in building their own contextual knowledge of stakeholders, but insights on organisational structure and processes from donors and funders, as well as introduction to relevant personnel, would be invaluable.
Capacity building improves potential long-term and sustainable impact
Building on the point in earlier recommendations around low awareness around impact and capacity and capability to utilise and engage with research findings, donors and funders should continue to fund capacity building initiatives focused on strengthening the demand for, and use of, research.
These types of capacity building activities can have a very positive effect on long term research impact.
Researchers in Zambia combined engagement activities with capacity activities including training reproductive and sexual health practitioners on the legal framework around providing abortion services and interacting and communication with media. They also trained journalists, presenters and media producers on some of the key around Sexual Reproductive Health and abortion which helped shaped future content and public debates.
The project on abortion services in Zambia led by Professor Ernestina Coast at the London School of Economics (LSE) was successful in achieving a number of significant impacts.
- Capacity Building: It developed the skills of researchers working on the project in qualitative data analysis, research methodology and research ethics as well as building the capacity of the medical research partner to measure and report research impact. It worked with sexual and reproductive health (SRH) practitioners including doctors and gynaecologists to build their awareness of the legal framework around abortion and to develop their confidence in communicating with the media. In addition the project also provided training to journalists around SRH issues and abortion in Zambia and one of the recipients of this training said:
'[The training] was highly beneficial in equipping the producers to produce programmes that are of good quality and contain information that is helpful for the people.’
(Research user, Zambia)
The researchers also built links between these different stakeholders, contributing to the creation of new networks that had not existed previously.
- Conceptual Impact: The project produced the first national estimates of the health system costs of abortion in Zambia. They showed that, annually, unsafe abortions cost the Zambian health system up to US$0.4 million more than if the pregnancies had been terminated safely and legally. This addressed an existing knowledge gap and helped reconceptualise unsafe abortions as a public health issue.
'The research came in very handy especially the cost of unsafe abortions because then people were able to see unsafe abortions are a big health care problem.'
(Research user, Zambia)
The project has also led to new research collaborations with partners in the USA to develop a conceptual framework for the study of trajectories to abortion which has been positively received by other academics and NGOS working on SRH issues at a global level. The team also gained funding from LSE to do further research on abortion in rural areas in Zambia responding directly to a gap in knowledge identified by a Zambian Ministry of Health official.
- Instrumental impact: As a result of training provided by the research programme BBC Media Action in Zambia have adapted how they write stories on SRH and abortion issues placing a much greater emphasis on the experiences of real people. New collaborations with NGOS such as Marie Stopes were established and The UNICEF MDGi programme drew on the study’s recommendations in their programme design around access to safe abortion services for young people. To date there has been no measurable impact on specific laws and services however there has been much encouraging progress towards achieving this. For example the research team was invited to comment on national guidelines on reducing unsafe abortions and continues to engage with senior officials at the Zambian Ministry of Health.
The study had a number of objectives in relation to the capacity building, instrumental and conceptual impact which they were seeking to achieve and which shaped their engagement strategies. These included:
- Building the research capacity of the research team
- Contributing to the knowledge base, particularly the reasons some women seek unsafe abortion while others access safe services and the individual cost of unsafe abortion.
- Influencing the debate on provision of safe abortion services
- Empowering advocacy NGOs with more evidence to use in their lobbying
- Increasing policy makers’ understanding on issues around unsafe abortion and encourage use of evidence in their decisions.
To achieve these objectives the project identified a number of key audiences that they were keen to engage with. These included:
- National policy makers including Zambian Ministry of Health
- Service providers and clinical practitioners including doctors and gynaecologists in sexual and reproductive health (SRH)
- NGOs and advocacy networks including International Planned Parenthood Federation (IPPF), the Planned Parenthood Association of Zambia (PPAZ) and Marie Stopes Zambia.
- Journalists and those shaping men and women’s understanding of reproductive sexual health rights and services
- Women seeking to terminate a pregnancy.
The project employed a variety of engagement activities:
- Meetings with stakeholders to discuss the findings of the research and how they could help them make best use of them i.e. providing tailored evidence, running training sessions, speaking at events.
- Holding workshops and presentations.
- Stakeholder analysis.
- Sharing their findings in the context of a wider body of existing evidence on abortion in Zambia.
- Website and social media channels.
- Peer-reviewed journal articles.
- Presentations at academic conferences such as Oxford Institute of Population Ageing and the London School of Hygiene and Tropical Medicine.
- Presentations to in-country practitioner networks such as the Zambian Association of Obstetricians and Gynaecologists.
- Media engagement with outlets including BBC World News and Zambia National Broadcasting Corporation.
- One to one meetings with key influencers such as Deputy Minister of Health.
Opportunities, Barriers and Lessons
The selection of Zambia as the focus country was an important factor that shaped the project’s approach to engagement and impact. Abortion in Zambia is legal but numbers of deaths as a result of unsafe abortions remain high and there is a shared interested amongst groups of NGOs, policy makes and practitioners to tackle this issue. In essence the appetite for change already existed amongst key stakeholders.
SRH and abortion issues were politically, culturally and socially sensitive and the researchers reported a broad lack of awareness across medical practitioners, NGOs, the media and the public more widely. The research project sought to address this awareness gap through its capacity building initiatives and its local partnerships, including with the medical practitioner who provided invaluable contextual insights to the project and was well connected to professional networks and senior policy makers. Researchers also invested time in consulting research users about how they could assist them in using the research findings. Indeed this collaborative approach was something that the impact evaluation report of the Joint Fund identified as an aspect that set the project apart from many others in the fund.
Time and money were also critical issues for the research project in achieving impact. Winning the impact maximisation grant (which provided follow on funding to support researchers to develop a response to emerging opportunities for research impact) was key to extending the reach and engagement of the programme – allowing them to undertake more detailed stakeholder analysis which shaped further targeted engagement activities.
There is no blueprint that projects and programmes can employ to ensure that the research findings that they are generating contribute to the evidence-base that informs and shapes decisions around development policy and practice. It is dependent on the capacities and capabilities of researchers to understand and respond to the policy environment in which they operate, to leverage and build networks for engagement and to make information available and accessible. While strategies and planning are key, the ability of researchers to adapt and respond to rapidly changing contexts is essential.
Being able to achieve specific changes in policy and practice through new evidence is also dependent on the resources and abilities of policy makers and practitioners to utilise this knowledge, their alignment and interest with the desired outcomes of the research programme and the broader social, economic and political context. Donors and research funders can play a critical role to play in improving the interface between research, evidence and policy and practice through targeted funding, the promotion an engaged approach to research design and uptake and encouraging a culture of learning and sharing.
- ESRC-DFID Joint Fund for Poverty Alleviation Research,
www.esrc.ac.uk/research/international-research/international-development/esrc-dfid-joint-fund-foTr-poverty-alleviation-research/ (accessed 29 November 2016)
- France, J.; Rajania, A.; Goodman, R.; Ram, M.; Longhurst, R.; Pelka, V. and Erskine, C. (2016) Evaluating the Impact of the ESRC-DFID Joint Fund for Poverty Alleviation Research, ECORYS and IDS
- What is Impact? ESRC Impact Toolkit,
www.esrc.ac.uk/research/impact-toolkit/what-is-impact/ (accessed 29 November 2016)
- Newman, K.; Capillo, A.; Famurewa, A.; Nath, C. and Siyanbola, W. (2013) ‘What is the Evidence on Evidence-informed Policy Making? Lessons from the International Conference on Evidence-Informed Policy Making’, paper prepared for the International Conference on Evidence-Informed Policy Making, Ile-Ife, Nigeria, February 2012, published by INASP
- Levitt, R. (2013) The Challenges of Evidence: Provocation Paper for the Alliance for Useful Evidence, NESTA, London: Alliance for Useful Evidence
- Young, J.; Shaxson, L.; Jones, H.; Hearn, S.; Datta, A. and Cassidy, C. (2014) ROMA – A Guide to Policy Engagement and Policy Influence, London: Overseas Development Institute
- Healthy urbanisation: tackling child malnutrition through intervening to change the social determinants of health in informal settlements and slums
www.theimpactinitiative.net/project/healthy-urbanisation-tackling-child-malnutrition-through-intervening-change-social (accessed 29 November 2016)
- Pridmore, P. et al. (2014) NICK Project (Nutritional Improvement for Children in Urban Chile and Kenya), ESRC Full Research Report, RES-167-25-0461, Swindon: ESRC
- Embedding poor people’s voices in local governance: participation and political empowerment in India,
www.theimpactinitiative.net/project/embedding-poor-peoples-voices-local-governance-participation-and-political-empowerment-india (accessed 29 November 2016)
- Williams, G. et al. (2011) Embedding Poor People’s Voices in Local Governance: Participation and Political Empowerment in India, ESRC End of Award Report, RES-167-25-0268, Swindon: ESRC
- Pregnancy termination trajectories in Zambia: the socioeconomic costs,
www.theimpactinitiative.net/project/pregnancy-termination-trajectories-zambia-socio-economic-costs (accessed 29 November 2016)
- Pregnancy termination trajectories in Zambia: the socio-economic costs,
http://personal.lse.ac.uk/coast/ZambiaTOP.htm (accessed 29 November 2016)
- Mining, social networks and rural livelihoods in Bangladesh,
www.theimpactinitiative.net/project/mining-social-networks-and-rural-livelihoods-bangladesh (accessed 29 November 2016)
- Gardner, K. (2011) Mining, Social Networks and Rural Livelihoods in Bangladesh, ESRC End of Award Report, RES-167-25-0297, Swindon: ESRC
- Researchfish: www.researchfish.net/ (accessed 29 November 2016)