Edited Transcript of Interview with John Young

ODE TALKS PODCAST

EDITED TRANSCRIPT OF INTERVIEW WITH JOHN YOUNG

Marsali Mackinnon (Office of Development Effectiveness):

Hello and welcome to ODE Talks. I’m Marsali Mackinnon.

Today I’m talking with John Young about how the latest international development policy research can make a real impact on the effectiveness of aid projects.

John’s the Head of the RAPID Program for the London-based Overseas Development Institute or ODI. He’s a strong advocate for greater use of research- based evidence in development policy and practice.

His earlier work with the Institute covered areas such as decentralisation and rural services, information and information systems, and strengthening southern research capacity.

Previously, John spent years in Indonesia working for the UK Government’s overseas aid agency, DFID. He was also Kenya Country Director for the Intermediate Technology Development Group, now called Practical Action.

Australian aid supports several programs in ODI, including RAPID.

John, welcome to ODE Talks.

John Young:

Thank you.

Marsali Mackinnon:

I would like to start with a rather simple question for this very complex area. Aid project managers are busy people, why should they care about keeping up with international development policy research?

John Young:

Okay. Well I mean I think aid project managers in AusAID and everywhere they’re professionals and you would expect professionals to keep up to date with the field they are working in. You would always hope, for example, that your doctor is up to date with the latest thinking in medicine and preventative medicine, and I think many of the sort of new initiatives in international development come out of the research arena – it often takes a long time for them to evolve, but some of the sort of major evolutions around, for example, sustainable livelihoods or around country ownership, the Paris Accord, the Poverty Reduction Strategy Papers, the increasing recognition of the importance of politics and political analysis in developing aid programs – these have all emerged out of research and it’s important for AusAID project managers to be aware of these. The problem of course is that when issues first start emerging in the research landscape they’re very often highly contested; there is a period in which there are opposing views; eventually a consensus is reached and it’s only after that time that clear recommendations start to emerge which are practically useful. But I think keeping an oversight of what’s bubbling up, if you like, through the research arena, and how it is settling out I think can really help to strengthen programs.

Marsali Mackinnon:

Well you’ve been working with ODI RAPID looking at how a number of overseas aid organisations use research. I think most recently you’ve been examining the UK Government development agency DFID’s research usage. Can you tell us a little bit more about that?

John Young:

I think I would like to start off by saying more generally, I think the key lesson which has emerged from all of our work looking at the many different organisations is the importance of integrating research with policy and practice. And what you tend to see happening in many agencies over the years is that the research department or whatever it’s called in different agencies often starts off as being a function within programsand then it very often becomes centralised and the advantage of doing that is it gets more critical mass, you can get more people working on the issues and it can get more credibility as an activity in its own right, but then very often what you find happens is that it starts to become detached from programs and practice and so it gets recentralised again. And if you look in DFID for example, over the last 15 years, you can see how at one stage it was embedded within programs, then it was established as a central research department working separately from the policy division and now it’s been reintegrated again into the research and evidence division, and they’ve introduced people and function specifically to try and make sure that the results of research are embedded in policy and practice.

So they’ve now got knowledge brokers and they’ve got research fellows based in research and evidence division to make sure that the value of research is internalised and is used within DFID, and you tend to see that in all organisations that use research.

And I would just like to give a couple of practical examples where I think it is being particularly well used:

One is the Swiss Development Corporation’s use of evaluation evidence, and they have a very well developed system to identify the evaluations they are going to do each year. They commissioned an independent external evaluator, but that evaluator then works very closely with an internal committee within SDC throughout the course of the evaluation, and also that provides a very good sounding board for the conclusions of the evaluation and then SDC has to make a management response to the results of the evaluation, and again, it’s the role of this evaluation committee in SDC to check whether the commitments made in the management response are actually implemented within the organisation. So there’s a very high chance that the recommendations and evaluations will be taken seriously and will be acted upon, unlike in some other organisations that regard evaluations as a kind of tick box exercise at the end of the evaluation, and it’s done and it goes onto a shelf and is completely ignored. So I think that is quite a good model of how research based evidence based on evaluations can be used effectively in an organisation.

And the other example I would like to give from DFID is the increasing collaboration between DFID and the research councils in the UK in order to try and guarantee the academic quality of the research which they fund. The particular program I would like to talk about is DFID ESRC – that is the Economic and Social Research Council Growth Research Program. And in that program they have put out a call for research proposals across three themes within that area. One is on agriculture; one is on innovation and one is on financial systems, to try and commission the best research in the world really on these issues. Then running alongside this kind of research commissioning process which is being run by the ESRC is a research uptake support team which is actually being run within ODI which will work closely with the researchers who are doing the research to encourage them to think about how can they do the research in a way which is likely to maximise its value, so to engage with policy-makers early in the research to identify the issues which policy-makers are worried about and are looking for questions for, and to share the results as they emerge through the research program with them, and also to look at the research which is emerging across the program as a whole and think about, well are there other fora into which these results could be presented and other ways in which the results could be synthesised, and communicated which would add value to the work that the individual researchers can do in their own right? So that’s I think quite a good example of a way in which DFID is trying to both guarantee the academic credibility and the rigour of its research, but also to make sure that useful research is done in a useful way, and to maximise the chance that it is used.

Marsali Mackinnon:

Indeed. I guess maybe this is a variation on the last question and your answers, but looking at the most effective ways that aid agencies and NGOs can and should use research, can you think of some, even one or two major current or recent in the field examples of where aid organisations have successfully applied research findings to planning and implementing development initiatives, for example, in our Asia-Pacific Region or in Africa, relating in other words what you’ve just been talking about to what happens on the ground, and you know all about that because you’ve been out there and done that too.

John Young:

Okay. Let me give an example actually from AusAID itself, which I’m working on quite closely anyway, and it’s the Knowledge Sector Initiative in Indonesia, which is an initiative which seeks to improve the generation and use of research-based evidence within Indonesia. And it’s working, it’s a 15 year program which is astonishing recognising that this kind of change takes time – there is quite a large investment, and its deliberately working across the whole spectrum of actors and processes that are involved in this, so working on the supply side with organisations producing research; on the demand side with organisations using it, and making policy and practical and program decisions, the various intermediary groups necessary to translate the research into useful findings and also looking at the enabling environment which facilitates or in some cases prevents this from happening effectively. And I think that’s a very good example of how AusAID has actually capitalised on a very strong body of global evidence, if you like, about the factors which contribute to innovation and growth in countries, factors which contribute or how education can contribute to that, and the balance between primary, secondary and tertiary education, global knowledge about policy processes and how they work and also about how research-based evidence gets into policy and practice.

So there is this global research-based evidence which has informed a decision to set up this program and then the AusAID team in Indonesia over the course of about three years commissioned I think, fourteen specific design studies looking at different dimensions of this, which are fed through into the design of a program which is in itself, deliberately a learning program. And two dimensions to that learning, well three dimensions to the learning I suppose: one is within the program itself, have a strong monitoring and evaluation component in order to check that it’s on course and take corrective action if it’s not; also to generate lessons for AusAID about how interventions in the knowledge sector can work in Indonesia and whether they could also work elsewhere in the region, and I know AusAID is looking at similar initiatives, I think, in the Philippines and also the Pacific; but also learning about how strategic interventions in emerging economies can catalyse change. So I think there’s lots of lessons that AusAID can learn from this project and also I think the lessons which will emerge will contribute to this global knowledge again, so the sort of cycle is completed. So I think that’s actually a really good example of how research-based evidence can contribute to identifying and designing a project which will in itself also, if it’s implemented with a strong sort of learning framework, will contribute to research-based evidence globally as well.

Marsali Mackinnon:

Well you’ve mentioned AusAID and of course you’re about to start examining AusAID’s research uptake and usage. I don’t know whether you’re in a position yet to tell us a little bit about how you plan to approach this evaluation, but I thought I’d ask you anyway.

John Young:

Okay. Well the methodology is still being developed, in fact I’ve just come from a meeting with the ODE Team about the methodology to try and make sure that the methodology does really answer the questions, and I suppose there are three main questions. The first question is what research is AusAID commissioning and using? And that is a very simple what question. And then there’s a question around, why is it being used differently in different places and what are the various factors that contribute to that? And then the third set of questions are about, okay what can be done? What have we learnt about that? What recommendations will it be possible to make in order to try and help AusAID to make better use of research-based evidence in its policies and its programs? How can AusAID be a better learning organisation, if you like? So that is the sort of overall direction of the evaluation and we’re going to be doing several things. One is we’re going to be looking at the research that AusAID commissions and try to understand what is produced; what does that investment produce in terms of products? How are those products being used both within AusAID and outside AusAID? What are the kinds of features in the way in which it’s commissioned and used which shape how well and how much it’s commissioned and used and what value it is to AusAID?

Marsali Mackinnon:

That sounds very valuable, I’m sure, and thank you very much for your time today John.

John Young:

Well thank you very much.

Marsali Mackinnon (outro):

You’ve been listening to ODE Talks, produced by the Office of Development Effectiveness at the Department of Foreign Affairs and Trade. For more information go to ode.dfat.gov.au

ODE Talks Podcast: John Young– Recorded September 20131