The Australian Journal of Public Administration, vol. 67, no. 1, pp. 1–11 doi:10.1111/j.1467-8500.2007.00564.x
RESEARCH AND EVALUATION Three Lenses of Evidence-Based Policy
Brian W. Head1 University of Queensland
This article discusses recent trends to incorporate the results of systematic research (or ‘evi- dence’) into policy development, program evaluation and program improvement. This process is consistent with the New Public Management (NPM) emphasis on efficiency and effective- ness. Analysis of evidence helps to answer the questions ‘what works? and ‘what happens if we change these settings?’ Secondly, some of the well known challenges and limitations for ‘evidence-based’ policy are outlined. Policy decisions emerge from politics, judgement and debate, rather than being deduced from empirical analysis. Policy debate and analysis involves an interplay between facts, norms and desired actions, in which ‘evidence’ is diverse and contestable. Thirdly, the article outlines a distinction between technical and negotiated approaches to problem-solving. The latter is a prominent feature of policy domains rich in ‘network’ approaches, partnering and community engagement. Networks and partnerships bring to the negotiation table a diversity of stakeholder ‘evidence’, ie, relevant information, interpretations and priorities. Finally, it is suggested that three types of evidence/perspective are especially relevant in the modern era – systematic (‘scientific’) research, program man- agement experience (‘practice’), and political judgement. What works for program clients is intrinsically connected to what works for managers and for political leaders. Thus, the prac- tical craft of policy development and adjustment involves ‘weaving’ strands of information and values as seen through the lens of these three key stakeholder groups. There is not one evidence-base but several bases. These disparate bodies of knowledge become multiple sets of evidence that inform and influence policy rather than determine it.
Key words: evidence-based policy, policy development, performance, program management
In many of the mature democracies, the recent public decision-makers have not always had ground-swell of interest in ‘evidence-based close and cordial relations; indeed, there has policy’, on the part of both government offi- been a history of mutual distrust between these cials and social researchers, represents both an sectors during the last two centuries. However, opportunity and a challenge. For public man- scientific and technical knowledge has been agers and political leaders, the opportunity is greatly prized in the evolution of the modern apparent for continuous improvement in policy state, initially because of its links to economic settings and program performance, on the basis growth and national defence, and later to ad- of rational evaluation and well-informed debate dress the aspirations for social improvement by of options. The prospect of mutual benefits for the citizens. Social sciences have been valued managers, researchers and citizens is alluring. for their contribution to understanding and in- This is the modern promise of evidence- fluencing social development and well-being. based policy improvement, albeit the attempt to Democratic decision-makers have increasingly link the social sciences and public policy has a aspired to anchor many of their social reform much older lineage in the history of progressive programs in the ‘relevant’ and ‘usable’ knowl- reform movements.2 The social sciences and edge provided by the social sciences.
C 2008 The Author Journal compilation C 2008 National Council of the Institute of Public Administration Australia 2 Three Lenses of Evidence-Based Policy March 2008
The evidence-based movement in modern aging and reinvesting in the ‘bank’ of use- public policy is thus the latest version of the ful knowledge. Large data sets are system- search for usable and relevant knowledge to atically collected. Evidence-based approaches help address and resolve problems. My argu- claim to fill important gaps in the value chain ment is that this is linked to the modern em- as data is transformed into information and us- phasis on rational problem-solving, with its fo- able knowledge. Large organisations have in- cus on accurate diagnosis and knowledge of troduced ‘knowledge management’ strategies causal linkages. It is also congruent with impor- to address the complex tasks of collection, tant modern strategic concerns with risk anal- analysis and dissemination. The management ysis and appropriate mitigation responses. In information (decision-support) systems on the the more technocratic version of the evidence- desktop of senior managers are hungry for based approach, the aspiration is to produce the information to underpin performance indica- knowledge required for fine-tuning programs tors and monitor program or business trends. and constructing guidelines and ‘tool-kits’ for Some disciplines are seen as more valuable than dealing with known problems. Hence, the cur- others for these purposes, eg, the quantitative rently famous phrase that defines much of the precision of financial accounting, cost/benefit movement – ‘what works?’ (Roberts 2005). analysis, risk auditing, and health eco- In the context of public policy, governments nomics may be more credible than the remain the major investors and users of ap- hermeneutic approaches of history and cultural plied social sciences. They do not simply re- sociology. ceive, scan and utilise research; they engage on In short, the rise and promotion of ‘evidence- many levels to influence the processes and the based’ orientations within government agen- products. Their direct methods of shaping the cies is consistent with the public sector’s applied sciences include: increased interest in efficiency and effective- ness. Evidence-based policy is believed to pro- • investment in government-funded research vide great assistance in answering some of units on specific problems; the key questions of New Public Management • managing the policy-research functions in- (NPM): side many government agencies; and • commissioning external consultants to under- • what options will ‘deliver the goods’? take specific contract research (Saunders and • how can programs be improved to get greater Walter 2005). ‘value for money’? • how can innovation and competition be ex- Governments also exercise strong indirect in- panded to drive productivity? fluence through: • how can program managers achieve spe- • determining national priority areas (eg, for cific ‘outcomes’ for clients and stakeholders allocation of competitive public funding of (rather than just ‘manage programs’)? and research); • in summary, ‘what works?’ (Davies, Nutley • providing rewards and recognition for and Smith 2000; 6 2002; Reid 2003) commercially-focussed knowledge and tech- nical forms of scientific excellence; and Context: Evidence for Addressing Complex • encouraging contestability in some policy Policy Problems arenas by diversifying their sources of advice, including think-tanks and contractors (Stone In the 1970s and 1980s the exponents of and Denham 2004); and encouraging a wider NPM managerialism used innovative analytical range of instruments to deal with policy chal- frameworks to tackle traditional problems and lenges, like market-based mechanisms and to improve program performance information de-regulatory options. in each portfolio area. During this period, the Major investment in the applied social sciences de-regulation of many policy domains and out- is part of a cycle of producing, analysing, man- sourcing of services by the state was linked to
C 2008 The Author Journal compilation C 2008 National Council of the Institute of Public Administration Australia Head 3 the key focus on program performance issues, social problems seemed to persist regardless of although this strategy posed some risk to the the massive funding directed towards their alle- steering capabilities in some public sector agen- viation. Performance information showed that cies (Pollitt and Bouckaert 2000; McLaugh- results were not being achieved. What could be lin, Osborne and Ferlie 2002). However, by done to deal better with domestic poverty; poor the 1990s there was a significant political shift educational attainment; juvenile crime and re- towards tackling difficult interlinked issues, cidivism; drug and alcohol abuse; preventable exemplified by the UK government’s champi- diseases; the appalling conditions facing many oning of evidence-based policy as a foundation indigenous communities; and the systemic dis- for joined-up government (UK Cabinet Office advantages suffered by peoples in developing 1999; Parsons 2002, 2004). This period saw an countries? increased investment in central units for policy This awakening of political interest created analysis and commissioning evidence-based opportunities for the behavioural and applied consultancy reports. Dealing more directly with social sciences to offer solutions, in the form major complex issues required taking a more of new approaches to gaining greater control comprehensive approach towards policy design over fuzzy and messy realities. Incremental and service delivery (Schorr 1988). adjustment around ‘business as usual’ would The 1990s saw the rise of policy processes no longer suffice. Solutions based on old- that were potentially less technocratic and more fashioned ideological recipes became much open to ‘network’ approaches. In essence, this less persuasive. New more integrated ap- meant that the new managerialist approaches proaches to policy interventions were warmly were often supplemented by new mechanisms welcomed in some agencies. and process loops, variously described as com- From the viewpoint of performance-based munity engagement, multi-stakeholder consul- programs and evidence-based policy, the ques- tation, and partnering across stakeholder sec- tion arose as to what kind of investment in tors (Kernaghan 1993; Kooiman 2000; Osborne data/information would be needed to generate 2000; Edwards 2003; Casey 2004; Head 2007). the necessary knowledge, both to understand Greater levels of cooperation and partnership complex problems and then to create viable so- among governments, non-governmental organ- lutions. This period saw a major increase in use isations (NGOs) and business, became asso- of new technologies for data-gathering and data ciated with a widespread rhetoric promoting analysis in order to measure the nature and ex- ‘collaboration’, ‘joined-up’ services, and mul- tent of problems, assess the current impacts of tiple ‘networks’ linking stakeholders and sec- service systems, and provide benchmarks for tors (Bruner, Kunesh and Knuth 1992; Bardach judging future performance. 1998; Head 1999; Mandell 2001; Bakvis However, some social scientists and policy 2002; Sullivan and Skelcher 2002; Reddel and analysts began to question whether the per- Woolcock 2004). sistence of complex social problems was re- Political leaders and public managers in- ally attributable largely to a lack of informa- creasingly moved to tackle complex unresolved tion, ie, ‘gaps’ in the database (Schon 1983; problems, in response to the demands and pres- Schon and Rein 1994). They suggested that sures of citizens for whom services remained obtaining more data to fill the known gaps inadequate, piecemeal or inappropriate. The would not necessarily get us onto the highway frustration felt by managers in public agencies toward good policy solutions, because much regarding the poor rate of return on major social of the policy puzzle is about reconciling dif- program investments, led them to search more ferent value perspectives. Continuing to in- widely for new approaches. These new direc- vest in building information banks for social tions were facilitated by the willingness of ma- scientists and decision-makers would still re- jor NGOs, professions and other stakeholders main important. But what kinds of information to engage in partnerships or collaborative ap- would be of most value for stakeholders and proaches to addressing major issues. Endemic decision-makers dealing with the challenges
C 2008 The Author Journal compilation C 2008 National Council of the Institute of Public Administration Australia 4 Three Lenses of Evidence-Based Policy March 2008 of complex inter-related problems with many program design, implementation, and evalua- stakeholders? tion. In short, our ideas about ‘evidence-based’ policy may change character as we move from a technical approach towards a more relational ‘Evidence’ Revisited: Types of Problems, approach. Types of Knowledge Traditionally, the ‘evidence’ base seen as the foundation for evidence-based policy is The problems addressed by policy-makers are the knowledge generated by applied research, many and varied. At one end of the spectrum, whether undertaken inside or outside of gov- problems may be seen as discrete, bounded, ernment agencies. This includes the general ev- and linked to particular sets of information and idence about broad trends and explanations of actors. In such policy arenas, a ‘technical’ ap- social and organisational phenomena, as well proach to problem-solving by relatively narrow as specific evidence generated through per- circles of actors may be dominant. This tech- formance indicators and program evaluations nocratic approach may be seen as sufficient to (Nutley, Davies and Walter 2002; Oakley et al. meet the requirements of efficiency and effec- 2005). However, my argument is that the effec- tiveness by those involved. Technical expertise tiveness (success) of policies and programs is is important in most policy areas, but its dom- not just a matter for applied social-science re- inant position in many areas is increasingly search (including program evaluation reports). contested. For example, many policy areas in As we come to a fuller appreciation of the com- which scientific and engineering expertise had plexities of modern inter-dependent problems, achieved substantial control (eg, transportation, with a corresponding broadening in the focus of energy, water supply) have now become subject policy attention, it becomes clear that there are to intense debate and uncertainty. multiple forms of policy-relevant knowledge, At the other end of the spectrum, problems that are vital to understanding the issues and may be seen as complex, inter-linked and cross- the prospects for the success of policy inter- cutting. Simple technical solutions by experts ventions. are unavailable or unworkable. In these cir- In this broader view, there is not one cumstances, a ‘negotiated’ and ‘relational’ ap- evidence-base but several bases (Pawson et al. proach to problem-solving may emerge (Innes 2003; Schorr 2003; Davies 2004). These dis- and Booher 1999; Hemmati 2002; Lewicki, parate bodies of knowledge become multiple Gray and Elliott 2003; Lovan, Murray and Shaf- sets of evidence that inform and influence pol- fer 2004). The latter approach is a prominent icy rather than determine it. In this broader feature of policy domains that are rich in ‘net- understanding of policy-relevant knowledge, work’ approaches, partnering and community the prestige and utility of ‘scientific evidence’, engagement. It is argued here that a techni- validated by the standards of scientific method- cal problem-solving approach to knowledge in ology, remains a very significant input to pol- each discrete policy area is increasingly inad- icy development. Thus, rigorous and system- equate. Policy development arrangements are atic research has great value, but needs to be experimenting with broader relational and sys- placed in a wider context. Hence, it is argued temic approaches. Networks and partnerships that effective policy – its design, implementa- bring to the negotiation table a diversity of tion, and evaluation – depends on several evi- stakeholder ‘evidence’, ie, relevant informa- dentiary bases. These are all involved, directly tion, interpretations and priorities. The argu- or indirectly, in the development and assess- ment is that addressing complex inter-linked ment of ‘good programs’ and help us to un- problems requires a strong emphasis on the so- derstand ‘effectiveness’ in a more holistic and cial relations and stakeholder perceptions in- networked policy environment. herent in policy direction and program systems. The general milieu in which government This has implications for how we think about policies and programs operate is of course problems, relevant knowledge, policy and the public sphere – of public debate, public
C 2008 The Author Journal compilation C 2008 National Council of the Institute of Public Administration Australia Head 5 opinion, civic awareness and popular culture. ial context. Policy, seen through the political This milieu both informs and responds to pub- lens, is about persuasion and support rather than lic policy, and colours the ways in which po- about objective veracity. sitions are argued and knowledge-claims are Partisanship, and bias in knowledge, are not advanced. However, evidence-based policy is solely confined to the political sphere – with not primarily concerned with how the general its characteristic polemics, debates, ideologi- political culture shapes policy directions. In a cal assertions and counter-claims. However the more particular sense, there are three impor- implications of ‘political’ know-how for the tant kinds of knowledge (and corresponding use of ‘evidence’ are very significant. Most views of ‘evidence’) that are especially salient simply, a selection of convenient ‘facts’ may for policy. These forms of knowledge arise be harnessed to an argument; and large areas from: of other information are then either ignored, dismissed as tainted, or otherwise deemed ir- • political know-how; relevant. This partisan usage of evidence is of- • rigorous scientific and technical analysis; and ten regarded as ‘typical’ political behaviour and • practical and professional field experience. part of the ‘game’ of political argument. In They provide three ‘lenses’ for policy anal- the political game, it is widely understood that ysis and three lenses for understanding special pleading and deception are normalised. the evidence-base(s) of policy debate (see Sometimes the partisan use of evidence is tac- Figure 1). They all work in their different ways tical, casual or opportunistic; but sometimes it with particular interpretations of the constraints is more systematically linked to a cohesive ide- and limitations of public opinion. ological outlook, characterised by some com- mentators as faith-based politics. Importantly for my argument, there are some Three Lenses areas of policy that become the subject-matter of clear government commitments. These com- 1. Political knowledge for this purpose is the mitments are no longer (if they ever were) open know-how, analysis and judgement of politi- to further debates about the nature of the prob- cal actors. These analysing and judging activi- lem, the best policy solution, and the range ties include several vital elements relevant to of evidence relevant to assessing policy effec- evidence-based policy – such as considering tiveness. This means some policy positions are and adjusting strategies or tactics; undertaking ‘data-proof’ or ‘evidence-proof’, in the sense agenda-setting; determining priorities; under- that their evidence ‘base’ has been narrowed taking persuasion and advocacy; communicat- and buttressed by political commitments, per- ing key messages and ideological spin; shap- haps closely linked to the values and ideological ing and responding to issues of accountability; positions of political leaders or parties. Some building coalitions of support; and of course policy preferences allow only certain kinds of negotiating trade-offs and compromises. Mak- ‘evidence’ to be noticed. Critical commentary ing contextual judgements about the possible under these circumstances is unwelcome. Some and the desirable are inherent in this form of contentious problems may become defined in knowledge. ‘official’ terms, in ways that tend to privilege This ‘political’ form of knowledge inheres some evidence as relevant and to rule out other primarily in politicians, parties, organised evidence as irrelevant or merely ideological. In groups, and the public affairs media. But al- this context, the ‘official’ framing of a prob- though some of the knowledge is private and lem is also crucial in regard to what research is esoteric, most of it is also widely dispersed commissioned and its terms of reference. Rela- in popular forms among the public and espe- tively few research and consultancy projects are cially by and through the mass media. This commissioned without some expectation that knowledge is diffuse, highly fluid, and heavily the reports may assist in upholding a certain contested owing to its partisan and adversar- viewpoint.
C 2008 The Author Journal compilation C 2008 National Council of the Institute of Public Administration Australia 6 Three Lenses of Evidence-Based Policy March 2008
Figure 1. Three Lenses of Knowledge and Evidence – entails the use of randomised controlled trials (RCTs) to test the efficacy of specific interven- tions (Cochrane Collaboration website). This approach, often championed as the most rigor- ous strategy for assessing ‘what works’ in a spe- cific policy field, has also been promoted and applied in some social program areas, includ- ing criminology (Davies 2004; Petticrew and Roberts 2005; Campbell Collaboration web- site). Rigour is thus sometimes associated with a preference for quantitative behavioural data, although qualitative (attitudinal) data are in- creasingly seen as central in helping to explain 2. Scientific (research-based) knowledge,is the conditions and nature of behavioural change the product of systematic analysis of current (Davies 2004; Percy-Smith 2005). At the other and past conditions and trends, and analysis end of the spectrum, some methodologies asso- of the causal inter-relationships that explain ciated with a hermeneutic approach, including conditions and trends. In relation to policy re- a large proportion of ‘action-research’ projects, view and program assessment, there is a range tend to regard policy and program assessment of disciplinary and cross-disciplinary knowl- as more akin to iterative social learning projects edges (economics, law, sociology, public ad- than to the experimental sciences. ministration, evaluation, etc) that make highly useful contributions to policy and program un- 3. Practical implementation knowledge,in derstanding and improvement. There is seldom the present context of policy and program any consensus among social scientists on the effectiveness, is the ‘practical wisdom’ of nature of problems, the causes of trends or re- professionals in their ‘communities of prac- lationships, and the best approach for solutions. tice’ (Wenger 1998) and the organisational Various scientific disciplines may have differ- knowledge associated with managing program ent methodological approaches, and may offer implementation. These professional and man- complementary or sometimes competing per- agerial communities are often segmented rather spectives on complex issues. It is perhaps not than well connected. They operate within and surprising that inter-disciplinary approaches across the public sector, the private sector, and have come to the fore in recent decades for ad- the not-for-profit NGO sector. Relevant occu- dressing multi-layered social problems. pational groupings include program delivery Scientific (research-based) forms of knowl- managers, contract managers, enterprise man- edge primarily comprise the work of profes- agers, and the diverse range of professionals sionals trained in systematic approaches to and para-professionals who are engaged in di- gathering and analysing information. A con- rect service provision (Pawson et al. 2003) or cern with the quality and consistency of data is who provide support services linked into the fundamental to a scientific approach to analy- policy programs of government. sis. Nevertheless, methodological choices have These managers and professionals wrestle to be made. At one end of the spectrum in the with everyday problems of program imple- behavioural and applied social sciences, ‘sys- mentation and client service. Their roles may tematic reviews’ apply rigorous standards to ex- require managing upwards, downwards, and amine the state of current knowledge, giving outwards to external stakeholders (O’Toole, recognition only to those studies which clearly Meier and Nicholson-Crotty 2005). Their prac- focus on assessing the causal effects of specific tical experience in delivery often tends to be interventions. The so-called ‘gold standard’ for under-valued by the political and scientific sec- a rigorous experimental approach – adopted in tors. The sphere of ‘practice’ operates with the medical, biological and healthcare sciences evolving bodies of knowledge that tend to be
C 2008 The Author Journal compilation C 2008 National Council of the Institute of Public Administration Australia Head 7 specific to each professional niche. The train- that there are ongoing internal debates on such ing regimes for managers and professionals matters within each knowledge area. are often linked to concepts of ‘best practice’ How do these three lenses and forms of and to relevant research bases for assessing knowledge fit together? There is a considerable ‘effective’ practices. Their formal bodies of case-study literature on ‘policy communities’ knowledge evolve, and are subject to debate in and ‘policy networks’ that may include partici- ‘communities of learning’ (Wenger 1998). But pants from more than one institutional or indus- they also tend to become systematised and cod- try sector. There is also a considerable literature ified, and linked to standards and guidelines. in public administration concerning manage- In large organisations, best-practice guidelines rial challenges and processes in various portfo- may become overlaid with bureaucratic rules lio areas. There is also a literature on specific and protocols. managerial knowledges and other professional While the pressure towards systematisation knowledges; but rather less on how the sphere is significant, and the search for technical solu- of management practice interacts with the other tions (eg, build another IT system) is endemic, spheres of policy choices by politicians and the some areas of practice are less than impressed findings of systematic research. In fact, there by the business and engineering models. The has been surprisingly little research directly professional ethos in human services makes on the question of how the three clusters of room for unique cases and for the meaning- political, research and professional/managerial systems of clients (Schon 1983). This provides knowledge interact. By contrast there has been the ‘mental space’ for creating organisational increasing attention to the bilateral relations climates that are more favourable to case-based between public policy and the social sciences learning and more broadly the adoption of ‘or- (Davies, Nutley and Smith 2000; Stone 2001; ganisational learning’ approaches. Social real- UKCSS 2003; Edwards 2004; Saunders and ities are not seen as cut-and-dried and control- Walter 2005). lable, but as evolving challenges with unique Some useful research questions about the characteristics. Time is seen as a necessary in- three lenses might therefore include the follow- gredient of social and personal improvement. ing: However, the influence of this more open-ended • is there a low or high level of mutual aware- approach is severely tested whenever the politi- ness, recognition and understanding of each cal system shifts into crisis response mode, with other’s approaches? a heightened demand for rapid responses, rig- • is there is a substantial ‘cultural divide’ be- orous risk-management and standardisation. tween these forms of knowledge (Shonkoff 2000), and if so, are there any useful mech- Some Implications anisms/incentives to promote working to- gether more closely? • It has been suggested above that there are three in the jostling for salience or in the compe- broad types of knowledge and evidence that are tition between these sets of ideas, does one central to the design, implementation and eval- form of knowledge typically ‘trump’ the oth- uation of policies and programs. There is not ers – eg, in the policy domain, does politics one evidence-base but several bases. These dis- typically predominate over science and prac- parate bodies of knowledge become multiple tice? sets of evidence that inform and influence pol- Answers to all these questions are likely to vary icy rather than determine it. The three lenses greatly in different situations. From the view- of policy-relevant knowledge comprise three point of government, there is an expectation that perspectives on useful and usable information. research findings will assist but not determine Each of these types has its distinctive proto- policy directions and adjustments (UK Cabinet cols of knowledge, of expertise, of strategy, and Office 1999). There is a robust debate about what counts as ‘evidence’, albeit it is also clear where and how the research contributions can
C 2008 The Author Journal compilation C 2008 National Council of the Institute of Public Administration Australia 8 Three Lenses of Evidence-Based Policy March 2008
‘add most value’ to the policy process. For ex- icy decisions are not deduced in a neutral ample, is research most useful to government and objective manner from empirical-analytical processes at the commencement of the ‘policy work, but from politics, judgement and debate cycle’ (identifying and scoping a problem), or (Majone 1989). Policy debate and analysis in- in a later stage of options analysis (examining volves interplay between facts, norms and de- different costs and impacts), or in the program sired actions, in which ‘evidence’ is diverse evaluation phase when effectiveness is being re- and contestable. Different stakeholders within considered? Researchers are themselves some- the business, NGO and government sectors are times na¨ıve about what kind of policy analy- likely to have divergent views on what is the sis will be seen as relevant, and about how to key problem. For governments, problems and communicate and package their research out- issues become seen as worthy of investigation comes most effectively for government officials owing to a confluence of circumstances, such (Edwards 2004; UKCSS 2003). as: Governments tend to have a strong expec- • a perception of crisis or urgency; tation that the managers and professionals • the role of political mandates and priorities; who deliver services and implement programs • the role of expert judgement and advice (con- will do so with technical skill and efficiency. sultants, inquiries, etc); For those employed within the public sector, • organisational and issue histories; or effective implementation is the key consider- • the changing context of social values and pub- ation, and the government’s control over pri- lic opinion. orities and program design is very clear. For those employed at arm’s-length from govern- The definition and focus of problems are also ment, efficient implementation is reinforced by likely to be viewed differently through the three contractual relations and accountabilities for lenses of research, professional practice and service delivery. With a few powerful excep- government policy-making (and of course there tions, communities of practice may feel dis- will be some debates within each of these). enfranchised, especially when those who are Some policy arenas are more divergent and practical experts in delivery are not centrally in- strongly contested than others. Over time, as volved in early discussions about how programs Mulgan (2005) suggests, some fields may be- are designed and delivered. The political exec- come relatively stable and consensual; others utives of government are not especially adept may be subject to ‘profound disagreements’; at hearing and seeking out the voices of imple- and others again may be lacking a solid infor- menters, especially those who suggest that the mation base or a track record of on-ground ex- program goals cannot be delivered because ei- perience. The implications of these differences ther the framework is flawed or resources are in the nature of policy arenas are potentially clearly inadequate. Nevertheless, some shar- significant. Contentious (ie, unsettled and tur- ing of perspectives occurs. For example, both bulent) policy areas may tend to generate more public sector managers and political leaders heat than light, as the terms of evidentiary de- are successful only by making astute practical bate may be overwhelmed by partisan voices, judgements about priorities, garnering support despite the best efforts of those who wish for taking action, and persuading stakeholders to retain an ‘objective’ stance. Current policy about trade-offs and preferred options. examples might include strategies to address Researchers may believe that problem- major emergent issues such as climate-change definition and analysis should be closely responses; value-based issues at the intersec- linked to the data assembled by systematic tion of bio-ethics and bio-technologies; and de- research. However, in the realm of public pol- bates about procedural and substantive ‘fair- icy development, governments are in the busi- ness’ for workers and employers in industrial ness of framing issues and agendas. Strategic relations reform. Researchers who seek to make policy work is conducted in the context of an objective contribution in such areas may risk debates about issues and agendas. Thus, pol- being harnessed to positions proposed by strong
C 2008 The Author Journal compilation C 2008 National Council of the Institute of Public Administration Australia Head 9 advocates on one side of the debate and accord- management, policy development and program ingly disparaged by others. evaluation in the public sector (Agranoff and McGuire 2003). Networks bring to the table a diversity of lived experience and therefore a di- Conclusions versity of ‘evidence’ (relevant information, in- terpretations, priorities, and perspectives), not The demands for efficient and effective govern- only about what works but also about what is ment have fostered the need for performance worthwhile and meaningful. The evidence-base information. This has provided leverage for for understanding success factors for complex applied social research, concerned with pro- policy design and implementation may need to gram evaluation, implementation effectiveness, address the conditions under which innovation, and new models for tackling complex issues new thinking and new solutions may emerge in using new policy instruments and processes. a dynamic environment (Osborne and Brown This served to legitimate the general concept 2005). The three-lenses approach suggests that of evidence-based policy. However, there is a there may be importantly divergent perspec- large difference between a technical problem- tives on whether and how to increase mutual solving approach to knowledge, and a broader understanding and shared objectives. relational and systemic approach to knowledge that is located in multi-stakeholder networks. This article has suggested there are three Endnotes main kinds of challenge to the rational mission of ‘evidence-based’ policy. One arises from the 1. The author is grateful for comments on these inherently political and value-based nature of ideas from a number of colleagues over sev- policy debate and decision-making. Policy de- eral years, including John Alford, Meredith Ed- cisions are not deduced primarily from facts wards, Geoff Gallop, Richard Mulgan, John and empirical models, but from politics, judge- Wanna, and the anonymous referees for AJPA, ment and debate. Policy domains are inherently none of whom bear any responsibility for re- marked by the interplay of facts, norms and maining errors of argument and interpretation. desired actions. Some policy settings are data- resistant owing to governmental commitments. 2. The historical roots of such an alliance be- Secondly, information is perceived and used tween political power and systematic knowl- in different ways, by actors looking through edge reach back to the late Enlightenment, different ‘lenses’. From this perspective, there whose leading thinkers sought to undermine is more than one type of relevant ‘evidence’. the traditional capacity of governments to rely I have drawn attention to ‘three lenses’ that on appeals to precedent, authority and religious are especially important, centred on political values as the basis of government legitimacy know-how, systematic research, and profes- (Staum 1996; Head 1982; Berry 1997). Sys- sional practice. These perspectives all provide tematic social science later fortified the pro- important contributions to policy development, gressive impulses for social improvement un- but defensiveness and negativity are as com- derlying New Liberalism, early social democ- mon as cooperation. Although the context of racy, and the foundation of institutions such as decision-making is dynamic and negotiated, the London School of Economics and Political these key actors are anchored in institutional Science in 1895 (Lichtheim 2000: chapter 4). settings that make shared perspectives difficult to attain. References The third challenge to a rationalist concept of 6, P.2002. ‘Can Policy Making be Evidence-Based?’ evidence-based policy is that the complex mod- MCC: Building Knowledge for Integrated Care ern arrangements of networks, partnerships and 10(1):3–8. collaborative governance are difficult to har- Agranoff, R. and M. McGuire. 2003. Collaborative ness to the traditional forms of knowledge Public Management: New Strategies for Local
C 2008 The Author Journal compilation C 2008 National Council of the Institute of Public Administration Australia 10 Three Lenses of Evidence-Based Policy March 2008
Governments. Washington DC: Georgetown Uni- ence: Social Science and Public Policy in Aus- versity Press. tralia, eds P. Saunders and J. Walter. Sydney: Bakvis, H. 2002. ‘Pulling Against Gravity? Horizon- UNSW Press, 44–63. tal Management in the Canadian Federal Govern- Head, B.W. 2007. ‘Community Engagement – Par- ment.’ In Knowledge, Networks and Joined-Up ticipation on Whose Terms?’ Australian Journal Government, ed. M. Considine. IPSA Research of Political Science 42(3):441–454. Committee Proceedings, Centre for Public Pol- Hemmati, M. 2002. Multi-Stakeholder Processes for icy, Melbourne, 57–75. Governance and Sustainability. London: Earth- Bardach, E. 1998. Getting Government Agencies to scan. Work Together. Washington DC: Brookings Insti- Innes, J.E. and D.E. Booher. 1999. ‘Consensus tution Press. Building and Complex Adaptive Systems: A Berry, C.J. 1997. The Social Theory of the Scottish Framework for Evaluating Collaborative Plan- Enlightenment. Edinburgh: University of Edin- ning.’ Journal of the American Planning Asso- burgh Press. ciation 65(4):412–423. Booher, D.E. 2005. ‘Collaborative Governance Prac- Innes, J.E. and D. Booher. 2003. The Impact of tices and Democracy.’ National Civic Review Collaborative Planning on Governance Capac- 93(4):32–46. ity. Working Paper 2003/03, Institute of Urban Bruner, C., L.G. Kunesh and R.A. Knuth. 1992. What and Regional Development, University of Cali- Does Research Say about Interagency Collabora- fornia, Berkeley. tion? URL:
C 2008 The Author Journal compilation C 2008 National Council of the Institute of Public Administration Australia Head 11
Research Unit for Research Utilisation, Univer- Roberts, H. 2005. ‘What Works?’ Social Policy Jour- sity of St Andrews. nal of New Zealand 24:34–54. Oakley, A., D. Gough, S. Oliver and J. Thomas. Saunders, P. and J. Walter, eds. 2005. Ideas and In- 2005. ‘The Politics of Evidence and Methodol- fluence: Social Science and Public Policy in Aus- ogy: Lessons from the EPPI-Centre.’ Evidence tralia. Sydney: UNSW Press. and Policy 1(1):5–31. Schon, D.A. 1983. The Reflective Practitioner: How Osborne, S.P., ed. 2000. Public-Private Partner- Professionals Think in Action. New York: Basic ships. London: Routledge. Books. Osborne, S.P. and K. Brown. 2005. Managing Schon, D.A. and M. Rein. 1994. Frame Reflection: Change and Innovation in Public Service Organ- Toward the Resolution of Intractable Policy Con- isations. London: Routledge. troversies. New York: Basic Books. O’Toole, L.J., K.J. Meier and S. Nicholson- Schorr, L.B. 1988. Within Our Reach: Breaking the Crotty. 2005. ‘Managing Upward, Downward and Cycle of Disadvantage. New York: Anchor Dou- Outward: Networks, Hierarchical Relationships bleday. and Performance.’ Public Management Review Schorr, L.B. 2003. Determining ‘What Works’ in 7(1):45–68. Social Programs and Social Policies: Towards Parsons, W.2002. ‘From Muddling Through to Mud- a More Inclusive Knowledge Base. Washington, dling Up – Evidence Based Policy-Making and DC: Brookings Institute. the Modernisation of British Government.’ Pub- Shonkoff, J.P. 2000. ‘Science, Policy and Practice: lic Policy and Administration 17(3):43–60. Three Cultures in Search of a Shared Mission.’ Parsons, W. 2004. ‘Not just Steering but Weaving: Child Development 71(1):181–187. Relevant Knowledge and the Craft of Building Staum, M.S. 1996. Minerva’s Message: Stabilizing Policy Capacity and Coherence.’ Australian Jour- the French Revolution. Montreal and Kingston: nal of Public Administration 63(1):43–57. McGill-Queens University Press. Pawson, R., A. Boaz, L. Grayson, A. Long and C. Stone, D. 2001. Getting Research into Policy? Pa- Barnes. 2003. Types and Quality of Knowledge per for Global Development Network Conference, in Social Care. Knowledge Review No. 3, So- Rio de Janeiro, December. cial Care Institute for Excellence. Bristol: Policy Stone, D. and A. Denham, eds. 2004. Think TankTra- Press. ditions: Policy Research and the Politics of Ideas. Percy-Smith, J. 2005. What Works in Strategic Part- Manchester: Manchester University Press. nerships for Children? Ilford: Barnardo’s. Sullivan, H. and C. Skelcher. 2002. Working across Petticrew, M. and H. Roberts. 2005. Systematic Re- Boundaries: Collaboration in Public Services. views in the Social Sciences. Oxford: Blackwells. Houndmills UK: Palgrave Macmillan. Pollitt, C. and G. Bouckaert. 2000. Public Manage- UK Cabinet Office. 1999. Professional Policy Mak- ment Reform: A Comparative Analysis. Oxford: ing for the Twenty First Century. London: Cabinet Oxford University Press. Office. Reddel, T. and G. Woolcock. 2004. ‘From Consul- UKCSS [UK Commission on the Social Sciences]. tation to Participatory Governance?’ Australian 2003. Great Expectations: The Social Sciences in Journal of Public Administration 63(3):75–87. Britain. London: Commission on Social Sciences. Reid, F. 2003. Evidence-Based Policy: Where is the Wenger, E. 1998. Communities of Practice: Learn- Evidence for it? Working Paper 3, School for Pol- ing, Meaning and Identity. Cambridge: Cam- icy Studies, University of Bristol. bridge University Press.
C 2008 The Author Journal compilation C 2008 National Council of the Institute of Public Administration Australia