Mark Bunting & Suzannah Lansdell Designing decision making processes for data trusts: lessons from three pilots

April 2019

About the Authors Mark is a Partner of Communications Chambers and former senior manager at the BBC. He has published papers on platform governance and regulation for clients including Sky and Apple. He was a visiting associate at the Oxford Internet Institute in 2016-17; his peer-reviewed paper on new approaches to online content regulation was published in the Journal of Cyber Policy (October 2018).

Communications Chambers (www.commcham.com) is a policy and strategy advisory firm specialising in media, technology and telecoms.

Suzannah Lansdell is an Associate of Involve and advisor, designer and facilitator of stakeholder and public dialogue. She works on the Sciencewise programme to help government departments think through the role of public dialogue in new areas of science and technology, including data. She was also part of the team delivering the Citizen Assemblies on Brexit and Future Funding of Adult Social Care.

Involve (www.involve.org.uk) was founded in 2003 to create a new focus for thinking and action on the links between new forms of participation and existing democratic institutions. It promotes and practices participatory and deliberative decision-making to give people more power over the decisions that affect their lives. It aims to build a stronger democracy that works for everyone – that gives people real power to bring about change in their lives, communities and beyond. Involve has run a number of projects on public engagement in data.

Disclaimer This project was commissioned and run in collaboration with the Open Data Institute as part of a project funded by the UK Government’s Office for Artificial Intelligence and Innovate UK. It builds on research from the ODI's Innovation programme funded by Innovate UK. The views in this report are those of the authors.

This report is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License

[blank]

[1]

Contents

1. Executive summary ...... 3

2. About this report ...... 6

3. Designing a decision-making process for data trusts...... 8 What is a data trust? 8 Facilitating data sharing 8 Design questions 9

4. Consent, accountability and effectiveness ...... 10 A consent-based model 10 The components of legitimacy 13 Balancing accountability and effectiveness 14 Factors for a successful decision-making process 14 5. The decisions to be made ...... 15 Components 15 1. Scope 15 2. Co-design 17 3. Operate 20 4. Evaluate and retire 23

6. Engagement and deliberation ...... 25 What is deliberation? 25 Why deliberation matters 26 The benefits of using a deliberative approach more generally 27 When to use deliberative decision-making 28 Recommended deliberative methods 29

7. Summary of recommendations ...... 31

Annex 1. Summary of design questions ...... 33

Annex 2. Assessment of deliberative methods and techniques ...... 35

[2]

1. Executive summary

Why a data trust’s decision-making Design questions process matters Our analysis is based on three pilot trusts 1 Data trusts are a tool for delivering the initiated by the ODI, working with partners. In potentially great benefits of data sharing, considering decision-making processes for while protecting rights and balancing interests these pilots, we addressed five questions: in data use. They provide one form of • What decisions does the trust need to trustworthy data stewardship: through make? formalised decision-making processes, a trust • What objectives and values should govern makes independent, binding determinations those decisions? about how data may and may not be used. • Who are the stakeholders in the trust and A trust’s role can be powerful, especially when what are their incentives? data are highly valued and potential uses are • What policies, processes and activities will controversial. It may need to trade off the trust use to make and enforce its competing interests, coordinate diverse decisions? parties, act as honest broker, protect rights • What accountability mechanisms will the and enforce obligations, set technical trust use to demonstrate trustworthiness, standards and ensure compliance with protect stakeholders’ interests and relevant law and regulation. manage risks? Our analysis suggests that there are limits to A trust’s decision-making process is: how standardised and repeatable trusts’ “the set of policies, procedures and decision-making processes can be. The practices by which a data trust promotes answers to these questions will be bespoke to the beneficial use of data and manages each trust. But our analysis of the ODI’s pilots risks, balancing stakeholders’ interests in has identified some general considerations. accordance with the purposes and values of the trust.” A common set of decisions

Data trusts may be most valuable when there The decisions most trusts are likely to need to are many data providers, many potential use make can be grouped into the four stages of 2 cases, and different views about how data their lifecycle: should be used. In that scenario, how the trust • Scope: definition of the trust’s purpose makes decisions is crucial to its legitimacy and and values consent. The goal of this report is to provide • Co-design: definition of the trust’s guidance on how trusts might go about proposition – what data will be shared, designing their decision-making processes – from whom, for what uses, and on what what factors to take into account and some of terms; how benefits will be distributed the options that may be available. and risks managed

1 For more information on the pilots, see the Food Waste, Illegal Wildlife and GLA/Greenwich pilot reports 2 For more information on a trust’s lifecycle, see the ODI’s report on these pilots

[3]

• Operate: decisions about the trust’s The trade-off between rights and governance structure, approach to discretion stakeholder engagement, technical policy, A trust’s decision-making process must be enforcement and compliance processes, both accountable and effective. These can be and resources in tension. Accountability requires inclusivity, • Evaluate: decisions about how responsiveness and transparency, while performance will be assessed and effectiveness depends on speed, efficiency disclosed, how changes to rules and and scalability of decision-making. practices will be considered, when and how the trust will close down When a trust is formed, stakeholders may The decision-making process relating to each seek to protect their interests through stage should be captured in a set of governing establishing enforceable rights and documents and policies that establish guarantees; for example, many of the data stakeholders’ rights and obligations, and providers interviewed for these pilots wanted ensure transparency in the trust’s activities. to be able to control precisely the uses to which their data could be put.

The need for a solid foundation However, uses of data cannot always be Decision-making processes need a solid predicted, and it may not be feasible or foundation: an agreed purpose, common desirable to establish consensus or even understanding of a problem to be addressed, majority support for every use case. In agreement that a trust is the right vehicle to deciding how data are made available, a trust solve it, and a commitment to shared ethical needs discretion to adjudicate between values. The purpose of any data trust needs to different stakeholders’ interests and be resolved, with careful, deliberative input especially to protect the interests of smaller or from stakeholders, together with an less powerful stakeholders. assessment of the resourcing required to set The balance between stakeholders’ rights and up and run it, to ensure benefits outweigh the trust’s discretion needs to be resolved costs. carefully, with dialogue. In general, the larger Ambiguity at this stage can jeopardise the number of stakeholders, and the less stakeholders’ support for the trust and aligned their incentives, the more important consent to its authority. There is a risk, that to the discretion and accountability of the trust some extent all the pilots faced, of putting becomes. technical development ahead of rigorous definition of purpose and need. The value of a deliberative approach

The public have high expectations of a data Deliberation – a participant-led approach to trust. They need to see a clear purpose and problem solving and decision-making – can benefit from data sharing, particularly since play a crucial role in ensuring accountability. confusion, distrust and uncertainty on this Deliberative tools and techniques provide topic is widespread and pervasive. means of understanding different stakeholders’ perspectives, demonstrating honesty, competence and reliability, and

[4]

ensuring that all relevant interests are • Are contentious or involve trade-offs which identified and taken into account. benefit from greater understanding of what is driving those issues Three requirements must be met, in order for a process to be truly deliberative: Time and resources • Discussion between participants A data trust is not a ‘quick fix’ to complex • Involvement of a range of people governance issues. Developing the right • A clear task or purpose decision-making process requires resources, Deliberation is best used for decisions that: commitment and time. With respect to the three pilots, we • Require ownership of the outcomes by recommend further detailed work with stakeholders stakeholders to refine their purposes, develop • Need to demonstrate or benefit from a set of governing values, and progress taking account of a wider range of views, detailed co-design work. values, insights and experiences

[5]

2. About this report

Purpose The three pilots

This report provides recommendations on the Greater London Authority (GLA)/Royal design of decision-making processes for data Borough of Greenwich trusts, based on lessons learned from three This pilot explored two use cases: pilots conducted by the Open Data Institute. • Mobility use case (parking) – This use case The purpose is to provide advice to was to trial technology that increases organisations setting up, or considering available data on parking in the Borough in setting up, a data trust; and to inform the relation to coach parking and spaces that wider policy debate on data trusts as a form of are reserved for electric vehicles and data access and governance. Some points may electric vehicle car clubs, with the aim be relevant to governance of data sharing being making less-polluting transport arrangements more generally. options more attractive • Energy use case – This use case was to Each trust is likely to need its own decision- improve the energy efficiency of a council- making process, designed to meet the needs owned social housing block through of its particular stakeholders in its particular installing sensors to monitor and control context. The trust’s purpose and values the activity of a retrofitted communal provide the basis for decisions about which heating system. interests to prioritise and which rights take precedence. Therefore, the design of a data Illegal wildlife trade trust’s decision-making process is highly This report focuses on one of two use cases dependent on its purpose and values. considered by the illegal wildlife pilot.3 Wildlife image and shipping invoice data can Further work and stakeholder input would be be used to train recognition algorithms with needed to finalise the purpose and values of the potential to help border control officers all three of the pilot trusts considered in this identify illegal animals and animal products project. Our report therefore does not specify using services on their smartphones. Images a detailed decision-making process for the sourced from researchers, NGOs and others pilots. Instead it focuses on how a decision- involved in conservation activities around the making process should be designed – options world are a potential source of training data to consider and factors to take into account. for these algorithms. We have sought to identify a wide range of This pilot considered whether a data trust possible decisions and decision-making could provide a legal and technical techniques for data trusts. Not all of them will infrastructure for the identification, collection, be relevant to all trusts, and many trusts may assurance and storage of data, and the sharing be able to operate in a more slimmed-down of data with relevant organisations. way.

3 Regarding the second case, the pilot identified that the immediate opportunity lies in free hosting of open data by cloud hosting providers, and it is unclear at this stage whether a trust would offer additional value, so we have not considered it in detail in this report

[6]

Food waste Section 4 explores the factors driving the Food manufacturers and retailers play an success of a trust’s decision-making process, important role in addressing food waste. specifically accountability and effectiveness. Consistent measurement of food waste We suggest criteria for evaluating a decision- requires negotiation and coordination making process. between numerous stakeholders with different processes and definitions. Section 5 describes four sets of decisions most trusts need to make, linked to the data trust A number of food waste data sharing lifecycle described in the ODI’s report on the initiatives already exist in the UK. This pilot pilots. explored whether a data trust could support global food waste reduction efforts by Section 6 provides an overview of the deliberative techniques trusts may use to improving the ability of stakeholders to track and measure food waste within supply chains. support decision-making, and assesses the benefits of a deliberative approach. The pilot concluded that there may be value in a data trust but whether the benefits would The report concludes with a summary of recommendations. outweigh the costs and risks was unclear, based on the currently available evidence. We have sought to identify lessons for any Note on terminology potential trust operating in this area. When we refer to a data trust in this report, we refer either to the legal entity that Method comprises the trust (if one exists) or otherwise In preparing this report, we: to the organisations or individuals who take decisions in the trust’s name. • Reviewed selected relevant literature The concept of a ‘decision-making process’ is • Analysed the transcripts of the stakeholder closely related to governance. However, ‘data interviews carried out for each pilot governance’ has wider meanings, for example • Developed decision maps, as a general “everything designed to inform the extent of framework (see section 5), and supported confidence in data management, data use and service mapping for the GLA pilot the technologies derived from it.”4 On the • Assessed potential decision-making tools, other hand, ‘governance’ is also used more with a focus on deliberative techniques narrowly, to refer to an organisation’s board • Tested recommendations with the Open and other formal decision-making institutions. Data Institute (ODI) and its pilot partners. We therefore use ‘decision-making process’, Structure of the report rather than governance, specifically to mean The next section provides an introduction to “the set of policies, procedures and practices data trusts and our approach to designing a by which a data trust promotes the beneficial decision-making process for them. use of data and manages risks, balancing stakeholders’ interests in accordance with the purposes and values of the trust.”

4 British Academy and the Royal Society, Data management and use: Governance in the 21st century, June 2017

[7]

3. Designing a decision-making process for data trusts

What is a data trust? taking all relevant stakeholder interests into account. The use and stewardship of data raise some of the most vexed issues in the development and Facilitating data sharing deployment of digital technologies. There are risks, to privacy and security, safety and To succeed, a data trust must enable and encourage data providers to share data, fairness. But an equal concern is that beneficial innovation may be held back by a promote its availability to potential users, and failure to address both real and perceived monitor and mitigate the risk of harmful use. risks, and thereby promote safe, secure and These pilots highlighted potential barriers to trustworthy access to data for social and data sharing including:7 economic benefit. • Lack of evidence of the business case for The ODI has defined a data trust as ‘a legal sharing data structure that provides independent third- • Misaligned incentives between data party stewardship of data.’5 Data trustees take providers/subjects and data users on binding responsibilities to ensure that data • Time and resources needed to make data is shared and used for the benefit of identified available, and/or use it beneficiaries and other stakeholders. • Confusion and uncertainty about data Data trusts may be thought of as ownership, rights and control intermediaries between data subjects, data • Data standardisation and quality providers and potential data users. They • Reputational risk and mistrust support coordination between these diverse A data trust may not be able to overcome participants, including by setting and these barriers. But it can provide enforcing terms on which data may be made infrastructure – policies, processes and available for new uses. In doing so, the trust practices – that helps: for example, by establishes clear expectations on all parties bringing parties together, defining a shared and gives participants confidence that their purpose, providing a framework and standard interests will be protected. protocols for collaboration, establishing and protecting stakeholders’ rights, providing There are many possible approaches to data redress mechanisms, defining and enforcing governance. For example, Nesta has identified standards and offering resources and 14 models, just pertaining to personal data.6 expertise (technical, legal and so on). Data trusts are a particular form of “[A trust] might be a useful mechanism, governance in which data providers cede at like a container or a structure for least some control of data to the trust, which voluntary agreements within different then makes binding decisions about its use sectors. Whether it would materially

5 ODI, Defining a ‘data trust’, 19 October 2018 6 Mulgan, G. and Straub, V., The new ecosystem of trust: How data trusts, collaboratives and coops can help govern data for the maximum public benefit, Nesta, 21 February 2019 7 See also the ODI’s full report on this pilot project

[8]

change the outcomes I don't know but it individuals who may not themselves have could facilitate the process of setting been the source of data, such as friends, things up” Stakeholder, Food Waste pilot family and other community members; and may include third parties who do not Design questions directly engage with the trust but may still be affected by its decisions The British Academy and Royal Society’s • What policies, processes and activities will report8 on data governance identifies four the trust need to achieve its purpose and principles of data governance: adhere to its values? How will they ensure • Protect individual and collective rights and all relevant stakeholders’ perspectives are interests represented and considered? Who will be • Ensure that trade-offs affected by data involved in making trade-offs, and how? management and data use are made • What accountability mechanisms will the transparently, accountably and inclusively trust use to demonstrate trustworthiness, • Seek out good practices and learn from protect stakeholders’ interests and success and failure manage risks? What will it disclose, publicly • Enhance existing democratic governance and to particular stakeholders? How will the trust ensure data use complies with Designing a decision-making process that wider legal, administrative and democratic meets these principles involves answering five obligations? How will the trust resolve sets of questions: disputes and address complaints, and what • What decisions does the trust need to redress will be available? What role might make? For example, on what terms will there be for independent validation and/or data access be enabled – to whom, for arbitration in the decision-making process? what purposes? What data will be made The answers to these questions are likely to be available, and from whom? What forms of specific to each trust, and dependent on data combination will be permitted? What context: the sensitivity of the data involved, its security arrangements will they put in potential uses and value, the nature and place and how will those differ for different number of the trust’s stakeholders, and so on. kinds of data? As with the legal analysis conducted for these • What objectives should govern these pilots, it is not possible to recommend any decisions? In particular, what benefits single form of decision-making process or should the trust prioritise? Whose rights even a set of templates from which would-be must be protected, and what are those trusts could choose. rights? • Who are the stakeholders in the trust, However, there does appear to be a standard what are their motivations, and which of set of decisions that a trust is likely to need to their interests may be promoted or make. This is considered further in section 5. jeopardised by the trust’s decisions? This Before that, the next section discusses factors includes data subjects, including for a successful decision-making process.

8 British Academy & Royal Society, supra note 4

[9]

4. Consent, accountability and effectiveness

A consent-based model of the trust, and see it as an appropriate use of the data under the trust’s Data trusts’ intermediary role is crucial and stewardship often challenging. By definition, different • Stakeholders must also see the trust as stakeholders in a trust are likely to have legitimate, that is having the competence different incentives – otherwise there would and moral authority to decide how data be no need for a trust, a data-sharing may be used. agreement would suffice. Data communities are often fragmented, with stakeholders who Meeting these conditions obliges trusts to may know little about each other’s needs, and identify and align with the interests of the have varying levels of data expertise. As stakeholders affected by its activities. This can 10 GovLab put it, with respect to use of private be achieved in two ways: majority/ data for policy-making: consensus decision-making, in which parties individually or in sufficient number have the “The process of establishing data power to allow or veto specific actions; or collaboratives and leveraging privately accountability mechanisms, in which the trust held data…is onerous, generally one-off, decides, but consults, considers and informs not informed by best practices or any before doing so. shared knowledge base, and prone to dissolution when the champions involved Majority/consensus decision-making move on to other functions”9 Some of the data providers interviewed for these pilots anticipated a consensus (or at In the absence of a legal or regulatory least majority-voting) process, in which framework that compels participation, data providers would in effect have veto rights, for trusts depend on consent; not (just) the example over who could access a trust, for consent of data subjects required by data which purposes, on what terms. protection law, but a broader ‘consent of the governed’ that provides the basis for a trust’s There are certainly circumstances in which 11 decision-making authority. Existing food majority decision-making is viable. Where waste data sharing arrangements operated by there are relatively few parties to a data WRAP, for example, rely on the voluntary sharing arrangement, and their interests and participation of manufacturers and retailers, values are relatively homogenous, wide or which in turn is based on their confidence that universal agreement may be desirable and WRAP will hold it securely and ensure that achievable. Or, particularly sensitive decisions commercially sensitive information is not may require majority or consensus support. made available. For example, the Articles of Association of the Consent has two preconditions: TeX contract club, described in the Food Waste pilot report, require that proposed • Stakeholders – particularly data providers and data users – must support the purpose

9 GovLab, Data stewards: data leadership to address 21st century challenges, 12 June 2018 10 Assuming that there are no shared commercial goals to align incentives 11 ‘Majority’ includes consensual decision-making in this section

[10]

changes are approved by a majority of voting Finally, as trusts scale, and new purposes, members in each affected stakeholder group. uses, users and data sources emerge, the need to achieve majority approval can lead to But in complex situations with diverse complexity and ossification. Governance of stakeholders, majority decision-making has data trusts needs to be dynamic, to adapt to flaws. Participants with the greatest power, changing conditions, exploit new strongest incentive or best information are opportunities or address newly identified likely to dominate. In this situation, some risks. socially harmful uses may be allowed while socially valuable uses may be blocked. New and emerging technologies may have potential to achieve consensus and build trust. Majority decision-making within trusts may This may reduce the inefficiency of majority also conflict with other democratic or legal decision-making, but it is unclear that it requirements. Take the GLA pilot as an addresses the other problems described here example: a majority decision-making process (see Figure 1). New technological possibilities may result in outcomes that clash with the reframe the governance challenge, but don’t priorities of the local authority or undermine remove it: who decides the rules of the residents’ rights. Although mitigations could decision-making process, and how those rules be built into the trust’s rules to prevent this, are embedded in technical solutions, remain this merely reframes the governance crucial questions. challenge: who decides when other factors should prevail over a majority within the trust? Figure 1 Can data trusts use ‘distributed trust’?12

Emerging technologies may provide new ways of building trust in data, ensuring trustworthiness of providers and users, enforcing terms of data use and disincentivising rule-breaking, without the need for a central authority. For example, smart contracts, which allow data sharing and use contracts to be completed and verified via a distributed ledger, could be used both to incentivise participants to pursue the trust’s goal of safe data use and provide transparency and enforcement functions.13 Auditing techniques can help demonstrate that only uses consistent with the trust’s purposes and rules have been allowed.14

However, platforms based on distributed trust still need governance, for example to establish the criteria for which technical solutions optimise, and to establish sanctions for breaches. Indeed, the design of blockchain governance is turning out to be as complex and multidimensional as for any other institution. For example, voting does not become inherently less problematic as a decision- making mechanism on blockchains,15 and may even be worse in some respects.16

Finally, as the General Legal Report on these pilots points out, technological solutions are still subject to data protection and other law, and governance structures will be required to ensure compliance and establish liability.

12 Botsman, R., Who Can You Trust?: How Technology Brought Us Together and Why It Might Drive Us Apart, Cambridge, MA: Perseus Books, 2017 13 Dixon, C., Crypto Tokens: A Breakthrough in Open Network Design, 1 June 2017 14 Bennett, E., Legal trust + technical trust = data trusts, 4 January 2019 15 Buterin, V., Governance, Part 2: Plutocracy Is Still Bad, 28 March 2018 16 Daian, P., Kell, T., Miers, I. and Juels, A., On-Chain Vote Buying and the Rise of Dark DAOs, Hacking, Distributed, 2 July 2018

[11]

Accountability mechanisms the port’s use and sustainability. Much For these reasons, we think, trusts are likely to depends on the integrity, conscientiousness need other kinds of accountability and competence of the port’s independent mechanisms, that ensure trusts are responsive boards: to different stakeholders’ interests while retaining their discretion to make trade-offs “There are bound to be conflicts of between them. That means data providers will interest from time to time between — and in some cases within — the various need to cede a degree of control. This may be seen as implicit in the name ‘data trust’, since stakeholder groups. It is the duty of the the legal concept of ‘entrustment’ involves at [port’s] boards, at all times, to strike a balance that respects the interests of all least partial delegation of responsibility for decision-making in the trustor’s interests. stakeholders, not just one group, in the light of the objectives of the port, However, in a legal trust the trustee typically including commercial considerations, and has discretion only within a set of parameters what constitutes the 'common good' for defined by the trustor. This model does not all stakeholders (current and future) and read across to data trusts, which have to the port itself... Trust ports should always balance a number of competing interests, in deal with stakeholders in an accountable which providers’ wishes may not be decisive.17 manner although the board has ultimate This is a harder task, with more discretion responsibility for any decisions taken”18 balanced by more complex systems of ‘Bottom-up’ trusts, proposed by Delacroix and accountability. Lawrence, are likely to face similar issues.19 “How you go about setting the red lines is These are trusts to which data subjects really hard...you’re trying to increase transfer or cede control of data, for specified sharing, by taking away a lot of the purposes, which are “bound by a fiduciary onerous component of dealing with obligation of undivided loyalty.” But in [sharing] requests and how you say yes or practice even subjects who have voluntarily no to people. But in doing so, you’re trying provided data are likely to have interests that to develop quite general criteria that can diverge from each other’s in practice; there fit different situations. Whenever there’s will be disputes about whether particular uses some grey, you need someone who knows are aligned with the trust’s purpose; and there enough about the data and enough about may be external costs or benefits that the trust the purpose to make the call” should respond to. Fiduciary obligations do Stakeholder, Illegal Wildlife pilot not necessarily make the decision-making process simpler (as well as creating legal This is not a new challenge. For example, trust ambiguity, see General Legal Report). ports, independent statutory bodies which self-administer over 100 ports in the UK, Data trusts dealing with citizen-generated, operate with a similar model. They are guided personal or sensitive data face the additional by the interests of the diverse stakeholders in challenge of complying with data protection

17 This is similar to the finding in the legal analysis that legal trusts are unlikely to be appropriate structures for data trusts 18 Transport Scotland, Modern Trust Ports for Scotland: guidance for good governance, 2012 19 Delacroix, S., and Lawrence, N.D., Disturbing the ‘one size fits all’ approach to data governance: bottom-up data trusts, October 2018

[12]

law and regulation as well as meeting citizens’ Deliberative methods (described in section 6) expectations regarding public benefit and the come to their fore in openly and actively need to demonstrate trustworthiness. exploring these issues. The outcomes of deliberation enable decision makers to take a The components of legitimacy more informed decision which takes account of differing views. The right to make the call – the trust’s ‘social 20 licence to operate’ – must be earned, not The distinctive features of data trusts make assumed. When legitimacy breaks down, and them well suited to deliberative decision- consent is lost, the potential benefits of data making:24 diverse participants, reliance on 21 sharing may be at risk. Nesta highlights the consent, and potential unforeseen benefits case of inBloom, a $100m US education data and harms of data use, including for sharing initiative which despite laudable aims stakeholders who are not direct parties to the failed to achieve sufficient buy-in from trust. The delicate balancing of rights involved stakeholders, and closed after barely a year. in trusts’ decision-making requires a nuanced The consequent backlash resulted in greater understanding of stakeholders’ interests, and 22 regulation of student data privacy. mechanisms to ensure that those interests are respected by the trust in practice. How data trusts make decisions is crucial to legitimacy. They will be expected to adhere to Data trusts must, as O’Hara puts it, “help align principles of good governance, including trust and trustworthiness, so that we trust all transparency, responsibility, accountability, and only trustworthy actors.”25 But they must 23 participation and responsiveness. also demonstrate trustworthiness themselves, which requires stakeholders to be “Provide me with peace of mind by being able to assess their honesty, competence and trustworthy and sharing it with the right reliability. Deliberative processes give people” Citizen workshop participant stakeholders a chance to make this As noted above, trusts are likely to face power assessment at first hand, in part overcoming imbalances, with some stakeholders having the challenges of remoteness and insufficient stronger incentives, better information or evidence that often undermine judgements of more resources than others. A particularly trustworthiness;26 the more representative of important role for trusts is to recognise and the participant base, the more representative adjust to these imbalances. Their rules and the outcomes of deliberative processes. practices should ensure smaller and less Engagement is not just about communication; powerful organisations have a voice and are providing information on the benefits of data taken into account. sharing is necessary but not sufficient to demonstrate accountability.

20 O’Hara, K., Data trusts: ethics, architecture and governance for trustworthy data stewardship, February 2019 21 Mulgan and Straub, supra note 6 22 Bulger, M., McCormick, P., and Pitcan, M., The legacy of inBloom, Data & Society, 2 February 2017 23 UN Commission of Human Rights, Resolution on the role of good governance in the promotion of human rights, 6 April 2000 24 Patel, R., Public deliberation could help address AI’s legitimacy problem in 2019, Ada Lovelace Institute, 8 February 2019 25 O’Hara, supra note 20 26 O’Neill, O., Can more accountability increase trust?, Lecture at the British Academy, 28 June 2016

[13]

Balancing accountability and Factors for a successful decision- effectiveness making process The outcomes of decision-making also matter We have suggested in this section that: trusts to legitimacy and the licence to operate. A rely on consent; consent requires not only that data trust may be transparent and stakeholders support the trust’s purposes, but accountable, and engage widely and sincerely, also see it as legitimate; legitimacy in turn but if it is not competent to do its job – for depends on accountability and effectiveness, example, if data under its stewardship is not and on the balance between them. held safely and securely – it will still fail. Components of accountability include: Accountability mechanisms can have unintended consequences that make it harder • Inclusivity – does the decision-making for organisations to act quickly and effectively. process allow all stakeholders’ interests to They can bog organisations down in policy and be represented? procedure; they may be susceptible to being • Responsiveness – does the decision- gamed; they impose transaction costs on making process compel the trust to take participants; they may not scale, or be capable stakeholders’ interests into account? of evolving as the trust grows. There is also a • Transparency – is it visible to stakeholders risk of ‘democracy theatre’, if an organisation how their interests have been addressed attempts to use engagement to win support, and balanced with other objectives? or the appearance of support, for a decision it Components of effectiveness include: has already made. • Speed – can the trust make timely A trust’s legitimacy therefore relies on striking decisions, including allowing quick a balance between accountability and responses to risks? effectiveness. Too many rules and • Efficiency – is the cost of running the trust stakeholders with vetoes, and nothing gets proportionate to the benefits? done. On the other hand, too much autonomy • Scalability – is the decision-making process risks distrust and neglect of stakeholders’ sustainable as the volume of data and interests. Effective engagement and number of uses grow? deliberation preserve a trust’s autonomy while ensuring it is embedded in its wider We suggest these components provide the social context and responsive to all its starting point of a framework for evaluating a stakeholders’ demands. We discuss this trust’s decision-making process. further in section 6.

[14]

5. The decisions to be made

Components Not all trusts will need to make all these decisions. A formative trust should look at the Although every trust will be to some extent decisions shown in Figure 2 (and listed in more bespoke, we think it is possible to identify a detail in Annex 1. nnex 1), decide which are standard set of decisions most trusts will need relevant, how important they are, and based to make, and some general observations on this prioritise particular aspects of the about how they should be made. design process. Decisions can be grouped according to their 1. Scope place in a trust’s lifecycle:27 scoping, co-design, operation and evaluation. A data trust is founded on a clear statement of purpose and values. These should be captured Figure 2 provides a schematic overview of the in a governing document, constitution or key decisions that must be made regarding articles of association. They rely on a common each stage, and the documents in which the understanding between the trust’s instigators answers may be captured. In principle, each of a problem that can be addressed by data phase requires answers to the questions in the sharing, and a shared view that a data trust is prior phases – although in practice the design the right vehicle. work is likely to be iterative.

Figure 2 Common components of the decision-making process for a data trust

Engagement/ Key Key: Start/end Decision deliberation document most needed

1. Scope 2. Co-design 3. Operate 4. Evaluate

Change Decision to launch: Control What is the shared need and What is the trust’s Policy problem to be solved? governance structure? Data Is a data trust the What data will be made How will changes to rules or Provider best solution? accessible, by whom? processes be considered? When can data be removed? Agreement Third Reporting Party Policy What is the trust’s strategy for Rights What are the trust’s purposes? stakeholder engagement, What are its values? openness and disclosure? What criteria will determine Data How will success be who can access data and on User measured, evaluated and reported? what terms? Agreement How will the trust’s rules be enforced? What monitoring systems will Constitution Benefit How will benefits of it need? Sharing In what circumstances should use be distributed? Policy the trust close? What does What role will the trust play in closedown involve? data storage, processing, How will risks be Risk standards, security? managed? Register Decision to close: Have the closure conditions What responsibility will the How will disputes be be met? Dispute trust have for legal and resolved and what Resolution regulatory compliance? redress will be Process Privacy provided? Policy Closedown What is the trust’s sustainable What resources and funding Procedure business model? does the trust need?

27 This analysis was prepared as an early input to the ODI’s wider project, and differs slightly from the framework in the ODI’s final report

[15]

The purpose statement - the trust’s ‘North foundation, but it is the superstructure of Star’ - needs to consider and as far as possible practices, policies and processes which give reconcile the different interests of all the them practical force. Equally, values can be trust’s stakeholders. So decisions about scope undermined if stakeholders believe they are should be made in a way that exposes, rather not truly embedded in how the trust works. than submerges, differences which may result in conflict further down the road. For example, details of the Royal Free NHS Trust’s relationship with DeepMind sparked Alongside the purpose statement, we suggest controversy despite the protections in the a trust should define a set of values that data sharing agreement29 and DeepMind’s underpin its policies and guide decisions. much vaunted ‘ethics and society principles’.30 While values will to some extent be bespoke An independent audit of the revised to each trust, they are likely to include: agreement (following the Information Commissioner’s Office’s finding of non- • integrity (acting to fulfil the purposes of the compliance with data protection law) was not trust and in the interests of all immediately sufficient to quell stakeholders’ stakeholders; not unduly influenced by any concerns.31 party or by trustees’ own interests) • objectivity (decisions based on merit and Systematic analysis of stakeholders’ evidence) perspectives, importance and support will be • openness (accessible by all stakeholders, essential to the trust’s development of its open about decisions and their reasons for purpose and values. Stakeholder mapping decisions, with relevant information helps identify priorities.32 Mapping needs to disclosed in a timely way) consider stakeholders who are affected by the • equity (a fair balance of risk and reward use of data, not just parties to the trust. between stakeholders) Further stakeholder input would be needed to • respect for rights (ensuring individuals’ and finalise the purpose of all three of the pilot organisations’ rights are protected, trusts considered in this project – revealing including by guarding against misuse of the potential complexity and time required to data) make Scope decisions. Many ethical frameworks have been developed to help guide the development of However, the context in the three pilots was data-driven technologies.28 It is beyond the very different: the illegal wildlife trade pilot scope of this report to consider their found general enthusiasm for the concept of a applicability to data trusts in detail. The key data trust but less consensus about its specific point is that without effective and purpose. Potential data providers are accountable decision-making processes, heterogeneous and potentially numerous, so ethics risk being empty slogans. Values are an a working group of core partners could be essential part of a trust’s underpinning

28 Floridi, L. et al, AI4People—An Ethical Framework for a Good AI Society: Opportunities, Risks, Principles, and Recommendations, Minds and Machines, December 2018 29 New Scientist, Revealed: Google AI has access to huge haul of NHS patient data, 29 April 2016 30 DeepMind, Ethics & Society Principles 31 Techcrunch, Audit of NHS Trust’s app project with DeepMind raises more questions than it answers, 13 June 2018 32 ODI, Stakeholder Analysis

[16]

established to engage more widely to define a Incentivising data provision and use unifying purpose. A trust is unlikely to be able to transform stakeholders’ incentives, but it may be able to In the food waste case it was unclear whether align them, for example by: incentives were sufficiently aligned for further data sharing, beyond the initiatives already in • Providing benefits to providers in return for place by WRAP and others; an opportunity to data (such as access to more data, or share sales data was identified, although benchmarking of their data relative to peer stakeholders recognised this data was highly group organisations, as with current food commercially sensitive. waste data sharing initiatives) • Establishing means of sharing commercial In the GLA pilot, we would suggest that the value created by data use senior decision-makers define an overarching • Imposing legally enforceable sanctions for purpose, and test it with stakeholders, breaches of the rules including the public. Deliberation will be • Defining different levels of data access for important to build consensus, expose and different types of user (open access, open- work through differences in an open way – the to-approved user classes, access-with- use of an independent third party facilitator permission, invite-only) may help to level power imbalances and • Implementing or requiring providers to enable open dialogue. implement privacy-enhancing technologies to prevent unnecessary or unwanted 2. Co-design processing of personal data, without loss of The co-design stage determines what the trust functionality, including by anonymising will do – its offer to data users and re-users, data (although interviewees pointed out and its terms of use. Key outputs of this phase that it may be impossible to fully include agreements detailing the rights and anonymise some data, and as noted in the obligations of data providers and data users; a GLA case, the value of data may lie policy on the distribution of benefits; a risk precisely in the ability to link different register, describing the trust’s strategy for datasets to the same individual or source) identifying and mitigating risk; and a method • Aggregating data (although some pointed for resolving disputes once the trust is out that may limit the uses to which data operational. can be put)

As the name suggests, this stage also requires A data use policy, developed deliberatively close engagement and collaborative work with with input from stakeholders, should specify stakeholders. A discovery phase is likely to be the criteria by which data use and access needed, to investigate: decisions will be made. Engagement should include the public in the case of personal data • the type and quality of the available data or data that could have significant public (data holders do not always know what impacts. The scale of deliberative effort they’ve got or who owns it; and, as the depends on the value of data and the illegal wildlife pilot found, mis-labelling and magnitude of any risks. a lack of consistent definitions can lead to significant complexity and inaccuracy) For example, the GLA pilot addressed the • users’ incentives. potential value of citizen generated data, such

[17]

as movement data. The greatest value may lie • Establishing an independent arbitration in personalized movement data, but this may process not be deemed acceptable for wider use. • Preventing providers who have left from Indeed in the workshop held with public rejoining within a certain time period. participants whilst some benefit was seen in sharing mobility data there were concerns Balancing benefit and risk about tracking and surveillance. Deidentified Fear, misplaced concerns or over-reaction to movement data may be more acceptable but risks can lead to underuse of data 33 likely of less value both for public and technologies. Trusts need techniques to commercial use. make a balanced assessment of the benefit and risk of data use, and to establish processes There may be different proposition options. and technical solutions that capture benefit The value of different options needs to be while mitigating risk. assessed, both financial and non-financial, over time. The size and distribution of For example, researchers developed a quantitative benefits may need to be decision tree for assessing the risks and 34 modelled under different scenarios. benefits of publishing biodiversity data. The decision tree provides a risk management The data provider agreement needs to give protocol that takes data holders through a sufficient reassurance to providers that they structured process and prescribes actions for will commit to making data available, on an different types of risk. ongoing basis, without them having to approve every use (see, for example, A risk register could be established to identify providers’ fears about unintended uses in and assess risk, and capture mitigation both the illegal wildlife and food waste pilots). actions, with regular review by the trust’s Providers will look for robust risk management board. processes and means for them to escalate When risks cannot easily be foreseen, the concerns. trust should focus on establishing effective The data provider agreement should also monitoring systems. Technical solutions may specify terms on which data providers can be available; one interviewee suggested that a withdraw their data from the trust. Providers condition of data access is that the trust may prefer a simple exit-with-notice right, but creates an automated log that shows how where this has significant knock-on effects on data is being used, which can be monitored by users or other providers, the trust needs to trustees or the wider data community, subject consider a more balanced approach. Ideas to privacy considerations. suggested in the interviews include: Distribution of value Concern about commercial use of data • A process to determine whether the loomed large in the interviews for these pilots; provider has valid reason to withdraw, as either a general view that the data in question defined in the provider agreement should be reserved for not-for-profit use, or a • Agreement of a majority of other providers concern that value would be captured by or affected users

33 Floridi et al, supra note 28 34 Tulloch, A.I.T. et al, A decision tree for assessing the risks and benefits of publishing biodiversity data, Nature, July 2018

[18]

commercial organisations and not flow back to provide an order-of-magnitude assessment of data holders. value. This work will need to take into account the value of data in combination with other This is echoed by fears expressed by some that datasets, as well as stand-alone; and recognise opening access to public datasets could result that some benefits will emerge unpredictably in value being transferred from the public over time, requiring the policy to adapt to new sector to the private sector. For example, a sources of value. number of witnesses to the House of Lords Select Committee on AI were critical of public- There is no normative ‘right answer’ to the private data deals which, they believed, distribution of value. Most often, the right allowed data to flow from the public sector to answer is what can be negotiated. However, the private sector without securing proper the trust has a particular role in (i) defending value for the taxpayer.35 the interests of parties who cannot negotiate effectively, because they are too dispersed Careful deliberative work will be needed to (e.g. the public) or lack sufficient information understand and respond to stakeholder views about the other parties’ incentives; and (ii) on this issue. Previous work with citizens has ensuring that social and non-financial benefits shown that the public do not necessarily reject are taken into account. commercial involvement out of hand as long as there is a clear wider interest The trust also needs to decide how it will be demonstrated.36 funded: from fees charged to data users, by its creators, from philanthropic or public funds, Discussions with citizens for the GLA pilot or from commercial value created by opening found a resistance to data being shared for up data. These options need to be tested with commercial use beyond the purpose of the stakeholders.37 data trust. Independent mediation “Don’t sell it to anyone...not the highest The parties forming the trust may have a bidder...but for improving things” Citizen direct interest in the outcomes of the co- workshop participant design stage, and/or may not yet have secured These concerns need to be balanced by a the trust of all stakeholders. For this reason, recognition that market signals are often the many formative partnerships use an best way of identifying valuable uses of data. independent third party as ‘honest broker’, to It would be odd and self-defeating in many manage the negotiation process, prepare key cases for trusts to rule out commercial uses agreement documents, and work towards an entirely and as a matter of principle. equitable and sustainable outcome. For example, Southampton University drafted the Instead, we suggest that trusts should have a legal data sharing agreement between policy on distribution of value. This should be corporations and start-ups that underpins the informed by market testing and economic Data Pitch innovation programme. impact assessment, which should seek to identify potential commercial applications and

35 House of Lords Select Committee on Artificial Intelligence, AI in the UK: ready, willing and able?, §77, April 2018 36 Research Councils UK/Involve, Public dialogue review: Lessons from public dialogues conducted by the RCUK, July 2012 37 Whether the trust can be a for-profit entity is a separate question, see section 2 of the General Legal Report

[19]

Similarly, in developing the TeX contract club, Key questions to be addressed in governance the Tax Incentivised Savings Association (TISA) design are: employed law firm Pinsent Masons to manage stakeholder engagement, co-design policies • Who should be represented? and processes that aligned different interests, • In what forums and processes? ensure no one group was able to dominate the • With what rights? decision-making process, and shepherd • What level of openness and transparency? negotiations of the club’s rules to a Interviewees for the pilots were clear that 38 conclusion. their consent depended on them being confident that the trust’s decision-makers 3. Operate understood their community’s concerns: Figure 2 identifies the types of operational “A governance structure that is mostly decisions that need to be made, namely: made of people who actually understand governance and stakeholder engagement; the needs of their community is probably technical policy; enforcement and better…my vision has always been for an compliance. We discuss each in turn. organisation that’s purpose-driven by its Governance and stakeholder engagement own community” Stakeholder, Illegal As discussed in section 4, governance Wildlife pilot arrangements must strike a balance between As discussed in the General Legal Report on discretion (for the trust) and certainty (for these pilots, independent governance – in the stakeholders); and between effectiveness and sense of dispersed power – is crucial, accountability. particularly where: Some data sharing frameworks provide very • stakeholders have varying degrees of little discretion to the data steward, such as capacity to participate, risking power the Administrative Data Research Network, imbalances and under-representation which enables access to research data only • the risk of conflict is high. under closely specified conditions and processes. These approaches may be less Some interviewees suggested Advisory relevant to data trusts that are dealing with Groups, with a formal mandate and less predictable and more diverse use cases responsibilities, to allow the board to consult and sources of data. different stakeholder communities, as an alternative to bringing them into the decision- In the illegal wildlife case, for example, a data making body itself. These could be organised trust is likely to have significant discretion, by stakeholder, geography or use type.39 For given the fragmentation and diversity of the example, in the food waste context, WRAP has data provider community. So governance working groups for different industry sectors. mechanisms are needed to allow providers to see and influence the trust’s decisions, Advisory Groups work well where stakeholder without introducing cumbersome approval communities are relatively small or processes. homogenous. Where they are not – for example, if stakeholders include the general

38 See Food Waste pilot report 39 See Section 6 of the General Legal Report for further discussion of advisory groups

[20]

public – broader consultation and deliberation Technical policy is likely to be required, as discussed in section Data providers and users interviewed for 6. these pilots almost universally anticipated a technical role for the trust. Both the food The board may be empowered to delegate waste and illegal wildlife stakeholders some tasks to a committee or oversight group emphasised the potential for inconsistencies with independent members. These might take and/or low quality data to undermine the on tasks that are particularly sensitive, such as trust’s value. More generally, as noted in this resolving disputes or adjudicating on data project’s General Technical Report, uses; or that require specialist technical or technology choices both constrain what data legal knowledge; or in which independence is can be used for and may provide governance particularly important, such as evaluating solutions (for example, enabling the risk of impact. data misuse to be managed and mitigated).

Trusts must also decide what level of Decisions include: transparency and disclosure is appropriate. Transparency does not itself build trust – • How and where should data be stored? indeed it may undermine it, if it can be • What gatekeeping, authentication and exploited by bad actors.40 On the other hand, authorisation systems should be deployed transparency helps demonstrate integrity and to enforce the trust’s data access and use honesty, communicate goals and show rules? benefits. • What technical standards, formats, security and interoperability requirements Trusts need to consider transparency of what, should apply to data and metadata (if any)? for what purpose, and how they will • How are the quality, accuracy and security proactively communicate to stakeholders, and of data assured? under-represented groups in particular. Areas • To what extent and how should data be in which to consider disclosure include: categorised, anonymised, aggregated, and • What classes of data are held combined with other data? • How decisions about data access are made, • How does the trust check for error, bias and who is making them and discrimination in data? • How data has been used There was no consensus about the answer to • How risks are being managed (although not these questions, nor how they should be in sufficient detail to allow exploitation) answered. Several interviewees urged against • What the outcomes of the data trust’s designing “overwrought” solutions that activities have been require the trust to predetermine data uses • How any value created by data use has and required quality standards. Some warned been shared, and what policies have that the trust may lack credibility in making governed commercial use of the data technical decisions. • How to complain, and how complaints have been handled. “We would let the user decide what the data tells them. We give them the data and they can define what metrics they

40 O’Neill, O., A Question of Trust: The BBC Reith Lectures 2002, Cambridge: CUP, 2002

[21]

want to use” Stakeholder, Illegal Wildlife was often on considerations of data gathering, pilot quality, storage and security, rather than the purpose of data sharing. This risks developing Others expected the trust to play a bigger technical solutions to a problem that is not technical role, especially where: sufficiently clearly specified, or where • the data is complex or technically specialist stakeholders’ interests are not fully understood. • the range of uses is large or unknown • the quality of data is uncertain The extent of the trust’s direct responsibility • the number of providers is large for data will have implications for its legal • an honest broker is needed to resolve structure (see General Legal Report). As a rule disputes between stakeholders about of thumb, the greater a data trust's level of technical issues responsibility (and potential liability), the Many were concerned about loss of control of more likely it will be that an entity with its own data and the potential for shared data to legal personality (in the UK, likely some form ‘escape’ or be copied. Security was seen as of corporate entity) will be the appropriate essential to trust, and a challenge. structure.

“We are really strict on data security, over Enforcement and compliance time [stakeholders] have come to trust us Once the trust’s rules are established, the on that because we’ve been doing this a trust must decide what role it plays in long time...data security was seen as enforcing them. This role may be reactive much more of an issue [in a neighbouring (reliant on notifications of alleged sector] because we hadn’t built the infringement by participants or third parties) reputation there” Stakeholder, Food or proactive (monitoring for it). Waste pilot Where a particular trust sits on this spectrum An alternative approach would be for a trust depends on: to provide a data oversight function, rather • where liability sits (in general, the greater than storage and processing. Its role might be the liability trusts or data providers are to verify the data held by providers, for exposed to, the more proactive they are example by carrying out data audits, or likely to need to be in monitoring requiring them to self-certify, or some mixture compliance). A key question is how many of both. links in the data use chain can go wrong, In either case, a technical audit is likely to be and the ability of one participant to have needed to understand the quality of the implications for others’ liability; for available data, consider the case for example if a user undermines subjects’ harmonisation, assess potential for use privacy rights, what liability if any is borne tracking, consider appropriate authentication by the data provider or the trust itself? The and security measures, and so on. General Legal Report considers these questions in more detail But, not too early. There is a risk of putting the • the technical feasibility of monitoring. Can technical cart before the governance horse. tools be used to track data use without For example, the focus in both the GLA pilot infringing users’ legal rights or creating and the interviews for the illegal wildlife pilot excessive disincentives for use?

[22]

• the magnitude of the potential harms from might take quite a significant and potentially misuse costly role in ensuring data quality. • the capacity of different stakeholders to Cost-benefit analysis and an assessment of identify and flag misuse funding options will therefore be needed in • which jurisdiction the trust operates in. this phase. Resources and funding Assessing resource requirements is highly 4. Evaluate and retire context-specific. For the three pilots Performance assessment and reporting considered for this project, interviewees As noted above, a trust is likely to need to variously recommended the following regularly disclose its activities and outcomes functions and skills: with reference to its purposes and impacts – • ‘Data steward’ - a mixture of technical, akin to annual reporting by companies or management and stakeholder skills. (It was charities. not clear whether this was genuinely Evaluation may require periodic assessment, distinctive from a general management not only of financial aspects of performance, function, although we note that GovLab is but also more complex and subjective undertaking a project to define data questions, such as whether the trust is stewardship as a corporate function with adequately delivering its purposes, has specific associated responsibilities)41 effective processes for promoting beneficial • ‘Account managers’ (in one interviewee’s use and mitigating harm, is appropriately experience, every ten partners require one assessing its non-financial impacts, is account manager, although this will clearly achieving an equitable balance between the vary from case to case) needs of different stakeholders, and so on. • Technical expertise • Sector expertise, for example to inform For example, trust ports are encouraged to decisions about data uses (this could be in- report performance on financial measures, house or via advisory groups) value added, labour productivity, profitability • Operations. of land holdings, channel depth management and berth utilisation to provide a rounded It appears that most interviewees considered picture of impact – while also recognising not that a trust would be a fairly traditional all benefits can be quantified: “The organisation, with premises, in-house staff improvement and modernisation of [a port’s] and so on, albeit with a strong emphasis on assets, services and infrastructure for the operating as lightly and cost-effectively as benefit of its users cannot always be valued in possible. this way.”42 The trust’s costs could be significant, and need Frameworks have been developed for to be proportionate to the benefits. For assessing value against non-financial metrics example, in the illegal wildlife pilot, the trust and purposes.43 For example the BBC and Ofcom each have an elaborate set of metrics

41 GovLab, supra note 9 42 Transport Scotland, supra note 18 43 Kelly, G., Mulgan, G. and Muers, S., Creating public value: An analytical framework for public service reform, 2002

[23]

to help them assess whether the BBC is • a process for the trust to make a decision, fulfilling its statutory purposes.44 including any consultation on that decision, and/or any right or obligation on the trust However, few data trusts are likely to have the to refer it to an independent committee or resources and measurement capability to arbitrator. assess performance in such a rigorous way. The change control process may be one of the Evaluation should therefore be nuanced, and trust’s most important and sensitive policies. potentially complex, but also proportionate. For example, in the case of the TeX contract Metrics need to be chosen carefully to avoid club mentioned above, elaborate rules and over-simplification and distorted incentives. processes were put in place to govern changes For example, assessing a trust’s effectiveness to policies and activities.45 The change control solely by measuring how many complaints are process is designed to ensure that changes can resolved quickly would be highly problematic. neither be forced through nor blocked by any single stakeholder. Independent evaluation is preferable, and might include internal and external Closedown stakeholder interviews, public research Closedown is a rather fundamental question (including deliberative approaches), and of evaluation, which can be easy to overlook stress-tests of trust processes and policies. in the formation phase. A trust should include in its governing documents a statement of: Change control Trusts need processes for changing their rules. • the circumstances in which the trust will be The more unpredictable their context – new wound up uses, unforeseen risks, heterogenous provider • who will decide when those circumstances and user groups – the more important the are met, and how change control process is. • what happens as a result, including to providers’ data, the services using that The change control policy comprises: data, and third parties who may be • a mechanism for proposing a change – benefiting from (or harmed by) those uses either by a stakeholder or by the trust, at • how any assets (or obligations) held by the its own discretion, or in response to a trust are to be distributed (or discharged) stakeholder complaint The closedown policy will need to be the • circumstances in which proposals may be subject of discussion with stakeholders in the considered (‘change triggers’) formation phase. • a process for assessing whether the change triggers have been met • a process to assess the merits of the proposal, including mechanisms for consulting stakeholders

44 BBC, Annual Plan 2018/19; Ofcom, Holding the BBC to account for delivering for audiences: The BBC’s performance, October 2017 45 TISA Exchange Ltd, Articles of Association, §5

[24]

6. Engagement and deliberation

o The results of these discussions are What is deliberation? considered; the results themselves Deliberation is a participant-led approach to may or may not be different from problem solving and public decision-making. It the original views of some/all of allows participants to make decisions or the participants, but they will have recommendations based on consideration of been arrived at through collective relevant information, and the collaborative discussion and consideration. discussion of issues and options. Participants, 2. Working with a range of people and depending on the situation, may include information sources stakeholders, the public, either in their role as o The information within a stakeholders (for example in a community deliberative project (some of issue) or as “mini publics” recruited to which may have been specifically represent the views of the wider public (for requested by participants) example regarding city developments or contributes to a clear context and national policy), or experts/specialists. the consideration of various factors within decision-making The key aspect is that the participants’ own o The participants themselves input forms the basis of the results and represent a diversity of findings (this forms part of the legitimacy of perspectives and interests. deliberative decision-making). Deliberative discussions can be Whilst a collaborative and deliberative managed to ensure that these approach to decision-making has benefits, perspectives and interests – even if clearly not all decisions can (or should) be they represent a minority – are made deliberatively. However, there are some included within a balanced key points in the life cycle of a data trust that discussion. warrant a deliberative approach to build 3. A clear task or purpose o insight, value and trustworthiness into the Related to influencing a specific data trust’s operation, practices and decisions. decision, policy, service, project or programme. There are three requirements which must be involved in order for a process to be truly The ODI’s Invitation to Tender (ITT) specified deliberative: that a “key motivation behind data trusts is their potential to increase trust in the way that 1. Discussion between participants at data is shared and used. In some cases this will interactive meetings or events involve the trust of individuals whom the data o These meetings, which may be might be about or otherwise have an interest supplemented by the use of online in; in others it will involve the trust of technologies, are designed to organisations that hold data.” The process of provide time and space for learning deliberation is conducive to producing results new information and discussing that are legitimate and trustworthy. the significance of this knowledge (when considering existing This is especially pertinent to a topic such as attitudes, values and experience) data (specifically its storage and its use), which

[25]

– evident through discussions of Cambridge purpose and uses…don’t misuse data… Analytica and Facebook, for example, as well allow data users to have some choices as electoral interference – remains a source of about big decisions” Citizen workshop uncertainty and public distrust. The uses and participant misuses of data are often widely-discussed only in the context of scandals and ongoing Deliberative methods provide a wealth of data investigations. on public and stakeholder attitudes and values. They also provide opportunities to Why deliberation matters explore why these attitudes and values are held. One practical reason is that deliberative The ODI’s hypothesis was that a data trust techniques often allow more time to be spent must “[engage] and [make] decisions with with the participants.46 In addition, the use of different stakeholders so that the decisions it deliberative methods can (depending on the makes – such as who has access to the data, location) help to encourage a sense of under what conditions and how the benefits community discussion and representation. of that use are distributed equitably – are made openly and deliberatively.” In doing so, For this reason, deliberative methods often a data trust would increase the benefit the participants themselves. The trustworthiness of the way that data is shared experience provides opportunities for and used. collective discussion and reflection in depth; sharing views and developing these As also noted in the ITT, central to building collaboratively, and presenting them to trustworthiness is ensuring that different experts and decision-makers. These experts stakeholders are engaged with as part of an can help participants to learn about the key inclusive, open and deliberative decision- issues in question, to talk about them with making process. (not past) each other, and to benefit from diverse points of view, discussions and ideas. The lessons from the pilot work support this approach – for stakeholders and the public to The process of undertaking deliberative have trust in a data trust, it has to reflect their methods is in itself of importance to trust and issues, expectations and perspective on trade- legitimacy (of the results, the process, and the offs; build consensus; and be open, honest data trust itself). This legitimacy is derived and accountable. from the participants, and the fact that their input is the basis of subsequent decision- The deliberative element of this process is making. crucially important; it validates and strengthens the recommendations made, “I hope they will understand the public’s because they directly reflect the issues, hopes concerns in regards to privacy. But I hope and concerns of the stakeholders (and the they make sure [data] is used for good ways in which these priorities can be rather than bad” Citizen workshop balanced). participant “Give some benefit back to data givers, fully inform the public about benefits,

46 Stoker et al., Fast thinking: Implications for democratic politics, European Journal of Political Research, Vol. 55, No. 1, September 2015. See also Kahneman, D., Thinking, Fast and Slow. New York: Farrar, Straus and Giroux, 2011

[26]

The benefits of using a deliberative campaigning or lobbying) at any point in the approach more generally policy cycle.

As noted in section 4, a data trust may derive Figure 3 summarises some of the benefits of its legitimacy – and, by extension, the trust of deliberative processes; benefits which are stakeholders and citizens – from its capacity to relevant to decision-makers, policy-makers, enable, encourage, and benefit from collective and participants themselves. discussion, reasoning, and decision-making. Figure 3 Benefits of deliberative processes

As argued in a recent article by Nesta, “trust However, the specific benefits of deliberation has to be continually earned, and is not for stakeholders, the public (or ‘a’ public) – generic: it is trust to do particular things and and the promises that can therefore be made at particular times.”47 The importance of trust to them – depends on the dynamic between underlines the potential of data trusts as new, decision/policy-makers and participants. It is accountable institutions that can manage data influenced by the level of commitment to security and maximise the value of data. involve participants in collective decision- “Existing public services”, as described in the making by those holding the power to make aforementioned Nesta article, “will not be the decision. able to generate trust through their existing This goes beyond simply informing, for machineries, but can benefit greatly from more data sharing.” instance, and necessitates an involvement and empowerment of those taking part.48 This is Deliberative public engagement can be used visualised in the International Association for across all levels of government: local, regional, Public Participation Federation’s Spectrum of national and international. It can be used Public Participation (Figure 4).49 across all types of services, delivered by public, private or voluntary sectors. Moreover, it can help to inform, consult, involve or empower, alongside other forms of participation (e.g. opinion polls, written consultations, community development,

47 Mulgan & Straub, supra note 6 48 Arnstein, S.R., A Ladder of Citizen Participation, JAIP, Vol. 35, No. 4, July 1969 49 IAP2, Spectrum of Public Participation, reproduced with permission

[27]

Figure 4 IAP2’s spectrum of public participation

Increasing impact on the decision

Inform Consult Engage Coproduce Empower To provide the public with To obtain public feedback To work directly with the To partner with the public To place final decision balanced and objective on analysis, alternatives public throughout the in each aspect of the making in the hands of the information to assist them and/or decisions. process to ensure that decision including the public. in understanding the public concerns and development of problem, alternatives, aspirations are consistently alternatives and the

opportunities and/or understood and identification of the Participation goal Participation solutions. considered. preferred solution. We will keep you informed. We will keep you informed, We will work with you to We will look to you for We will implement what listen to and acknowledge ensure that your concerns advice and innovation in you decide. concerns and aspirations, and aspirations are directly formulating solutions and and provide feedback on reflected in the incorporate your advice how public input alternatives developed and and recommendations into influenced the decision. provide feedback on how the decisions to the

Promise to public to Promise public input influenced the maximum extent possible. decision.

When to use deliberative decision- public are often overstated and exaggerated, making and, for more complex or controversial decisions, are overshadowed by the costs of Ultimately it is a judgement as to when and ‘non-engagement.’ where a data trust uses deliberative approaches. We have indicated key points at For example, research findings from the which it would appear most useful, in this Environment Agency on “the experience of report and in the pilot-specific reports. Here two cities in trying to implement controlled we make some more general points about parking schemes… found that non- how to identify when deliberation is likely to engagement came with significant costs in the be valuable. form of delays and conflict. Without considering the true costs of not engaging it is Developing and instituting deliberative no wonder that engagement can seem approaches, as with any other form of expensive.”51 decision-making, has costs. These include: Extending this to a data trust, the risk of not • The time needed to plan and design a working deliberatively with the public and deliberative approach, including engaging stakeholders and not creating a mechanism the right stakeholders, slowing down that is trusted by the public and stakeholders decision-making means that they are less likely to give • Often increased direct costs compared to permission for their data to be used or other more direct forms of decision- accessed, negating the potential benefits of making50 data access. However, it is also important to consider the other side of the argument: what the cost of There are multiple ways for deliberative not applying deliberative approaches would approaches to be used. Making the wrong be. For example, the costs of engaging the choice can cost time and money in failed

50 Costs such as use of skilled practitioners to design/facilitate the process, but also in venues and any incentive payments for recruited participants 51 Anderson, E. et al, From fairy tale to reality: dispelling the myths around citizen engagement, undated

[28]

implementation. Decisions which tend to • Be committed to using the results and clear benefit from a deliberative approach include how it will use the results, and have the those which: authority to do so • Be clear about what is “up for grabs” – if • Require greater ownership of the key decisions have already been made and outcomes by stakeholders there is nothing to influence, a deliberative • Need to demonstrate or would benefit approach will be damaging to trust. from taking account of wider views, values, insights and experiences A deliberative approach exposes and asks • Still have aspects that are open to questions – its job is to drive better decisions formation, influence or change with the insights gained. If it is used without • Are contentious, have underlying or real integrity and impact then it is likely to be more conflict and/ or involve trade-offs which damaging to the process of building trust. benefit from a greater understanding of what is driving those issues and the Recommended deliberative methods underlying values There are a large number of deliberative • Are at an impasse, and would benefit from methods which would be relevant to the data wider perspectives to help break deadlock. trusts under consideration. The suitability of Decisions about which techniques will be these methods depends to a great extent on the purpose of the project and its scope (i.e. informed by: the number of citizens/groups/interests that • The context, purpose and values of the the data trust would represent, and therefore data trust and the stakeholders involved. what level of cost and energy would be • The available resources to apply to reasonable within a deliberative process to deliberative approaches. design it). Figure 5 shows a number of possible • The willingness of decision-makers to listen deliberative methods. to and take account of the views as a The choice of deliberative method(s) is contribution to their decision making, dependent on a number of factors, including • The decision being open to influence and the prospective number of participants, the change; and time that could be committed to the project, • The willingness of participants and the the available budget and probable benefit public to engage with the topic at hand. compared to more traditional forms of Finally in choosing a particular method, part of engagement. It is also important to point out the significance is the ‘message’ it sends. The that a combination of methods is possible; for method(s) used will be highly influential in the example, the use of face to face methods subsequent dynamic the data trust may then supplemented by online deliberation or a have with the immediate community Citizens’ Assembly and ongoing stakeholder or (public/stakeholders). public Advisory forums.

As well as when to use a deliberative approach it is worth reiterating when it is damaging or ineffective. To use a deliberative approach effectively an organisation needs to:

[29]

Figure 5 Map of deliberative methods52

Citizens’ Large scale continuing 1000 summits liaison and consultation s programmes, e.g. Pop up Deliberative virtual panels, regular democracy citizens’ conferences panels Deliberative stakeholder events

Online deliberations Numbers of 100 participants s Citizens’ Facilitated assembly stakeholder dialogues

Small scale continuing liaison Deliberative workshops Participatory Citizens’/stakeholder groups, e.g. local strategic planning advisory forums partnerships

Deliberative Deliberative Citizens’ mapping focus groups juries 10 s one off several months ongoing

Length of process

Annex 2 discusses these methods, and others, • In the Co-design stage, to develop criteria in more detail; not only in terms of their time or principles which the data trust will use commitments and participant numbers, but to ensure that the decisions it makes meets also in terms of their cost, and the strengths the needs and expectations of wider and challenges that they represent. stakeholders and the public; and to develop policy on distribution of value. As Each of the pilot reports discusses what might an example this might involve further be suitable for their cases, but some broad facilitated stakeholder dialogue workshops conclusions would be that deliberative work at combined with a citizens’ assembly or the following points will be necessary: deliberative stakeholder consultations with a wider group of stakeholders • At the Scope stage to align purpose and • In the Evaluate stage, as a way of ensuring values of the trust. For example, this should that the data trust continues to meet the involve, as a minimum, facilitated expectations of stakeholders and the workshops with the emergent data trust’s public, and is responsive to what is likely to stakeholders and some initial soundings be a rapidly changing context for data with the wider public through, for example, collection, synthesis, sharing and use. deliberative focus groups Advisory forums or reference panels will be useful to enable this, potentially augmented by less frequent citizens’ assemblies or juries.

52 National Consumer Council/Involve, Deliberative public engagement: nine principles, 2008

[30]

7. Summary of recommendations

The decision-making process should be A structured design process designed to meet three criteria for We have suggested a structured approach to accountability: designing a decision-making process for a data • trust, aligned to the trust lifecycle: Inclusivity (are all stakeholder interests represented) • Scope: agreement of the trust’s purpose • Responsiveness (are there mechanisms to and values ensure stakeholders’ interests are taken • Co-design: development of the trust’s into account?) policies on data provision, data use, and • Transparency (are the outcomes of the distribution of benefits, plus an approach trust’s decisions visible to stakeholders, to risk management and can they see how their interests have • Operate: design of the trust’s governance been balanced with other objectives?) structure, technical policy and Criteria for effectiveness include: enforcement mechanisms; and assessment of its funding and resource needs • Speed (can the trust make timely • Evaluate and retire: processes for decisions?) performance assessment and review, • Efficiency (is the cost of operating the trust transparency, and change control. proportionate to the benefits?) Provision should also be made for the • Scalability (can the decision-making trust’s closedown, including the process cope, as the volume of data and circumstances that should trigger number of uses grows?) closedown and a process for establishing There may be trade-offs between these whether those circumstances exist. criteria; there is no single right answer about While in practice the design work will be achieving the right balance. However, in iterative, the decisions made in each phase general, we would advise against decision- will depend on those made in preceding making processes that involve too many veto phases. points, which risk undermining effectiveness.

The design process may benefit from involving The value of deliberation an ‘honest broker’ who can coordinate negotiations and try to find mutually Deliberative methods can be an important acceptable solutions. accountability tool, enabling greater shared understanding of stakeholders’ issues, A good decision-making process concerns, views and values, and in particular helping resolve trade-offs. A good decision-making process must sustain stakeholders’ consent, which will partly be But deliberation should not be undertaken driven by their support for the trust’s purpose, lightly – it requires commitment, time and and partly by its accountability and resource, particularly for trusts operating in effectiveness. sensitive or complex areas. It requires genuine desire to hear the voices of wider stakeholders and reflect that in decision making.

[31]

It is particularly important to engage • Food waste: exploration of the scope to stakeholders in defining the purpose, values extend existing data sharing arrangements and criteria of the trust. into a trust format, based on an agreed common purpose, and seeking to expand Governance and technology, hand in the data gathered by the trust (for example hand to include sales data, data from adjacent sectors, or from international markets). Data trusts don’t solve governance complexity, although they can provide a More generally, we recommend the ODI planning structure. They are not an ‘out of the continues to work on data trusts from a box’ solution. Indeed, they are likely to be practical perspective, refining the concept and most needed when there are many data clarifying its place amongst the growing set of providers, many potential uses (good and data governance tools by describing what bad), and data are highly valued. problems it may solve in particular contexts.

Data trusts are likely to play a technical role. This need not mean actually hosting and storing data; it may mean providing a technical oversight function, including data verification and auditing

Technical solutions may be available to promote trust and ensure transparency. These do not in themselves remove governance challenges, but they may support both accountability and effectiveness, including by lowering the costs of transparency and responsiveness.

Next steps Across all three pilots, we recommend further work with stakeholders to refine the purpose of the pilot trust, draft statements of values, and progress detailed co-design work:

• GLA: definition of an overarching purpose by senior decision-makers, for deliberative testing and development with stakeholders, including the public, so that public benefit can be demonstrated • Illegal wildlife: formation of a working group with government, law enforcement and conservation groups to define a clear problem statement, agree purpose, and map other stakeholders in a potential trust

[32]

Annex 1. Summary of design questions

The table below represents a comprehensive set of questions that a trust’s decision-making process may need to address, regarding different stages in its lifecycle. Not all trusts will need to answer all these questions. The intention is that this table provides a checklist; a formative trust should consider which of these questions are relevant, and, for those that are, how it will answer them in its particular context, including how it will engage relevant stakeholders.

Scope What are the trust’s aims? What problems does it address? Why is a trust needed? Purpose What values and standards will it uphold? How will changes to purpose and values be proposed and how will Values they be considered? In what circumstances will the trust be wound up, and who decides if they are met? Co-design What data will be made available? How will the availability of data be communicated to potential users? Who can provide data, and how will they be incentivised? What criteria will determine who can access data and what uses are permissible? Data Provision What are the rights and duties of the parties to the trust? Data Use How will new providers, datasets, uses or users be judged in terms of their compatibility with the trust’s criteria? How and when can data be removed or disposed of? What are the enforcement mechanisms? How will the trust identify what enforcement action is needed? How will the trust promote data availability and use, and to whom? Operate What role (if any) will the trust play in: gathering, storing, processing, validating, securing data? Technical What role (if any) will the trust play in setting standards and ensuring interoperability? How will it ensure compliance with its technical rules? What is the governance structure? How will directors be appointed? What information will directors need? Who are the trust’s stakeholders and how will they be represented? What committees and advisory groups will be appointed, and how? How is the trust held to account? What external oversight will there Governance be? How will the trust handle complaints, resolve disputes and provide redress, including to third parties? How will independence be protected and demonstrated? How will ‘edge cases’ be considered?

[33]

Will the trust be responsible for legal and regulatory compliance, e.g. for establishing consent? Legal How will liability arising from the trust’s activities be allocated? What is the right legal/corporate/contractual model? What resources and skills will the trust need? How will the costs of the trust be covered, initially and as it scales? Can it make a profit? Is it an entity or a set of relationships? What staff will it need, if any? Resources Will the trust pay any external costs (of data providers, users or third parties)? What will the trust do if its resources are inadequate to achieve its purpose, comply with its rules or regulatory/legal requirements, or mitigate risks? Evaluate What are the trust’s measures of success? How will they be assessed and disclosed? Assessment How will the need for any changes be identified and assessed, in Reporting light of evaluation? How will stakeholders’ views be gathered during evaluation? Change control How will changes to the trust’s rules, processes or practices be Closedown proposed and assessed? How will breaches of the trust’s rules be assessed? What is the closedown procedure, and how can it be triggered?

[34]

Annex 2. Assessment of deliberative methods and techniques

Further to the general ‘map’ of deliberative techniques provided in the previous section, the table below provides a description of several relevant deliberative methods. It also gives an outline of their key strengths, and potential challenges in their implementation.

Method & description Strengths Challenges

Citizens’ /stakeholder advisory forums Participants asked to complete ‘homework’ between Meetings are usually quite short which can limit meetings and come prepared to deliberate, making deliberation Participants: 10-30 sitting as a committee to the best use of their time inform and advise decision making over an Because they are often not involved it is a challenge to extended period of time. Provides early warning of potential problems and a ensure insights reach decision-makers useful sounding board to test plans and ideas Cost: Low Long-term commitment from participants makes Events usually not expensive, but costs of Regular meetings over extended periods give recruiting and retaining participants difficult recruiting, supporting and rewarding participants a chance to get to know each other, participants can be high. aiding discussions Can appear exclusive to those who are not included

Time expense: Medium Citizens/stakeholders introduce a fresh perspective to Small number of people involved so statistically Minimum 3 months to set-up and run group. discussions, encouraging innovation significant data not generated Scale of the project and the level of expertise required can affect the time Citizen/stakeholder involvement increases Participants can become less representative over time; required. accountability in governance due to the transparency advisory groups may need to be renewed regularly of the process

Deliberative focus groups Works well with small groups in short amounts of time Limits on how much information can be presented and (when the topic is clearly focused and a specific absorbed in a limited time; can impact depth of Participants: 6-12 per group sharing views output has been identified) deliberation and attitudes on a subject, with a report produced and distributed to participants. High level of participant interaction due to the small Heavily dependent on a skilled facilitator size of the group Cost: Low-medium Easily dominated by one or two strong opinions Can lead to a greater understanding of how people think about issues Some participants may feel inhibited to speak

[35]

Method & description Strengths Challenges Generally not very high unless using random selection. May include incentives, venue Members can be specially recruited to fit Responses are not quantitative and so cannot be used hire, catering, etc. (demographic) profiles to gauge wider opinion

Time expense: Low Good for getting opinions from people who would not Usually 1-2 hrs. Time required to plan, be prepared to give written answers recruit participants, write up & respond to results. May require reading in advance.

Deliberative workshops and structured Very flexible and versatile method, allowing for A representative sample of the population is important dialogues creativity in meeting the needs of the project for the evidence to be generalisable

Participants: 8-12 in group discussion to The same workshop design can be used in a variety Conclusions are not always clear and collective explore an issue, challenge opinions and of locations, or with different groups develop informed conclusion(s). Open to manipulation: how discussions are framed; Large numbers of people involved in addressing a how the topic is introduced; the questions asked Cost: Low-medium single policy question without a large-scale event Stratified random selection can add Involves small numbers of people and therefore can’t significant costs. An incentive is sometimes Time and resources to consider an issue in-depth gather statistically significant data on opinions offered. Additional costs include venue and catering. Must sometimes reconvene. Discussing with others gives participants an insight Participants' views develop through deliberation; may into alternative perspectives mean that final views aren’t representative of wider Time expense: Variable public, who haven’t experienced deliberation A few hours or several days depending on Can build relationships between participants topic and intended outcome. It is a method that is rapidly acquiring increased social legitimacy and political buy-in

Facilitated stakeholder dialogues Deals well with conflict, can help address low trust Extremely reliant on the skills of a facilitator or mediator; can be expensive and time consuming Participants: a handful of people to several Ensures a balanced approach to decision-making, hundred, defining the problem, devising allowing all voices to be heard The need for participation by all stakeholders can slow progress or even render it impossible

[36]

Method & description Strengths Challenges methods and creating solutions, mainly Develops jointly-owned and implemented solutions, through workshops and similar meetings. often preventing the need for legal challenge or Challenging to ensure communication between litigation stakeholder representatives and their constituencies Cost: Medium Costs can increase for expert facilitation and Highly flexible and can be applied at all levels of A risk that organisational and individual positions may numerous meetings. government. not be explicitly acknowledged

Time expense: Medium-high Good in controversial or contested contexts; dialogue May only highlight areas of agreement without other Most effective over a long period of time due is one of the few practicable options once a conflict parts of the picture; problematic for campaigning to the slow process of building relationships has reached a certain point organisations for which positions are important and trust between groups.

Citizens’ jury A recognised and proven method, with institutional Usually requires participants to take in large amounts legitimacy of information; can be challenging to present this in Participants: 12-24; representative of the engaging ways demographic, deliberating on an issue Can involve people who have previously not engaged (generally one clearly framed question). with an issue The issue/decision can be highly specific

Cost: High Designed to deliver clear, agreed outputs, The framing of the question, and the evaluation of the Average: £15,000-£20,000 for two days; interrogating issues and experts/evidence results, can be very ‘top-down’ recruitment of jurors, venue hire/catering, facilitation, Per diem honorarium for jurors, Useful for controversial or sensitive policy issues that High cost accommodation and travel. require careful weighing up of options Small sample of citizens involved, although this should Time expense: Low Direct citizen input through extended deliberation and be highly representative of the demographics of the Mostly take place over two days, mainly focused discussion given area because of time and cost constraints. Impartial, specific and objective decisions, delivered through a verdict

Citizens’ assembly Can explore diverse perspectives on complex issues Recruiting a representative group of people at this and reach consensual recommendations scale can be challenging and expensive

[37]

Method & description Strengths Challenges Participants: 50-250 citizens deliberating an When run on a large scale they can bring a diverse Assemblies are very intensive and resource- issue, or issues, of local, regional or national array of opinions and experiences into one event demanding processes importance. Participants usually selected to create a ‘mini-public’ (broadly representative Combines learning phase with deliberation; can help Running a Citizens Assembly is a highly complex of the population). understand, develop and change initial views process requiring significant expertise

Cost: High Brings decision-makers face-to-face with consumers Risks being seen as a publicity exercise if not followed Includes recruitment of participants, with lived experience of the issues by real outcomes facilitation, participant expenses, planning, communication and promotion. Can be a high profile process and provide an opportunity to draw wider attention to an issue Time expense: Medium Takes place over several weekends. Offers policy makers an insight on public opinion on a contested issue

Citizens’ panel/ Community panel Can be sponsored and used by a partnership of local Needs considerable staff support to establish and agencies maintain the panel Participants: 500-5,000 in a representative, consultative body of local residents, taking Allows for the targeting of specific groups if large Can exclude non-native speakers and/or certain part in a rolling programme of research and enough residents who do not feel comfortable participating in consultation. this way Allows surveys or other research to be undertaken at Cost: Medium short notice Responses to surveys often reduce over time, Depends on the size of the Panel, the particularly among young people methods in which the members are Useful in assessing local service needs & priorities consulted and frequency of consultation. Should not be the only form of engagement Can determine appropriateness of developments Time expense: Medium within the area Time needed to keep the Panel database up to date, recruit new participants, and to run Can track local sentiments over time & analyse consultations.

[38]

Method & description Strengths Challenges

Distributed dialogue Ability to engage a large number of stakeholders and Distributed dialogues can take a long time to organise, lay people in different locations not suitable when fast action is needed Participants: various, participating in dialogue events organised by interested Insights into concerns and aspirations in different Encouraging others to run workshops can be time parties (rather than centrally) in different localities around the same issues consuming and resource intensive areas and media (including online). Indicates how priorities and opinions differ in different The commissioning body retains little control of how Cost: Low-medium areas or between different groups discussions are framed or facilitated in practice Planning and promotion; materials for workshops; communications. Depends on Can be a cost effective way of enabling large Data collected can be inconsistent scope and breadth. Costs contained by local numbers to participate, as costs and organisational groups running their own events. tasks are decentralised Difficult to ensure inclusiveness and transparency of local/stakeholder-led dialogues Time expense: Varies Opportunities for continuous engagement integrated Distributed dialogues take place at different into the process The process may produce contradictory or inconsistent times, organised by participants. data Gives a high degree of autonomy and control to citizens

Deliberative mapping Gives consumers and experts the opportunity to learn Can only be used with quite small groups from each other and work together Participants: 20-40 citizens and topic experts Findings can be inconclusive if there are difficulties consider complicated issues. Can show how Useful for understanding the differences between finding common ground support for a proposed action is weighed expert and public assessments of options against economic, social, ethical and The results of the process can be contradictory, scientific criteria. Good for dealing with complicated issues where a leaving decision-makers without clear guidance range of different considerations must be balanced Cost: Medium Can be high cost, with considerable time demands on Numerous meetings and event costs, Can demonstrate values and concerns behind public expert participants facilitation, expenses of citizens & experts. preferences and acceptability of options Often difficult to ensure that experts buy in to the Time expense: Medium-high process and engage with public as equals

[39]

Method & description Strengths Challenges Requires several months for numerous Can deliver greater legitimacy for decisions and Highly specialised expertise in running this process meetings and workshops. information about public preferences towards policy options. Often ineffective in building better relationships between groups Experts take a more active role than in many engagement processes, but are prevented from dominating

Participatory strategic planning Effective in involving the public in meaningful The demand of reaching agreement between policy/action planning, particularly on complex and stakeholders can weaken the ambition of policy Participants: 5-50 in a community, coming technical issues recommendations together in explaining how they would like their community or organisation to develop Brings public and expert stakeholders together Requires active participation of all stakeholders over the next few years. throughout the whole process A cost-effective way of enabling a diverse group to Cost: Medium identify common ground and reach agreement Often difficult to ensure that experts buy in to the Usually two trained and experienced process and engage with public as equals facilitators for two-day event. Can deliver clear, realistic policy recommendations Requires trained and experienced facilitators Time expense: Low Flexible and applicable to multiple settings A two-day event with recommended follow- Requires all major stakeholders to be present in the up after 6 months. Works for people with auditory/visual preferences room

Participants often find process & outcome inspiring

Online deliberations Can be a cost-effective and time-efficient alternative Can be difficult or impossible to replicate the depth of to face-to-face workshops deliberation in face-to-face engagement Participants: 1-500+, using software emulating face-to-face methods. Different An effective way of presenting complex and technical May alienate people with a lack of IT skills, people who templates allow participants to brainstorm information don’t/can’t access or navigate the internet ideas, identify issues, prioritise solutions, or comment on consultation documents. People can participate in their own time and at their If not carefully planned, online consultations can own convenience generate unmanageable amounts of material Cost: Medium

[40]

Method & description Strengths Challenges Online consultation cuts costs for venues Game design can engage participants interactively Written communication can be a barrier for some and postage. Costs include design, set up, already marginalised groups and incentivising participation. Allow large numbers of people to contribute equally Any perceived complexity, such as registration, can be Time expense: Medium Can reach people who are unlikely to respond to a barrier to participation Most exist for a few months to discuss a traditional engagement methods current event or situation. Anonymity can encourage open discussion

Allows information gathering and giving without the constraints that group size or travel

Pop up democracy Can help reach out to people that might not otherwise Many installations tend to be aesthetic in nature, rather participate than transformational Participants: 500+. Creates local participation spaces, enabling Utilise a range of possible tools to gather people's Many pop-up interventions lack a framework for experimentation. Residents can reimagine views and ideas to tackle specific issues measuring success spaces and existing power structures. Can reinvigorate interest in political institutions by Limited emphasis on collecting or disseminating data Cost: Variable tailoring spaces to people's needs and interests or feeding back to the community (during/after) Depends on scope and timeframe. Using empty venues creatively can reduce costs. Use spatial and cultural context of the site to build the Some projects demarcate, rather than bridge, the gap Costs include staff and props. core of the project around it, responding to specific between practitioners ("creators") and participants local needs and enhancing local assets (“receivers”) Time expense: Variable As little as one day and as long as needed.

[41]