Landing the Plane: Data Ethics Where It Hits the Ground

Landing the Plane: Data Ethics Where it Hits the Ground

Katherine O’Keefe Noviembre 2019

#damaweek2019 Katherine O’Keefe, PhD [email protected] @okeefekat

• Head of Training and Research, Ethicist, Castlebridge • Data Governance and Privacy Consulting and Training • Data Protection Compliance: worked with clients in a variety of sectors, from telco to healthcare to charities, on consulting and training engagements since 2013. • Lectures on Data Ethics and Data Privacy at the Law Society of Ireland • Presents internationally on Data Ethics, Data Governance, Data Privacy • Co-author, Ethical Data and Information Management, Concepts, Tools and Methods. Kogan Page 2018 Agenda

• Overview of concepts • Information Ethics as an Information Quality System • Risk management, Information management practices • Using GDPR Compliance to focus Ethical Data use beyond compliance • Methods to align ethics with Information Governance Introduction

Why Ethics? What’s that I hear Gartner says Digital when it’s at Ethics is going to be a home? strategic trend next year… “With great power comes great responsibility” This crisis is real. Those of us who believe in tech for good must ask ourselves a fundamental question: What kind of world to we want to live in? -- Tim Cook © 2018 Castlebridge

Ethical Risk Appetite

Ethics are a Risk issue and a Management issue for an “Organization leaders who espouse a organization. belief that expectations for ethical behaviour have become part of the ‘risk environment’ for top management. So, arguably, both stakeholders and managers view ‘ethical risk’ management as important.” -- Peter Young Data meets ethics: Some recent concerns

• Equality and accountability: “Machine learning is like money laundering for bias. It's a clean, mathematical apparatus that gives the status quo the aura of logical inevitability.” - Maciej Cegłowski • Life and death consequences: “We kill people based on metadata” – Michael Hayden • “Disrupting” democracy and civil rights protections: “We’re sorry this happened” – Mark Zuckerberg • Effects on the environment: “If its current rate of growth continues, block chain technologies will generate as much carbon emissions worldwide as the whole of the United States.” – Giovanni Buttarrelli “In one case the MPs highlighted, a student who received a Christmas gift from their parent was told the interaction on Facebook meant they were ineligible to be classed as estranged, which would have qualified them for a maintenance loan without means-testing.”

Example: “Smart Cities”

Some Possible Benefits Some Possible Harms • Smart Grid to Reduce emissions • Uneven coverage: Benefits • Improve efficiency of transport richest, poorest are harmed. • Water supply: realtime data • Resources distributed unfairly, about health and operation of increasing injustice system • Surveillance and loss of privacy • Better targeting of resources • “Predictive” policing creates a vicious cycle of increased bias Some questions: “Smart Cities”

• What data is being collected? • For what benefit? • How is it governed? • Who has access to what data? • Who owns the sensors? • Who profits? From what? • How can you maximize benefit? Minimize harms? © 2018 Castlebridge

So act as to treat humanity, Act only on that maxim both in your own person, by which you can at and in the person of every the same time will that other, always at the same it should become a time as an end, never universal law simply as a means. The Second Formulation of the Categorical Imperative in Business Ethics

“That is, each of these stakeholder groups has a right not to be treated as a means to some end, and therefore must participate in determining the future direction of the firm in which they have a stake.”

Basic Ethical Principles What are we talking about? What is a “right” action?

Consequentialism Deontology

• The action that results in the most • The action that is taken according to good or best overall consequences. one’s duty to do the right thing. • Whether an act is right or wrong (Rules based.) depends on the consequences of the • “Do it because it’s the right thing to act. do.” • Outcomes are more important. • Motive is more important.

@cbridgeinfo Approaches to Ethics . . .

The Common Good: Act considering the best results for the greater community

• Utilitarianism: Act to maximize human welfare or well-being, minimize harm. • Justice based Ethics: Act to treat all human beings equally, increase justice. • Rights Based Ethics: Act to protect the moral rights of those affected by action. • Virtue based Ethics: Act in a way that would make you a better person.

@cbridgeinfo Based on “Ethics and Information Systems: The Corporate Domain”, MIS Quarterly, Vol.23, No.1 (March 1999) Normative Theories of Ethics

Stockholder/Shareholder Theory Stakeholder Theory Social Contract Theory

 Identify Stakeholders  Reject Actions that are  Maximising Shareholder Value fraudulent/deceptive, dehumanise  Determine Legal and Moral Rights  The sole purpose of a firm is to people, involve invidious make money for its shareholders of Each discrimination  Seek best balance of interests  Maximise Profits  Eliminate options that reduce  Create mechanisms for defining & welfare of Society’s Members  Conform to laws and regulations respecting stakeholders interests  Develop mechanisms to identify  Avoid fraud and deception  Develop ‘algorithm’ for balancing dehumanising processing interests  Consider tangible and intangible aspects of “well-being”

Ethical Focus: Ethical Focus: Ethical Focus: Meet needs of Social Welfare and Compliance with Letter of law Balancing Interests Justice

@cbridgeinfo Are Ethical Principles too abstract to guide business and decision making around data?

“Action is indeed the sole medium of expression for ethics” Jane Addams (1860-1935) Understanding business and innovation in a framework of care and relationship: Ethics of Care 1. Identifying a need and 2. Innovating to address that need, Responsible 3. Having the skill to do so, and 4. Thinking of the possible risks and how it can Development be abused. Relational Ethical Viewpoints: Ethics of Care as Guidance for Responsible Development Finding a solution the problem that needs solving, not building a thing to find a problem for.

Attentiveness Responsibility Competence Responsiveness • proclivity to • a willingness to • the skill of • consideration of become aware of respond and take providing good the position of need care of need and successful others as they see care it and recognition of the potential for abuse in care

What is your framework for determining “the right thing to do”?

How do people in your organization make decisions about “the right thing to do”?

“Nowadays people know the price of everything and the value of nothing” Oscar Wilde Compliance / Ethics "Not everything that is legally compliant and technically feasible is morally right.“

“Ethics come before, during, and after the law.” .. it fills the gaps where Law is interpreted.

-- Giovanni Buttarelli (ICDPPC conference, 23 October 2018) https://www.flickr.com/photos/andrewhurley/6254409229 - Image Source: Andrew Hurley, Flickr)

Ethical Information Management

Compliance Privacy, “Persönlichkeitsrecht”; Respecting the Person & Ethics

The principle which protects personal writings and all other personal productions, not against theft and physical appropriation, but against publication in any form, is in reality not the principle of private property, but that of an inviolate personality. (Warren and Brandeis)

“Everyone has the right to the free development of his personality insofar as he does not violate the rights of others or offend against the constitutional order or the moral code.” (Article 2 paragraph 1 of the German Constitution qtd. In Bartvan der Sloot “Privacy as Personality Right”) “However, the Central Bank shares the view of the international financial regulatory community that rules and regulations should be reinforced by an additional focus on the organisational cultures of regulated firms. The boards and the senior managers of regulated firms are responsible for ensuring that organisational cultures are focused on enhancing consumer protection and ensuring that risks are well understood and managed. Accordingly, how best to achieve this has become an increasing area of focus for regulators” Philip Lane, Governor of Central Bank of Ireland, discussing Behaviour and Culture Report at Oireachtas Committee 4th October 2018 Preventing Risk to the Individual: Recital 75, 76 GDPR Example: “EdTech” Education Analytics The “Right to Privacy” as a Fundamental Human Right

“That the individual shall have full protection in person and in property is a principle as old as the common law; but it has been found necessary from time to time to define anew the exact nature and extent of such protection.”

“Recent inventions and business methods call attention to the next step which must be taken for the protection of the person, and for securing to the individual . . . the right "to be let alone."

Louis Brandeis and Samuel Warren. “The Right to Privacy.” Harvard Law Review 4(1891):197-219

@cbridgeinfo EDPS : Considerations for Data Ethics Regulations to guide 1. Future-oriented regulation of data Societal processing and respect for the rights to Ethics privacy and to data protection.

Accountability 2. Accountable controllers who determine and Governance personal information processing.

Privacy by 3. Privacy conscious engineering and design Design and of data processing products and services. by Default

Human 4. Empowered individuals. Dignity, Autonomy Not reinventing the wheel . . . The Belmont Report (1979) Basic Ethical Principles for Medical Research 1. Respect for Persons: Consent must be 1. individuals should be treated as autonomous informed and agents freely given 2. persons with diminished autonomy are entitled to protection. 2. Beneficence Risk-based approach 1. do not harm 2. maximize possible benefits and minimize possible harms. Balancing 3. Justice fundamental rights

Respect for Human Dignity and Autonomy Not reinventing the wheel . . . The Menlo Report (2012) Ethical Principles Guiding Information and Communication Technology Research

1. Respect for Persons: 1. Informed Consent 2. Beneficence Consent must be 1. Identification of Potential Harms and Benefits informed and freely given 2. Balancing Risks and Benefits 3. Mitigation of Realized Harms Risk-based 3. Justice: Fairness and Equity approach 4. Respect for Law and Public Interest. 1. Compliance Balancing 2. Transparency and Accountability fundamental rights Respect for Human Dignity and Autonomy Some examples of Principles and Guidelines in Data Ethics

European Commission World Economic Forum IEEE Ethical AI Respect for Human Autonomy Prevention of Harm Fairness Explicability Based on “Ethics and Information Systems: The Corporate Domain”, MIS Quarterly, Vol.23, No.1 (March 1999) Normative Theories of Ethics (In Business)

Stockholder/Shareholder Theory Stakeholder Theory Social Contract Theory

 Identify Stakeholders  Reject Actions that are  Maximising Shareholder Value  Determine Legal and Moral fraudulent/deceptive, dehumanise people, involve invidious  The sole purpose of a firm is to Rights of Each discrimination make money for its shareholders  Seek best balance of interests  Eliminate options that reduce  Maximise Profits  Create mechanisms for defining welfare of Society’s Members  Conform to laws and regulations & respecting stakeholders interests  Develop mechanisms to identify  Avoid fraud and deception dehumanising processing  Develop ‘algorithm’ for balancing interests  Consider tangible and intangible aspects of “well-being” Ethical Focus: Ethical Focus: Ethical Focus: Meet needs of Social Welfare and Compliance with Letter of law Balancing Interests Justice This is the focus required by GDPR © 2018 Castlebridge

The aim proposed here for any organization is for everybody to gain – stockholders, employees, suppliers, customers, community, the environment – over the long term.

W. Edwards Deming “The New Economics

This Photo by Unknown Author is licensed under CC BY-NC-SA Ethics, Governance, and Quality Management Information Ethics and Quality Systems Management

“QualityETHICS means doing it right when no one is looking.”

W. Edwards Deming

This Photo by Unknown Author is licensed under CC BY-NC-SA

Quality Characteristics of Ethical Data Use (1/2) Can you identify or measure the following outcomes?

• To what extent will it do good in society? Promote happiness? Utility • Does it solve a problem? Create stakeholder satisfaction? Is it useful?

Beneficience / non- • How well does it promote well being? Support the good of society? maleficence • Does it cause harm?

• Does it result in equal treatment of people or even increased equality? Justice / Fairness: Consider algorithmic accountability

Verity / Non- • What is the level of integrity, truthfulness, accuracy, of your representation? deceptiveness • Does it meet requirements for transparency? Quality Characteristics of Ethical Data Use (2/2) Can you identify or measure the following outcomes?

• How much does your process respect or infringe on people’s ability Autonomy to choose an action for themselves?

Privacy / Non- • How intrusive is your process into the personal life, relationships, invasiveness correspondence, or communication of people?

• What harms may result if the issue you are addressing with this Necessity process are left unaddressed? • Is the processing necessary to meet your desired outcome?

• Could you achieve the same objective with more limited impact on Proportionality individuals and their autonomy? Or with a more positive impact? Ethical Innovation by Design: Quality Engineering How are ethical values communicated and instilled in the organization you work in?

What are those ethics? Information Fiduciaries Data Stewards ‘Information Fiduciaries, assume Data stewards ‘manage data special duties with respect to the assets on behalf of others and information they obtain in the in the best interests of the course of the relationship.’ organization’ ‘Information fiduciaries have (McGilvray, 2008) special duties to act in ways that do not harm the interests of the people whose information they collect, analyze, use, sell, and distribute.’

(Balkin, 2016) Ethics and the Zachman Framework Data Action Location Actor Event Motivation Executive Inventory Process Distribution Responsibility Timing Motivation Scope & Identification Identification Identification Identification Identification Identification Context Business Inventory Process Distribution Responsibility Timing Definition Motivation Business Manager Definition Definition Definition Definition Definition Concepts Architect Inventory Process Distribution Responsibility Timing Motivation System Logic Representation Representation Representation Representation Representation Representation Engineer Inventory Process Distribution Responsibility Timing Motivation Technology Specification Specification Specification Specification Specification Specification Physics Technician Inventory Process Distribution Responsibility Timing Motivation Tool Configuration Configuration Configuration Configuration Configuration Configuration Components Enterprise Inventory Process Distribution Responsibility Timing Motivation Enterprise Instantiation Instantiation Instantiation Instantiation Instantiation Instantiation Data Action Location Actor Event Motivation

Executive Inventory Process Distribution Responsibility Timing Motivation Scope & Identification Identification Identification Identification Identification Identification Context

Business Inventory Process Distribution Responsibility Timing Motivation Business Manager Definition Definition Definition Definition Definition Definition Concepts

Architect Inventory Process Distribution Responsibility Timing Motivation System Logic Representation Representation Representation Representation Representation Representation

Engineer Inventory Process Distribution Responsibility Timing Motivation Technology Specification Specification Specification Specification Specification Specification Physics

Technician Inventory Process Distribution Responsibility Timing Motivation Tool Configuration Configuration Configuration Configuration Configuration Configuration Components

Enterprise Inventory Process Distribution Responsibility Timing Motivation Enterprise Instantiation Instantiation Instantiation Instantiation Instantiation Instantiation Data Action Location Actor Event Motivation

Executive Inventory Process Distribution Responsibility Timing Motivation Scope & Identification Identification Identification Identification Identification Identification Context

Business Inventory Process Distribution Responsibility Timing Motivation Business Manager Definition Definition Definition Definition Definition Definition Concepts

Architect Inventory Process Distribution Responsibility Timing Motivation System Logic Representation Representation Representation Representation Representation Representation

Engineer Inventory Process Distribution Responsibility Timing Motivation Technology Specification Specification Specification Specification Specification Specification Physics

Technician Inventory Process Distribution Responsibility Timing Motivation Tool Configuration Configuration Configuration Configuration Configuration Configuration Components

Communicating Assessing Ethical Ethics by Design Values & Impact Principles

 Impact Assessments  Help you identify, prioritise, and mitigate risks  Help you do it BEFORE you have created the problem  Help you implement Privacy by Design Principles  Help you communicate Values & Principles Ethical Impact Assessments: A Scalable Method

Might look a little bit familiar . . . Development

10 Steps for Ethical Impact Assessment and Development Adapted from Danette McGilvray, 10 Steps to Quality Data. Define Business Need and Approach Example: Define Business Need and Approach Analyse Information Environment

Four Compass Points of an Ethical Impact Assessment Assess Ethical Quality Characteristics and Impact

This Photo by Unknown Author is licensed under CC BY-SA-NC A few questions…

• Does this process / design manipulate people? • How invasive is this? • Are we building “models” of people that impinge upon their autonomy and personhood? • What are some “edge cases” where this could go really wrong? Testing Outcomes against First Principles

What is the outcome of this process? 1. Does it preserve human dignity? Does it enhance human dignity? 2. Does it preserve the autonomy of the human? 3. Is the processing necessary and proportionate? 4. Does it uphold the common good?

Does it violate any of these? Identify Root Causes of Quality Issues / Risks

• Five Whys Analysis • Fishbone Diagrams

“We don’t want to overmonetize. If anything, one of the things we’ve done is to is to make sure that the utility is maximized for the users” ~ Sataya Nadella Utility versus Invasiveness – Understanding Impact

H H Utility

L Invasiveness How to use…

• On Post-it Note each participant ranks the proposed processing as follows: • Utility – Scale of 1-10 (10 being high) and their “margin for error” (8 +/- 2) • Invasiveness – Scale of 1-10 (10 being high) and “margin for error” (5 +/- 4) • Should be done SILENTLY after a discussion (to allow for dissent) • Facilitator plots all scores to map “Acceptability Space” Utility versus Invasiveness – Understanding Impact

10 U=7 +/-3 10 I=4 +/- 3

U=5 +/-3 Consensus

Utility I= 5+/- 4 Zone Consensus zone = average U=2 +/-3 Utility/Invasiveness tradeoff I=5 +/- 3 Actions: • Increase communication of utility, • reduce invasiveness, • abandon 1 Invasiveness Develop Improvement Plans to address root causes of risks / issues Ethics and Governance

“Create constancy of purpose toward improvement of product and service…”

“The lack of cultural integration between competitive values and moral values plagues many companies.” - Stephen P Feldman Best efforts are essential. Unfortunately, best efforts, people charging this way and that way without guidance of principles, can do a lot of damage.

W. Edwards Deming “Out of the Crisis”

This Photo by Unknown Author is licensed under CC BY-NC-SA Financial Perspective

Customer Internal Business Vision and Strategy Perspective Process (Efficiency)

Learning and Growth

Mistake #1: Driving UP from Data/Process level

Get bogged down in detail, no over- arching vision of what “Future” will be Mistake #2: Driving DOWN from Top

Plan fails on first contact with reality Critical Sustainability Risk: Misalignment Ideal State: Alignment of “Tone from the Top”

Principles are embedded into Policies, Procedures, and Processes. “Unplanned for” things addressed through Principles 1. Establish a clear “tone from the top” for data ethics in general. 2. Implement data governance to create situational modifiers for individual behaviour – Checks and Balances, not Tickboxes! 3. Implement training to ensure “Ethic of Individual” is aligned with “Ethic of Organisation” 4. Ensure appropriate metrics and KPIs are in place to support cascade of accountability from top down 5. Ensure ALIGNMENT of metrics and messages from Principles to Processes 6. Take a “Balanced Scorecard” view of the investment in data protection, and look for opportunities to improve efficiencies in the organisation 7. Ensure a transition from “Project” to “Operational” modes so that key controls are updated because people want to not because they are told they have to.