Fundamentals of a Regulatory System for Algorithm-Based Processes
Total Page:16
File Type:pdf, Size:1020Kb
Prof. Dr. Martini – Regulatory System for Algorithm-based Processes Prof. Dr. Mario Martini Chair of Public Administration, Public Law, Administrative Law and Euro- pean Law, German University of Administrative Sciences Speyer, Director of the program area “Transformation of the state in the digital age” at the German Research Institute for Public Administration, Member of the Data Ethics Commission appointed by the German Feder- al Government FUNDAMENTALS OF A REGULATORY SYSTEM FOR ALGORITHM-BASED PROCESSES – Expert opinion prepared on behalf of the Federation of German Consumer Organisations (Verbraucherzentrale Bundesverband)* – 1/5/19 1 Prof. Dr. Martini – Regulatory System for Algorithm-based Processes Recommended citation: Martini, Fundamentals of a Regulatory System for Algorithm-based Processes, Speyer, 2019. All contents of this report, in particular texts and graphics, are protected by copyright. Un- less expressly stated otherwise, the copyright belongs to Prof. Mario Martini, Chair of Public Administration, Public Law, Administrative Law and European Law, German University of Administrative Sciences Speyer. 2 Prof. Dr. Martini – Regulatory System for Algorithm-based Processes A. OUT OF CONTROL? ALGORITHMS AS NAVIGATORS IN THE DIGITAL WORLD ___________ 6 B. REGULATORY INSTRUMENTS _______________________________________________ 8 I. Transparency requirements ______________________________________________________________ 8 Ex-ante obligations to mark and inform ___________________________________________________ 9 a) Status quo under existing legislation___________________________________________________ 9 b) Recommendation for future legislation _______________________________________________ 10 2. Ex-post information _________________________________________________________________ 11 a) Obligation to state reasons _________________________________________________________ 11 Legal status quo ________________________________________________________________ 11 Recommendation for future legislation ______________________________________________ 12 cc) Limitations of obligatory justifications ______________________________________________ 12 b) Right of access to underlying data and the decision basis _________________________________ 13 aa) Status quo under existing legislation ________________________________________________ 13 bb) Right to appropriate algorithmic conclusions as a recommendation for future legislation? ____ 14 3. Expansion and publication of the data protection impact assessment _________________________ 16 II. Content control _______________________________________________________________________ 18 1. Regulation mechanisms ______________________________________________________________ 18 a) Approval procedure in sensitive areas of application ____________________________________ 18 b) Specifying regulations for the legitimacy of profiling, in particular the requirement of mathematical-statistical validity of algorithm-based decisions ________________________________ 19 c) Anti-discrimination guardianship: Obligation to ensure lawful and, in particular, non-discriminatory decision results _______________________________________________________ 21 aa) Operator obligations under discrimination law _______________________________________ 22 (1) Extension of the General Equal Treatment Act (AGG) _______________________________ 22 (2) Technical measures of protection against indirect discrimination ______________________ 23 bb) Quality requirements for procedural justice __________________________________________ 24 cc) Obligation to implement a risk management system and name a responsible person ________ 24 d) Supervision by the authorities during operation of algorithm-based systems _________________ 25 aa) Control of decision results, specifically through audit algorithms _________________________ 26 bb) Authorities’ rights of access and inspection, in particular access rights/interfaces for tests and external controls _________________________________________________________ 26 (1) Interface for tests and external controls __________________________________________ 27 (2) Obligation of service providers to keep records ____________________________________ 28 2. Institutional design __________________________________________________________________ 29 a) General supervisory authority? ______________________________________________________ 30 b) Supporting service unit ____________________________________________________________ 31 3 Prof. Dr. Martini – Regulatory System for Algorithm-based Processes c) Additional non-authority control mechanisms __________________________________________ 32 d) Interim conclusion ________________________________________________________________ 34 III. Ex-post protection _____________________________________________________________________ 35 1. Liabilities __________________________________________________________________________ 35 a) Allocation of the burden of proof ____________________________________________________ 35 b) Strict liability? ____________________________________________________________________ 36 2. Legal protection ____________________________________________________________________ 36 a) Competitors’ powers to issue written warnings _________________________________________ 36 b) Consumer associations’ right to take legal action and creation of arbitration boards ___________ 37 IV. Self-regulation _____________________________________________________________________ 37 Auditing ___________________________________________________________________________ 38 2. Algorithmic Responsibility Code ________________________________________________________ 38 C. REGULATORY THRESHOLDS: CATALOGUE OF CRITERIA TO SPECIFY WHEN CERTAIN REGULATORY MEASURES APPLY ________________________________________________ 40 I. Defining content-related standards _______________________________________________________ 41 1. Regulatory thresholds in general _______________________________________________________ 41 a) Fixed thresholds (e. g. number of employees; turnover) __________________________________ 41 b) Number of data subjects (potentially) involved _________________________________________ 43 c) Fundamental rights sensitivity as the connecting factor to the purpose of protection __________ 44 aa) Impacts on fundamental rights other than the right to informational self-determination ______ 45 bb) Risk of excluding consumers from important areas of life - Relevance of participation and availability of alternatives _____________________________________________________________ 46 cc) Specially protected categories of personal data _______________________________________ 47 d) Interim conclusion ________________________________________________________________ 47 2. Regulatory thresholds in specific areas – Identification of applications requiring regulation ________ 48 3. Combined solution __________________________________________________________________ 49 a) Risk factors ______________________________________________________________________ 49 aa) Types of damage ________________________________________________________________ 50 bb) Extent of expected damage _______________________________________________________ 50 b) (Qualitative) specification of risk thresholds ___________________________________________ 51 II. Procedural instruments of specification – limits to the delegation of regulatory power ____________ 51 1. Specification of provisions in national law ________________________________________________ 53 a) Constitutional framework for the specification of provisions – delegated regulations as an instrument for specifying provisions _______________________________________________________ 53 4 Prof. Dr. Martini – Regulatory System for Algorithm-based Processes b) Limits of delegation of regulatory powers by law to private actors, in particular to an expert committee in technology and ethics _______________________________________________________ 55 2. EU legal framework for the specification of provisions ______________________________________ 59 a) Delegated acts of the Commission ___________________________________________________ 59 b) EU Guidelines ____________________________________________________________________ 61 III. Summary: basic structure of a normative risk threshold system for algorithm-based processes _____ 62 D. REGULATORY COMPETENCE: IMPLEMENTATION OF REGULATORY PROPOSALS IN A MULTI-TIERED SYSTEM – SUMMARY _________________________________________ 65 I. Transparency obligations _______________________________________________________________ 66 1. Labelling obligations (“if”) and content-related information obligations (“how”) _________________ 66 2. Obligation to publish a comprehensive impact assessment __________________________________ 68 II. Content control _______________________________________________________________________ 69 1. Instruments ________________________________________________________________________ 69 2. Regulatory thresholds ________________________________________________________________ 70 3. Institutional design __________________________________________________________________ 71 III. Liability and legal protection ____________________________________________________________ 72 IV. Self-regulation ________________________________________________________________________ 73 V. Table with regulatory proposals for exemplary applications __________________________________ 73 E. LIST OF REFERENCES _______________________________________________________