DOCSLIB.ORG

Black-Box Models from Input-Output Measurements

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Black-Box Models from Input-Output Measurements BÐack¹b ÓÜ ÅÓ deÐ× fÖÓÑ ÁÒÔÙعÓÙØÔÙØ Åea×ÙÖeÑeÒØ× ÄeÒÒaÖØ ÄjÙÒg DeÔaÖØÑeÒØ Óf EÐecØÖicaÐ EÒgiÒeeÖiÒg ÄiÒkÓÔiÒg ÍÒiÚeÖ×iØݸ ËE¹58½ 8¿ ÄiÒkÓÔiÒg¸ ËÛedeÒ ÏÏÏ: hØØÔ:»»ÛÛÛºcÓÒØÖÓкi×ݺРiÙº ×e EÑaiÐ: ÐjÙÒg@i×ݺÐiÙº×e ÇcØÓb eÖ ¾¸ ¾¼¼½ ERTEK REGL NIK A UT OL OMATIC CONTR LINKÖPING ÊeÔ ÓÖØ ÒÓº: ÄiÌÀ¹ÁËY¹Ê¹¾¿6¾ FÓÖ Øhe ½8Øh ÁEEE ÁÒ×ØÖÙÑeÒØaØiÓÒ aÒd Åea×ÙÖeÑeÒØ ÌechÒÓÐÓgÝ CÓÒfeÖeÒce¸ BÙdaÔ e×ظ ¾¼¼½ ÌechÒicaÐ ÖeÔ ÓÖØ× fÖÓÑ Øhe AÙØÓÑaØic CÓÒØÖÓÐ gÖÓÙÔ iÒ ÄiÒkÓÔiÒg aÖe aÚaiÐabÐe bÝ aÒÓÒÝÑÓÙ× fØÔ aØ Øhe addÖe×× fØÔºcÓÒØÖÓкi×ݺÐiÙº×eº Ìhi× ÖeÔ ÓÖØ i× cÓÒØaiÒed iÒ Øhe ¬Ðe ¾¿6¾ºÔdfº IEEE Instrumentation and Measurement Technology Conference Budapest, Hungary, May 21–23, 2001 Black-box Models from Input-output Measurements Lennart Ljung Div. of Automatic Control Link¨oping University SE-58183 Link¨oping, Sweden email: [email protected] Abstract – A black-box model of a system is one that does not use any par- “successful in the past”. ticular prior knowledge of the character or physics of the relationships in- volved. It is therefore more a question of ”curve- fitting” than ”modeling”. This paper deals with Black-box models for dynamical sys- In this presentation several examples of such black-box model structures will be given. Both linear and non-linear structures are treated. Relation- tems, for which inputs and outputs can be measured. We shall ships between linear models, fuzzy models, neural networks and classical both deal with linear and non-linear systems. non-parametric models are discussed. Some reasons for the usefulness of these model types will also be given. First, in Section II we list the basic features of black-box mod- eling in terms of a simple example. In Section III we outline Ways to fit black box structures to measured input-output data are described, as well as the more fundamental (statistical) properties of the resulting mod- the basic estimation techniques used. Section IV deals with els. the basic trade-offs to decide the size of the structure used (es- sentially how many parameters are to be used: the “fineness of I. INTRODUCTION: MODELS OF DIFFERENT COLORS approximation”). So far the discussion is quite independent of the particular structure used. In Section V we turn to particu- At the heart of any estimation problem is to select a suitable lar examples of linear black-box models and to issues how to model structure. A model structure is a parameterized family choose between the possibilities, while VI similarly deals with of candidate models of some sort, within which the search for non-linear black-box models for dynamical systems. a model is conducted. II. BLACK-BOX MODELS: BASIC FEATURES A basic rule in estimation is not to estimate what you already know. In other words, one should utilize prior knowledge and To bring out the basic features of a black-box estimation prob- physical insight about the system when selecting the model lem, let us study a simple example. Suppose the problem is g ´Üµ; ½ Ü ½ structure. It is customary to color-code – in shades of grey – to estimate an unknown function ¼ . The ob- Ý ´k µ Ü the model structure according to what type of prior knowledge servations we have are noise measurements at points k has been used: which we may or may not choose ourselves: Ý ´k µ=g ´Ü µ·e´k µ ¼ k ¯ White-box models: This is the case when a model is per- (1) fectly known; it has been possible to construct it entirely from prior knowledge and physical insight. How to approach this problem? One way or another we must decide “where to look for” g . We could, e.g., have the infor- ¯ Grey-box models: This is the case when some physical insight is available, but several parameters remain to be mation that g is a third order polynomial. This would lead to determined from observed data. It is useful to consider the – in this case – grey box model structure ¾ ½ two sub-cases: Ò g ´Ü; µ= · Ü · Ü · ::: · Ü ½ ¾ ¿ Ò (2) – Physical Modeling: A model structure can be built on physical grounds, which has a certain number of pa- =4 with Ò , and we would estimate the parameter vector rameters to be estimated from data. This could, e.g., be from the observations Ý , using e.g. the classical least squares a state space model of given order and structure. method. – Semi-physical modeling: Physical insight is used to sug- gest certain nonlinear combinations of measured data Now suppose that we have no structural information at all signal. These new signals are then subjected to model about g .8/01/$10.00 We would then©2001 still IEEE have to assume something about structures of black box character. it, e.g. it is an analytical function, or that it is piecewise con- ¯ Black-box models: No physical insight is available or stant or something like that. In this situation, we could still use used, but the chosen model structure belongs to fami- (2), but now as black-box model: if we assume g to be ana- lies that are known to have good flexibility and have been lytic we know that it can be approximated arbitrarily well by a 0-7803-6646- polynomial. The necessary order Ò would not be known, and We shall also generally denote all measurements available up we would have to find a good value of it using some suitable to time Æ by scheme. Æ Z (6) Note that there are several alternatives in this black-box situa- tion: We could use rational approximations: III. ESTIMATION TECHNIQUES AND BASIC ¾ ½ Ò PROPERTIES · Ü · Ü · ::: · Ü ½ ¾ ¿ Ò ´Ü; µ= g (3) ¾ Ñ ½ ½· Ü · Ü · ::: · Ü ·½ Ò·¾ Ò·Ñ ½ Ò In this section we shall deal with issues that are independent or Fourier series expansions of model structure. Principles and algorithms for fitting mod- els to data, as well as the general properties of the estimated Ò X models are all model-structure independent and equally well g ´Ü; µ= · cÓ×´` ܵ· ×iÒ´` ܵ ¼ ` ½ ¾` ¾ (4) applicable to, say, linear ARMAX models and Neural Network =½ ` models, to be discussed later in this paper. Alternatively, we could approximate the function by piece- A. Criterion of Fit wise constant functions, as illustrated in Figure 1. We shall It suggests itself that the basic least-squares like approach is 1.5 Ý ´Øj µ a natural approach, even when the predictor ^ is a more general function of : 1.4 Æ ^ = aÖg ÑiÒ Î ´; Z µ Æ 1.3 Æ (7) 1.2 where 1.1 Æ X ½ Æ ¾ Î ´; Z µ= kÝ ´Øµ Ý^´Øj µk Æ (8) 1 Æ Ø=½ 0.9 We shall also use the following notation for the discrepancy 0.8 between measurement and predicted value 0.7 ´Ø; µ=Ý ´Øµ Ý^´Ø; j µ 1 2 3 4 5 6 7 8 9 10 " (9) Fig. 1. A piece-wise constant approximation. This procedure is natural and pragmatic – we can think of it ´Øµ Ý^´Øj µ as “curve-fitting” between Ý and . It also has several in Section VI return to a formal model description of the pa- statistical and information theoretic interpretations. Most im- rameterization in this figure. portantly, if the noise source in the system is supposed to be e´Øµg It is now clear that the basic steps of black-box modeling are a Gaussian sequence of independent random variables f as follows: then (7) becomes the Maximum Likelihood estimate (MLE). Ý ´k j µ 1. Choose the “type” of model structure class. (For exam- If ^ is a linear function of the minimization problem ple, in the case above, Fourier transform, rational func- (7) is easily solved. In more general cases the minimization tion, or piecewise constant.) will have to be carried out by iterative (local) search for the 2. Determine the “size” of this model (i.e. the number of minimum: ´i·½µ ´iµ ´iµ Ò parameters, ). This will correspond to how “fine” the Æ ^ ^ ^ = · f ´Z ; µ (10) Æ Æ approximation is. Æ Î 3. Use observed data both to estimate the numerical values f where typically is related to the gradient of Æ , like the of the parameters and to select a suitable value of Ò. Gauss-Newton direction. See, e.g. [1], Chapter 10. We shall generally denote a model structure for the observa- It is also quite useful work with a modified criterion tions by Æ Æ ¾ Ï ´; Z µ=Î ´Z µ·Æ k k Æ Æ (11) Ý^´k j µ=g ´Ü ;µ k (5) Î with Æ defined by (8). This is known as regularization.It Ý ´k j µ ´iµ It is to interpreted so that ^ is the predicted value of the may be noted that stopping the iterations in (10) before the ´k µ observation Ý assuming the function can be described by minimum has been reached has the same effect as regulariza- the parameter vector . tion. See, e.g., [2]. !½ B. Convergence as Æ covariance matrix of this sensitivity derivative. This is a quite natural result. An essential question is, of course, what will be the properties of the estimate resulting from (8). These will naturally depend The result (14) - (15) is general and holds for all model struc- Æ on the properties of the data record Z . It is in general a dif- tures, both linear and non-linear ones, subject only to some ^ ficult problem to characterize the quality of Æ exactly. One regularity and smoothness conditions. They are also fairly nat- normally has to be content with the asymptotic properties of ural, and will give the guidelines for all user choices involved ^ Æ Æ as the number of data, , tends to infinity.
Load more

Recommended publications
  • Existing Cybernetics Foundations - B
    SYSTEMS SCIENCE AND CYBERNETICS – Vol. III - Existing Cybernetics Foundations - B. M. Vladimirski EXISTING CYBERNETICS FOUNDATIONS B. M. Vladimirski Rostov State University, Russia Keywords: Cybernetics, system, control, black box, entropy, information theory, mathematical modeling, feedback, homeostasis, hierarchy. Contents 1. Introduction 2. Organization 2.1 Systems and Complexity 2.2 Organizability 2.3 Black Box 3. Modeling 4. Information 4.1 Notion of Information 4.2 Generalized Communication System 4.3 Information Theory 4.4 Principle of Necessary Variety 5. Control 5.1 Essence of Control 5.2 Structure and Functions of a Control System 5.3 Feedback and Homeostasis 6. Conclusions Glossary Bibliography Biographical Sketch Summary Cybernetics is a science that studies systems of any nature that are capable of perceiving, storing, and processing information, as well as of using it for control and regulation. UNESCO – EOLSS The second title of the Norbert Wiener’s book “Cybernetics” reads “Control and Communication in the Animal and the Machine”. However, it is not recognition of the external similaritySAMPLE between the functions of animalsCHAPTERS and machines that Norbert Wiener is credited with. That had been done well before and can be traced back to La Mettrie and Descartes. Nor is it his contribution that he introduced the notion of feedback; that has been known since the times of the creation of the first irrigation systems in ancient Babylon. His distinctive contribution lies in demonstrating that both animals and machines can be combined into a new, wider class of objects which is characterized by the presence of control systems; furthermore, living organisms, including humans and machines, can be talked about in the same language that is suitable for a description of any teleological (goal-directed) systems.
    [Show full text]
  • Building the Observer Into the System: Toward a Realistic
    Building the Observer into the System: Toward a Realistic Description of Human Interaction with the World Chris Fields 243 West Spain Street Sonoma, CA 95476 USA fi[email protected] November 11, 2016 Abstract Human beings do not observe the world from the outside, but rather are fully embedded in it. The sciences, however, often give the observer both a “god’s eye” perspective and substantial a priori knowledge. Motivated by W. Ross Ashby’s state- ment, “the theory of the Black Box is merely the theory of real objects or systems, when close attention is given to the question, relating object and observer, about what information comes from the object, and how it is obtained” (Introduction to Cyber- netics, 1956, p. 110), I develop here an alternate picture of the world as a black box to which the observer is coupled. Within this framework I prove purely-classical analogs of the “no-go” theorems of quantum theory. Focussing on the question of identifying macroscopic objects, such as laboratory apparatus or even other observers, I show arXiv:1411.3405v6 [quant-ph] 10 Nov 2016 that the standard quantum formalism of superposition is required to adequately rep- resent the classical information that an observer can obtain. I relate these results to supporting considerations from evolutionary biology, cognitive and developmental psychology, and artificial intelligence. Subject Classification: 03.65.Yz; 04.60.Bc Keywords:black box; classicality; environment as witness; Landauer’s principle; pragmatic information; quantum Darwinism; separability; superposition; time 1 The theory of the Black Box is merely the theory of real objects or systems, when close attention is given to the question, relating object and observer, about what information comes from the object, and how it is obtained.
    [Show full text]
  • Black Box, Black Bloc” a Lecture Given at the New School in New York City on April 12, 2010 Alexander R
    “Black Box, Black Bloc” A lecture given at the New School in New York City on April 12, 2010 Alexander R. Galloway Facial recognition and autoblur software, Google Street View, 20 Rue de la Vicarie, Saint Brieuc, France. Of all the revivals in recent years--a period of history in which the revival itself has been honed to such a degree that it persists as mere “blank parody”--the revival of Hegel is the most startling, although certainly not for those involved. Hegelianisms of all shapes and sizes prevail today, from Catherine Malabou's dutiful reconstruction of the “plastic” dialectical transformations, to the hysterical antimaterialism of Slavoj Žižek and his convocation of the inescapable bind between the “determinate negation” and the “wholly Other,” from which explodes the terror of proletarian power. Is not Woody Allen's character Alvy Singer in Annie Hall the perfect summation of Žižek's political project: Okay I'm a bigot, but for the left! Or consider the unrepentant Hegelian Alain Badiou who stakes everything on being as a pure formalism that only ever realizes itself through the event, an absolute departure from the state of the situation. Only the Hegelian dialectic, and not the Marxist one, can snap back so cleanly to its origins like this, suggesting in essence that Aufhebung was always forever a spectralization and not a mediation in general, that in other words the 1 ultimate truth of the Hegelian dialectic is spirit, not negation or becoming or anything so usefully mechanical. The negation is thus revoked during synthesis, much more than it is resolved.
    [Show full text]
  • Opening the Black Box: the Contextual Drivers of Social Accountability
    Opening the Black B Opening the Black The Con new fronTiers of s oCial PoliCy Citizens across the world are calling for greater citizen-state engagement. Social accountability (SA) has steadily gained prominence for its intrinsic value as well as for its potential to bring about a range of development outcomes. But how and when does social T accountability support service delivery as well as institutional outcomes such as state ex T legitimacy and citizen trust? This book provides a framework to assess the contextual ual Drivers of drivers of social accountability effectiveness based on a definition of SA as the interplay of citizen action and state action, supported by civic mobilization, interface, and information. The proposed framework provides practitioners with a systematic way to assess context for SA in order to design, implement, and monitor SA approaches tailored to a specific context and issue. The book also applies the framework to challenging country contexts. In doing so, it addresses key knowledge gaps on how policy makers, practitioners, and development O institutions can best support development outcomes through social accountability and s o citizen engagement. x C ial “At last, we have in one place, a comprehensive picture of our current knowledge about Acc social accountability! This book presents cutting-edge analysis, provides a synthesis of the evidence, and advances our thinking on three key fronts. Foremost, the authors take oun the whole issue of context seriously, examine in micro detail the various contextual T factors that matter for social accountability work, what we know about them, and how abili they unfold in a variety of empirical cases.
    [Show full text]
  • The Cybernetic Brain
    THE CYBERNETIC BRAIN THE CYBERNETIC BRAIN SKETCHES OF ANOTHER FUTURE Andrew Pickering THE UNIVERSITY OF CHICAGO PRESS CHICAGO AND LONDON ANDREW PICKERING IS PROFESSOR OF SOCIOLOGY AND PHILOSOPHY AT THE UNIVERSITY OF EXETER. HIS BOOKS INCLUDE CONSTRUCTING QUARKS: A SO- CIOLOGICAL HISTORY OF PARTICLE PHYSICS, THE MANGLE OF PRACTICE: TIME, AGENCY, AND SCIENCE, AND SCIENCE AS PRACTICE AND CULTURE, A L L PUBLISHED BY THE UNIVERSITY OF CHICAGO PRESS, AND THE MANGLE IN PRAC- TICE: SCIENCE, SOCIETY, AND BECOMING (COEDITED WITH KEITH GUZIK). THE UNIVERSITY OF CHICAGO PRESS, CHICAGO 60637 THE UNIVERSITY OF CHICAGO PRESS, LTD., LONDON © 2010 BY THE UNIVERSITY OF CHICAGO ALL RIGHTS RESERVED. PUBLISHED 2010 PRINTED IN THE UNITED STATES OF AMERICA 19 18 17 16 15 14 13 12 11 10 1 2 3 4 5 ISBN-13: 978-0-226-66789-8 (CLOTH) ISBN-10: 0-226-66789-8 (CLOTH) Library of Congress Cataloging-in-Publication Data Pickering, Andrew. The cybernetic brain : sketches of another future / Andrew Pickering. p. cm. Includes bibliographical references and index. ISBN-13: 978-0-226-66789-8 (cloth : alk. paper) ISBN-10: 0-226-66789-8 (cloth : alk. paper) 1. Cybernetics. 2. Cybernetics—History. 3. Brain. 4. Self-organizing systems. I. Title. Q310.P53 2010 003’.5—dc22 2009023367 a THE PAPER USED IN THIS PUBLICATION MEETS THE MINIMUM REQUIREMENTS OF THE AMERICAN NATIONAL STANDARD FOR INFORMATION SCIENCES—PERMA- NENCE OF PAPER FOR PRINTED LIBRARY MATERIALS, ANSI Z39.48-1992. DEDICATION For Jane F. CONTENTS Acknowledgments / ix 1. The Adaptive Brain / 1 2. Ontological Theater / 17 PART 1: PSYCHIATRY TO CYBERNETICS 3.
    [Show full text]
  • Second Order Cybernetics 31/08/2008 10:08
    SECOND ORDER CYBERNETICS 31/08/2008 10:08 Search Print this chapter Cite this chapter SECOND ORDER CYBERNETICS Ranulph Glanville, CybernEthics Research, UK and Royal Melbourne Institute of Technology University, Australia Keywords: actor, analogy, argument, automata, autopoiesis, behavior, biology, Black_Box, causality, circularity, coding, cognition, communication, computation, concept, consciousness, control, conversation, cybernetics, description, design, distinction, ecology, education, environment, epistemology, error, ethics, feedback, form, function, generosity, goal, homeostasis, idea, identity, individual, information, interaction, internet, knowing, language, learning, logic, management, mathematics, meaning, mechanism, memory, meta-, metaphor, moment, mutualism, network, object, observe, organizational_closure, oscillation, person, phenomenon, philosophical, physics, positivist, (post-)_modernism, praxis, process, program, psychology, purpose, quality, reality, reason, recursion, relative, responsibility, science, second_order_cybernetics, (self-)_reference, semiotics, social, structure, subject, support, system, theory, thinking, time, truth, understanding, variety, whole_(and_part) Contents 1. Introduction: What Second Order Cybernetics is, and What it Offers 2. Background—the Logical Basis for Second Order Cybernetics 3. Second Order Cybernetics—Historical Overview 4. Theory of Second Order Cybernetics 5. Praxis of Second Order Cybernetics 6. A Note on Second Order Cybernetics and Constructivism 7. Cybernetics, Second Order
    [Show full text]
  • Copying Black-Box Models Using Generative Evolutionary Algorithms
    Black-Box Ripper: Copying black-box models using generative evolutionary algorithms Antonio Barb˘ al˘ au˘ 1;∗, Adrian Cosma2, Radu Tudor Ionescu1, Marius Popescu1 1University of Bucharest, Romania 2University Politehnica of Bucharest, Romania ∗[email protected] Abstract We study the task of replicating the functionality of black-box neural models, for which we only know the output class probabilities provided for a set of input images. We assume back-propagation through the black-box model is not possible and its training images are not available, e.g. the model could be exposed only through an API. In this context, we present a teacher-student framework that can distill the black-box (teacher) model into a student model with minimal accuracy loss. To generate useful data samples for training the student, our framework (i) learns to generate images on a proxy data set (with images and classes different from those used to train the black-box) and (ii) applies an evolutionary strategy to make sure that each generated data sample exhibits a high response for a specific class when given as input to the black box. Our framework is compared with several baseline and state-of-the-art methods on three benchmark data sets. The empirical evidence indicates that our model is superior to the considered baselines. Although our method does not back-propagate through the black-box network, it generally surpasses state-of-the-art methods that regard the teacher as a glass-box model. Our code is available at: https://github.com/antoniobarbalau/black-box-ripper. 1 Introduction In the last couple of years, AI has gained a lot of attention in industry, due to the latest research developments in the field, e.g.
    [Show full text]
  • Black Box Activities for Grades Seven-Nine Science Programs And
    DOCUMENT RESUME ED 371 950 SE 054 618 AUTHOR Schlenker, Richard M., Comp. TITLE BlacE Box Activities for GradesSeven-Nine Science Programs and Beyond. A Supplement forScience 1, 2, &3. INSTITUTION Dependents Schools (DOD), Washington, DC.Pacific Region. PUB DATE 93 NOTE 157p. PUB TYPE Guides Classroom Use Instructional Materials (For Learner) (051) Guides Classroom Use Teaching Guides (For Teacher) (052) EDRS PRICE MF01/PC07 Plus Postage. DESCRIPTORS Astronomy; *Electric Circuits; InserviceEducation; Junior High Schools; *ScienceActivities; *Science Instruction; *Science Process Skills;Science Teachers; Scientific Methodology IDENTIFIERS *Black Box Activities;Dependents Schools; Handson Science ABSTRACT Many times science does not provideus with exact descriptions of phenomena or answers to questions but only allowsus to make educated guesses. Blackbox activities encourage thismethod of scientific thinking because the activity is performed insidl.a sealed container requiring thestudents to hypothesize on the contents and operation of the activity.This collection of hands-on activities was developed by inserviceteachers employed by the Department of Defense Dependents SchoolsPacific Region and were designed to help students achievethese realizations. The activities included are as follows: (1) "TheMystery Box";(2) "A Black Box"; (3) "Black Box ActiviLya Project in Inferring"; (4) Black Box Creating Your Own Constellations"; (5) "Black Box Shadows";(6) Black Box Activity Electrical Circui,t's"; (7) "Black Box Grades 6-9"; (7) "Ways of Knowing-Activity 2The Black Box"; (8) "Black Box Experiment"; (9) "The Hands DownMethod of Random Sampling of Icebergs for Polar Bears and IceHoles. A Black Box Activity"; (10) "Black Box Materials Identification";and (11) "Milk Carton Madness." Each activity contains an overview, purpose, objectives,procedures, brief discussion of the resultsobtained in the classroom, and additional suggestions for classroomuse.
    [Show full text]
  • The Black Box Society
    THE BLACK BOX SOCIETY The Secret Algorithms That Control Money and Information FRANK PASQUALE Cambridge, Massachusetts London, En gland !"#$ CONTENTS ! Introduction—The Need to Know ! " Digital Reputation in an Era of Runaway Data !# $ The Hidden Logics of Search %# & Finance’s Algorithms: The Emperor’s New Codes !'! % Watching (and Improving) the Watchers !&' ( Toward an Intelligible Society !)# Notes ""! Ac know ledg ments $'% Index $'* [T]here is one world in common for those who are awake, but [when] men are asleep each turns away into a world of his own. —Heracleitus Million- fuelèd, ൏ nature’s bon+ re burns on. But quench her bonniest, dearest ൏ to her, her clearest-selvèd spark Man, how fast his + redint, ൏ his mark on mind, is gone! Both are in an unfathomable, all is in an enormous dark Drowned. —Gerard Manley Hopkins 1 INTRODUCTION—THE NEED TO KNOW EVERYBODY KNOWS the story about the man crawling intently around a lamppost on a dark night. When a police of+ cer comes along and wants to know what he’s doing, he says he’s looking for his keys. “You lost them here?” asks the cop. “No,” the seeker replies, “but this is where the light is.” This bromide about futility has lately taken on a whole new meaning as a meta phor for our increasingly enigmatic technologies. There’s a noble tradition among social scientists of trying to clar- ify how power works: who gets what, when, where, and why., Our common life is explored in books like The Achieving Society, The Winner-Take-All Society, The Good Society, and The Decent Society.
    [Show full text]
  • Meaningful Explanations of Black Box AI Decision Systems
    The Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19) Meaningful Explanations of Black Box AI Decision Systems Dino Pedreschi,1 Fosca Giannotti,2 Riccardo Guidotti,2 Anna Monreale,1 Salvatore Ruggieri,1 Franco Turini1 1University of Pisa, 2ISTI-CNR Pisa, Italy [email protected], [email protected] Abstract health status, personality profile. Black boxes map user fea- tures into a class or a score without explaining why, be- Black box AI systems for automated decision making, often based on machine learning over (big) data, map a user’s fea- cause the decision model is not comprehensible to stake- tures into a class or a score without exposing the reasons why. holders, even to expert data scientists. This is worrying not This is problematic not only for lack of transparency, but also only for the lack of transparency, but also for the possible bi- for possible biases inherited by the algorithms from human ases hidden in the algorithms. Machine learning constructs prejudices and collection artifacts hidden in the training data, predictive models and decision-making systems based on which may lead to unfair or wrong decisions. We focus on the data, i.e., the digital records of human activities and deci- urgent open challenge of how to construct meaningful expla- sions, such as opinions, movements, preferences, judicial nations of opaque AI/ML systems, introducing the local-to- sentences, medical diagnoses, performance scores, etc. Con- global framework for black box explanation, articulated along sequently, ML models may reflect human biases and preju- three lines: (i) the language for expressing explanations in dices, as well as collection artifacts and sample selection bi- terms of logic rules, with statistical and causal interpretation; (ii) the inference of local explanations for revealing the deci- ases, possibly leading to unfair or simply wrong decisions.
    [Show full text]
  • ORGANIZATIONAL PATH DEPENDENCE: OPENING the BLACK BOX JO¨ RG SYDOW GEORG SCHREYO¨ GG Freie Universität Berlin JOCHEN KOCH European University Viadrina
    ஽ Academy of Management Review 2009, Vol. 34, No. 4, 1–000. ORGANIZATIONAL PATH DEPENDENCE: OPENING THE BLACK BOX JO¨ RG SYDOW GEORG SCHREYO¨ GG Freie Universität Berlin JOCHEN KOCH European University Viadrina To enable a better understanding of the underlying logic of path dependence, we set forth a theoretical framework explaining how organizations become path dependent. At its core are the dynamics of self-reinforcing mechanisms, which are likely to lead an organization into a lock-in. By drawing on studies of technological paths, we conceptualize the emergent process of path dependence along three distinct stages. We also use the model to explore breakouts from organizational path dependence and discuss implications for managing and researching organizational paths. The discourse on organizational innovation supposed to mean exactly? In organizational re- and change has become more complex. On the search the term is used mostly as a broad label one hand, there is an ever-increasing demand indicating all kinds of imprinting effects of the for more flexible or even fluid “new” organiza- past on organizational behavior (e.g., recently, tional forms. On the other hand, studies stress- Beckman & Burton, 2008). A closer examination ing organizational inertia and the historical im- quickly reveals that the predominant usage is printing of decision making (“history matters”) more metaphorical than theoretical in nature. A have come to the fore in management and or- clear specification is usually missing. This ganizational theory. There seems to be a means, at the same time, that no indicators are broadly shared feeling that we need to under- available that allow for examining whether or stand better how organizations can lose their not the organizational phenomena in question flexibility and become inert or even locked in.
    [Show full text]
  • Deep Learning Black Box Problem Master Thesis Research Project
    Deep Learning Black Box Problem Master Thesis Research Project Jabbar Hussain Subject: Information Systems Corresponds to: 30 hp Presented: VT 2019 Supervisor: Prof. Dr. Andreas Hamfelt Examiner: Prof. Dr. Pär Ågerfalk Department of Informatics and Media Jabbar Hussain Master Thesis Spring 2019 Abstract Application of neural networks in deep learning is rapidly growing due to their ability to outperform other machine learning algorithms in different kinds of problems. But one big disadvantage of deep neural networks is its internal logic to achieve the desired output or result that is un-understandable and unexplainable. This behavior of the deep neural network is known as “black box”. This leads to the following questions: how prevalent is the black box problem in the research literature during a specific period of time? The black box problems are usually addressed by so- called rule extraction. The second research question is: what rule extracting methods have been proposed to solve such kind of problems? To answer the research questions, a systematic literature review was conducted for data collection related to topics, the black box, and the rule extraction. The printed and online articles published in higher ranks journals and conference proceedings were selected to investigate and answer the research questions. The analysis unit was a set of journals and conference proceedings articles related to the topics, the black box, and the rule extraction. The results conclude that there has been gradually increasing interest in the black box problems with the passage of time mainly because of new technological development. The thesis also provides an overview of different methodological approaches used for rule extraction methods.
    [Show full text]