Advancing the Future of AI IBM Research AI February 2021 Introduction Trusted AI Scaling and Automating AI
Total Page:16
File Type:pdf, Size:1020Kb
Advancing the Future of AI IBM Research AI February 2021 Introduction Trusted AI Scaling and Automating AI Dr. John R. Smith, IBM Fellow Dr. Karthikeyan N. Ramamurthy Dr. Lisa Amini, Director IBM T. J. Watson Research Center IBM T. J. Watson Research Center IBM Research Cambridge 2 Introduction Dr. John R. Smith, IBM Fellow IBM T. J. Watson Research Center 3 IBM Research AI AI Focus Areas include: • Global footprint with an integrated AI strategy Bot • Network of academic partnerships in AI Customer Agent Language Speech Code b c a d Time series Hardware Chemistry Dublin ... more Zurich Yorktown Cambridge Almaden Tokyo Haifa Delhi Bangalore MIT - IBM A I L A B Narrow AI Broad AI General AI Emerging Disruptive and pervasive Revolutionary Deep learning Neuro-symbolic AI True neuro-AI Single-task, Trusted AI capable of learning Cross-domain learning single-domain, with much less data and reasoning high accuracy Automating AI development Broad autonomy with Requires large and deployment moral reasoning amounts of data Reduced-precision and Wetware? CPU and GPU analog HW We are here now AI is making incredible impact Example Applications (Accelerated discovery) What’s Next (Advances in AI foundations) Knowledge Ingestion Neuro-symbolic and Reasoning Reasoning www.research.ibm.com/artificial- www.research.ibm.com/covid19/deep-search/ intelligence/vision/#neurosymbolic Deep Search (PDF Corpora) Logical Neurons Chemical Reaction Trusted AI Prediction www.research.ibm.com/artificial- intelligence/trusted-ai/ https://rxn.res.ibm.com Uncertainty Quantification RXN for Chemistry Cloud-based Auto AI Autonomous Labs https://www.ibm.com/cloud/watson- studio/autoai https://rxn.res.ibm.com/rxn/robo-rxn RoboRXN (AI-Driven Synthesis) AutoAI January 2021 IBM Research – Advancing AI for Science 6 Trusted AI Dr. Karthikeyan N. Ramamurthy IBM T. J. Watson Research Center 7 AI is powering critical workflows and trust is essential loan customer employment quality control processing management 8 IBM Research / February 21, 2021 / © 2021 IBM Corporation Multiple factors are placing trust in AI as a top client priority brand reputation increased regulation complexity of AI deployments focus on social justice 9 IBM Research / February 21, 2021 / © 2021 IBM Corporation What does it take to trust a decision made by a machine? is it fair? is it accurate? did anyone tamper with it? is it easy to understand? is it accountable? is it transparent? 10 IBM Research / February 21, 2021 / © 2021 IBM Corporation “A decision aid, no matter how sophisticated or ‘intelligent’ it may be, may be rejected by a decision maker who does not trust it, and so its potential benefits to system performance will be lost.” —Bonnie M. Muir, psychologist at University of Toronto 11 IBM Research / February 21, 2021 / © 2021 IBM Corporation “The toughest thing about the power of trust is that it’s very difficult to build and very easy to destroy.” —Thomas J. Watson, Sr., CEO of IBM 12 IBM Research / February 21, 2021 / © 2021 IBM Corporation Trust Verify 13 IBM Research / February 21, 2021 / © 2021 IBM Corporation No shortcuts 14 IBM Research / February 21, 2021 / © 2021 IBM Corporation No shortcuts in problem specification problem Take advice from a panel owner diverse problem data of diverse voices stakeholders specification understanding data engineer data preparation deployment and monitoring AI modeling operations engineer data scientist evaluation model validator 15 IBM Research / February 21, 2021 / © 2021 IBM Corporation No shortcuts in data understanding and preparation data preparation measurement sampling construct space observed space raw data prepared data space space data preparation social bias population bias bias temporal bias data poisoning threatens threatens threatens internal construct validity external validity validity 16 IBM Research / February 21, 2021 / © 2021 IBM Corporation No shortcuts in modeling 17 IBM Research / February 21, 2021 / © 2021 IBM Corporation No shortcuts in modeling initial model final model pre-processing pre-processed model training post-processing training dataset dataset 18 IBM Research / February 21, 2021 / © 2021 IBM Corporation No shortcuts in modeling general robustness adversarial robustness fairness explainability initial model final model pre-processing pre-processed model training post-processing training dataset dataset data augmentation invariant risk minimization post hoc correction data sanitization gradient shaping post hoc defense bias mitigation pre-processing bias mitigation in-processing bias mitigation post-processing disentangled representations directly interpretable models post hoc explanations 19 IBM Research / February 21, 2021 / © 2021 IBM Corporation Various routes to serve clients Consulting – Trust capabilities in IBM Cloud Pak for Data • Explainability Fundamental Early • Fairness Research Access • Drift detection – Open source toolkits • AI Explainabilty 360 • AI Fairness 360 Open IBM Cloud Source Pak for Data • Adversarial Robustness Toolbox – Early access to enhanced editions of toolkits 20 IBM Research / February 21, 2021 / © 2021 IBM Corporation Overview of trustworthy AI topics AI FactSheets 360 consulting human-computer interaction lending transparency specification elicitation transparent documentation and eliciting societal values and preferences from policymakers are critical for AI governance workforce FACT DART FreaAI VerifAI Uncertainty Quantification 360 testing uncertainty quantification health domain domain expertise all elements of trust should be tested and reported with error bars AI Explainability 360 AI Fairness 360 Adversarial Robustness 360 Trusted Generation 360 these are social issues too issuessocial are these ecosystem explainability fairness robustness generation causality and constraint-aware learning are important in their own right and also as foundations for the pillars of trust above topics and assets should not be viewed only through a technical lens; lens; technical a through only viewed be not should assets and topics Causallib Diffprivlib broader impacts government affairs government causal modeling constraint-aware learning 21 IBM Research / February 21, 2021 / © 2021 IBM Corporation Explanations Use master pages to ensure – With the variety of layout solutions, one or more layouts should work to showcase your content. consistency throughout your – Using this template will help avoid overly dense presentation. pages and fonts that are unreadable. This will present a more unified look across presentations. – Master layouts are grouped by background color: black and light gray. – Each master group includes 32 page designs for you to choose from. 22 IBM Research / February 21, 2021 / © 2021 IBM Corporation Open Source AI Explainability 360 Supporting diverse and rich – 8 unique techniques from IBM Research explanations. • data vs. model • global vs. local • direct vs. post hoc http://github.com/ibm/aix360 – LIME and SHAP – 2 explainability metrics – Extensive industry tutorials to educate users and http://aix360.mybluemix.net practitioners – Interactive demo 23 IBM Research / February 21, 2021 / © 2021 IBM Corporation AI Explainability Enhanced Edition Early access offering – New state-of-the-art explanation methods from IBM Research – Interactive explanation – Better support for text – Better support for regression 24 IBM Research / February 21, 2021 / © 2021 IBM Corporation Fairness 25 IBM Research / February 21, 2021 / © 2021 IBM Corporation Open Source AI Fairness 360 The most comprehensive – Comprehensive set of fairness metrics toolkit for handling bias in • Group fairness machine learning. • Individual fairness – 12 state-of-the-art bias mitigation algorithms • Pre-processing http://github.com/ibm/aif360 • In-processing • Post-processing – Extensive industry tutorials to educate users and http://aif360.mybluemix.net practitioners – Interactive demo 26 IBM Research / February 21, 2021 / © 2021 IBM Corporation AI Fairness Enhanced Edition Early access offering – New state-of-the-art bias mitigation algorithms from IBM Research and the external community – Support for natural language processing – Bias mitigation and verification for individual fairness – Support for transfer learning – Fair generation – Protected attribute extraction 27 IBM Research / February 21, 2021 / © 2021 IBM Corporation Adversarial robustness https://art-demo.mybluemix.net 28 IBM Research / February 21, 2021 / © 2021 IBM Corporation No shortcuts in evaluation, deployment and monitoring problem Take advice from a panel owner diverse problem data of diverse voices stakeholders specification understanding data engineer data preparation deployment and monitoring AI modeling operations engineer data scientist evaluation model validator 29 IBM Research / February 21, 2021 / © 2021 IBM Corporation No shortcuts in governance Predictive Performance Data Scientist Model Validator AI Ops Engineer AI Factsheets 360 - https://aifs360.mybluemix.net/ Recap of the Toolkits AI Fairness 360 AI Explainability 360 Adversarial Robustness 360 FactSheets 360 aif360.mybluemix.net aix360.mybluemix.net art-demo.mybluemix.net aifs360.mybluemix.net Most comprehensive open source Most comprehensive open source Most comprehensive open source Extensive website describing toolkit for detecting & mitigating toolkit for explaining ML models & toolkit for defending AI from attacks research effort to foster trust in AI bias in ML models: data • Supports 10+ frameworks by increasing transparency and • 70+ fairness metrics • 8