AI/ML Finding a Doing Machine
Total Page:16
File Type:pdf, Size:1020Kb
Digital Readiness: AI/ML Finding a doing machine Gregg Vesonder Stevens Institute of Technology http://vesonder.com Three Talks • Digital Readiness: AI/ML, The thinking system quest. – Artificial Intelligence and Machine Learning (AI/ML) have had a fascinating evolution from 1950 to the present. This talk sketches the main themes of AI and machine learning, tracing the evolution of the field since its beginning in the 1950s and explaining some of its main concepts. These eras are characterized as “from knowledge is power” to “data is king”. • Digital Readiness: AI/ML, Finding a doing machine. – In the last decade Machine Learning had a remarkable success record. We will review reasons for that success, review the technology, examine areas of need and explore what happened to the rest of AI, GOFAI (Good Old Fashion AI). • Digital Readiness: AI/ML, Common Sense prevails? – Will there be another AI Winter? We will explore some clues to where the current AI/ML may reunite with GOFAI (Good Old Fashioned AI) and hopefully expand the utility of both. This will include extrapolating on the necessary melding of AI with engineering, particularly systems engineering. Roadmap • Systems – Watson – CYC – NELL – Alexa, Siri, Google Home • Technologies – Semantic web – GPUs and CUDA – Back office (Hadoop) – ML Bias • Last week’s questions Winter is Coming? • First Summer: Irrational Exuberance (1948 – 1966) • First Winter (1967 – 1977) • Second Summer: Knowledge is Power (1978 – 1987) • Second Winter (1988 – 2011) • Third Summer (2012 – ?) • Why there might not be a third winter! Henry Kautz – Engelmore Lecture SYSTEMS Winter 2 Systems • Knowledge is power theme • Influence of the web, try to represent all knowledge – Creating a general ontology organizing everything in the world into a hierarchy of categories – Successful deep ontologies: Gene Ontology and CML Chemical Markup Language • Indeed extreme knowledge – CYC and Open CYC – IBM’s Watson Upper ontology of the world Russell and Norvig figure 12.1 Properties of a subject area and how they are related Ferrucci, D., et.al. “Building Watson: An overview of the DeepQA Project”, AI Magazine, Fall 2010. Watson • IBM Watson question answering computer system – DeepQA project • Famous for winning a Jeopardy contest against Ken Jennings (KenJen) and Brad Rudder, 2 champions – Jeopardy is difficult, ambiguous questions and average of 3 seconds to ring in – Excluded A/V questions and special instruction questions • Applications: tax preparation, teaching assistant, medicine, … • “ … Create general purpose, reusable natural language processing (NLP) and knowledge representation and reasoning (KRR) that can exploit as-is natural language resources and as- is structured knowledge rather than curate task-specific knowledge resources.” Distribution of Jeopardy Questions KenJen TREC Text Retrieval Contest https://trec.nist.gov/ • to encourage research in information retrieval based on large test collections; • to increase communication among industry, academia, and government by creating an open forum for the exchange of research ideas; • to speed the transfer of technology from research labs into commercial products by demonstrating substantial improvements in retrieval methodologies on real-world problems; and • to increase the availability of appropriate evaluation techniques for use by industry and academia, including development of new evaluation techniques more applicable to current systems. DeepQA Technology • Used more than 100 different techniques • Not specific to Jeopardy but specific to QA • Technological Principles involved: – Massive parallelism – Many Experts – Pervasive confidence estimates – Integrate shallow and deep knowledge – leveraging many loosely formed ontologies – No one algorithm dominates DeepQA High-Level Architecture DeepQA - wikipedia Deep QA/Watson • Uses Hadoop for distributed computing • Watson can process 500 gigabytes/sec, equivalent of 1 million books/sec • Resources included taxonomies and ontologies including DBpedia, WordNet and YAGO CYC • Goal to assemble a comprehensive ontology and knowledge base that spans the basic concepts and rules about how the world works hoping to capture common sense and implicit/tacit knowledge. • Combat brittle AI systems • Began in 1984 by Doug Lenat • Also OpenCyc (CYC) and ResearchCYC (Cyc) Thanks Wikipedia CYC factoids • Ontology (2017) 1.5 Million terms: – 416K collections e.g., fish and types of fishing • CYC KB of common-sense rules and assertions were largely hand-written 24.5 million taking >1K person years! • Uses community of agents (blackboard) architecture • ”every tree is a plant”, “plants die eventually”, asked whether trees die it will infer correctly CYC Elements • Individual items #$BillClinton or #$France (CycL) • Collections #$Tree-ThePlant – contains all trees • Functions produce new terms from given ones. #$FruitFn arg will return collection of fruits for a provided type or collection of plants Applications of CYC • Pharmaceutical Term Thesaurus Manager/Integrator (Glaxo) • Terrorism Knowledge Base • MathCraft – 6th grade level help in understanding math Lots of Opinions on CYC/cyc/Cyc but true to GOFAI NELL • Never Ending Language Learner, Tom Mitchell, Carnegie Mellon University • We will never truly understand machine or human learning until we can build computer programs that, like people: – learn many different types of knowledge or functions, – from years of diverse, mostly self-supervised experience, – in a staged curricular fashion, where previously learned knowledge enables learning further types of knowledge, – where self-reflection and the ability to formulate new representations and new learning tasks enable the learner to avoid stagnation and performance plateaus. • T. Mitchell, et.al. AAAI 2015 NELL-2 • Given: – an initial ontology defining categories (e.g., Sport, Athlete) and Binary relations (e.g., AthletePlaysSport(x,y)), – approximately a dozen labeled training examples for each category and relation (e.g., examples of Sport might include the noun phrases “BaseBall” and “soccer”), – the weB (an initial 500 million weB pages from the ClueWeb 2009 collection (Callan and Hoy 2009), and access to 100,000 Google API search queries each day), – occasional interaction with humans (e.g., through NELL’s puBlic weBsite http://rtw.ml.cmu.edu); NELL -3 • DO: – Run 24 hours/day, forever, and each day: • 1. read (extract) more beliefs from the web, and remove old incorrect beliefs, to populate a growing Knowledge base containing a confidence and provenance for each belief, • 2. learn to read better than the previous day. NELL-4 Knowledge Base uses Blackboard Architecture Mitchell, et.al. (2015) NELL Results High confidence indicates that one of NELL’s modules assigns a 0.9 confidence to the belief or multiple modules propose the belief Mitchell, et.al. 2015 Alexa • Siri, Google Assistant too AWS Technologies Semantic Web 29 Semantic Web Big Data, API/Platform, & M2M Intersection •What is Semantic Web? •A collaborate movement led by W3C to promote common data format aimed at converting the current web dominated by unstructured and semi-structured document into a “web of data” •Components of Semantic Web • Resource Description Framework (RDF) • RDF Schema • Web Ontology Language (OWL) •Implications of Semantic Web • Link heterogeneous data sources on web • Drive “meaning” of information on the web • Enable machine-readable web • Enable intelligent services • Improve quality of search, targeted ads, etc. 30 Early Adopters of Semantic Web Synergy with Effortless Customer Experince schema.org project: a joint effort of Google, Yahoo!, and Microsoft documents the concepts and attributes that these three search engines will look for when Web publishers include Semantic Web data on their sites. That is RDF like data. Typical Applications Early Adopters • Optimized search engine • Knowledge graph • Knowledge management • Speech recognition • Content management • Customer/product driven analytics • Customer acquisition & retention • Predictive analytics for security • Visual discovery - 30% increase in traffic (Best Buy). 15% increase in CTR (Yahoo!). External Adopters - Semantic Technology In The Wild • Department of Defense • US Intelligence Agencies • Library of Congress • Wikipedia - DBpedia • NASA • Amdocs • BestBuy • BBC • MITRE Internet Semantic Web Linked Data Semantic Web – Background • Tim Berners-Lee 1999 • “I have a dream for the Web [in which computers] become capable of analyzing all the data on the Web – the content, links, and transactions between people and computers. A ‘Semantic Web’, which should make this possible, has yet to emerge, but when it does, the day-to-day mechanisms of trade, bureaucracy and our daily lives will be handled by machines talking to machines. The ‘intelligent agents’ people have touted for ages will finally materialize” • The dream is beginning to emerge and it’s all based on W3C Standards • Alias: Web of Data Web 3.0 • Linked Data Machine Readable Web • Our goal is to understand this concept, it’s formats and technologies and begin to explore some of the uses and power of this technology. SemTech Terms • Resource Description Framework (RDF) • RDF Schema (RDFS) • OWL (Web Ontology Language) • SPARQL • TripleStore - GraphDatabase • InFerring • Metadata • RDFa • MicroFormats • Ontology Modeling Today – Data and Information • Web Search: q Keyword Based “Results” q Provides a List of Links q Answer to Question? • Business Applications: q Query Based results q Range of information