DOCSLIB.ORG

IBM Cloud Platform Overview for Data Scientists

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
IBM Cloud Platform Overview for Data Scientists IBM Cloud Platform Overview for Data Scientists Thomas Schaeck Distinguished Engineer, Watson Studio Data Science and ML in Enterprises Data must be cleaned and shaped for training need good data avoid bias Data … then models must be created Enterprises generate TONS OF DATA … requiring cataloging and governance SPSS …that must be hosted and monitored at scale … and trained on high performance compute to select an optimal model... IBM Cloud / Watson and Cloud Platform / © 2018 IBM Corporation 2 Why are enterprises struggling to capture the value of AI? Data Access Data Governance Skills Tools&Infrastructure • Data resides in silos • If the data isn’t • Data Science skills • Need an environment & difficult to access secure, self-service are in low supply and that enables quick isn’t a reality high demand experiments and real • Unstructured and outcomes external data wasn’t • Challenge • Nurturing new data considered understanding data professionals is • Discrete tools lineage and getting to challenging present barriers to 3 IBM Cloud / Watson and Cloud Platform / © 2018 IBM Corporation a system of truth productivity Watson AI and Data Watson Applications, solutions Healthcare Financial Logistics Virtual agent and services Services Studio & IoT Targeted solutions for Knowledge enterprise businesses Catalog Compare and Watson Machine Tone Personality Assistant Discovery API API API API Watson AI Comply Learning API Analyzer API Insight Cognitive building blocks for developers Nat Language Nat Language Visual Recognition Speech Document …more API API API Conversion API Understanding API Classifier Data Tools to prepare data Ingestion Storage Analytics Deployment Governance for AI and data persistence Cloud Integration Cognitive Micro-services DevOps Tooling Cloud Infrastructure A highly scalable, security Networking Compute Security Dedicated Compute Virtual Compute Object Storage …more enabled infrastructure Public Hybrid Private Collecting, organizing, analyzing Data à Training and deploying Models à Managing AI Ops Process/Business Owner Data Scientists/Engineers, Manage and oversee IBM + Business Partners Domain Experts models in production provide assets and services analyze data, gain insights, train models AI OpenScale Watson Studio Desktop Watson Studio Gallery Cloud/Local available soon and Community Cloud Manage AI Operations Create personal Projects Assets to get started with - Fairness Exchange Assets with teams through • Notebooks. Models, Dashboards, … - Explainability Cloud/Local Projects and/or Git Repos • Soon: Project Templates - Evolution Deploy Models to WML • - Health and Accuracy Community to communicate and connect End Users Create project benefit Contribute and collaborate from AI API API Assets Watson ML Reuse Assets Apps Watson Studio Projects Watson Knowledge Catalog Use Cloud/Local from Community Processes Models Cloud/Local Cloud/Local Devices Deploy, manage, retrain Models Deploy Web Sites Work with Data and AI Reuse Assets Share Enterprise Assets Deploy Notebooks, Scripts Feed- • Process/cleanse Data from Catalog • Data ... Health and Accuracy back • Create/run Notebooks • Connections • Analyze/visualize Data • Notebooks Use models • Create Dashboards • where they Models Customer’s Repo • Train AI/ML/DL Models • Dashboards are needed • … Export • Soon: Project Templates and Build Process Publish Assets Model as a team or individual. • to Catalog … Consume models e.g feed Intuitive collaboration governed at scale into custom build process Optionally integrate with Git to deploy to production Embed Models Models Models Add/connect Data Process Designer or App Developer Business Owner Data Owner or Receive insights Data Engineer integrates models into apps or processes IBM Other On as Dashboards, makes data available Cloud Clouds Prem Notebooks, … Connect to Data in IBM Cloud Data Services, On Prem Data, or Data on other Clouds Store your data in the IBM Cloud and/or connect your on-prem data or third-party cloud data ... ... ... IBMIBM Cloud Cloud SQL Query Query – Very – HighServerless Level Architecture SQL aaS (MVP 1Q 2018) Application • Completely self-service SQL on cloud data • Serverless scale-out SQL execution on TB of data 4. Read 1. Submit SQL results • Pay per query (pennies per query) • Full ANSI SQL support, incl. OLAP functions & Spatial SQL Query • Easy SQL operationalization for developers SQL • REST API, Cloud Function integration, SDKs (Python, Node.js) 2. Read 3. Write data results IBM Cloud Object Storage Archive & Export Land IBM Cloud Databases IBM Cloud Streaming Db2 on Cloud Watson IoT IBM Streams Streaming Events Eliminates the burden Allows you to focus on IBM Cloud Databases of managing your business A fully managed collection of open infrastructure and objectives rather than system health as well the operations and source databases and persistence as the associated support of your own services available through a consistent capital expenses platform stack consumption, pricing, and interaction model Avoid vendor lock-in and allows for easily coordinating multi-database strategies by using Open Source DBaaS like: • Postgres • etcd • Mongo • MySQL • ElasticSearch • ScyllaDB • Redis • JanusGraph IBM Cloud / V2 / 5-18-2018 / © 2018 IBM Corporation • RabbitMQ 9 Where to get started ? New: Watson Studio Sign Up Page for Germany / EU: https://eu-de.dataplatform.cloud.ibm.com IBM Cloud / Watson and Cloud Platform / © 2018 IBM Corporation 11 Sign Up or use existing IBM Cloud Account – Watson Studio + Knowledge Catalog + Cloud Object Storage will be created for you IBM Cloud / Watson and Cloud Platform / © 2018 IBM Corporation 12 Create your own Projects to connect, analyze, visualize Data and create and train Models IBM Cloud / Watson and Cloud Platform / © 2018 IBM Corporation 13 Watson Studio and Watson Machine Learning in the IBM Cloud Catalog IBM Cloud / Watson and Cloud Platform / © 2018 IBM Corporation 14 Object Storage IBM Cloud / Watson and Cloud Platform / © 2018 IBM Corporation 15 Database Services IBM Cloud / Watson and Cloud Platform / © 2018 IBM Corporation 16.
Load more

Recommended publications
  • History of AI at IBM and How IBM Is Leveraging Watson for Intellectual Property
    History of AI at IBM and How IBM is Leveraging Watson for Intellectual Property 2019 ECC Conference June 9-11 at Marist College IBM Intellectual Property Management Solutions 1 IBM Intellectual Property Management Solutions © 2017-2019 IBM Corporation Who are We? At IBM for 37 years I currently work in the Technology and Intellectual Property organization, a combination of CHQ and Research. I have worked as an engineer in Procurement, Testing, MLC Packaging, and now T&IP. Currently Lead Architect on IP Advisor with Watson, a Watson based Patent and Intellectual Property Analytics tool. • Master Inventor • Number of patents filed ~ 24+ • Number of submissions in progress ~ 4+ • Consult/Educate outside companies on all things IP (from strategy to commercialization, including IP 101) • Technical background: Semiconductors, Computers, Programming/Software, Tom Fleischman Intellectual Property and Analytics [email protected] Is the manager of the Intellectual Property Management Solutions team in CHQ under the Technology and Intellectual Property group. Current OM for IP Advisor with Watson application, used internally and externally. Past Global Business Services in the PLM and Supply Chain practices. • Number of patents filed – 2 (2018) • Number of submissions in progress - 2 • Consult/Educate outside companies on all things IP (from strategy to commercialization, including IP 101) • Schaumburg SLE Sue Hallen • Technical background: Registered Professional Engineer in Illinois, Structural Engineer by [email protected] degree, lots of software development and implementation for PLM clients 2 IBM Intellectual Property Management Solutions © 2017-2019 IBM Corporation How does IBM define AI? IBM refers to it as Augmented Intelligence…. • Not artificial or meant to replace Human Thinking…augments your work AI Terminology Machine Learning • Provides computers with the ability to continuing learning without being pre-programmed.
    [Show full text]
  • Big Blue in the Bottomless Pit: the Early Years of IBM Chile
    Big Blue in the Bottomless Pit: The Early Years of IBM Chile Eden Medina Indiana University In examining the history of IBM in Chile, this article asks how IBM came to dominate Chile’s computer market and, to address this question, emphasizes the importance of studying both IBM corporate strategy and Chilean national history. The article also examines how IBM reproduced its corporate culture in Latin America and used it to accommodate the region’s political and economic changes. Thomas J. Watson Jr. was skeptical when he The history of IBM has been documented first heard his father’s plan to create an from a number of perspectives. Former em- international subsidiary. ‘‘We had endless ployees, management experts, journalists, and opportunityandlittleriskintheUS,’’he historians of business, technology, and com- wrote, ‘‘while it was hard to imagine us getting puting have all made important contributions anywhere abroad. Latin America, for example to our understanding of IBM’s past.3 Some seemed like a bottomless pit.’’1 However, the works have explored company operations senior Watson had a different sense of the outside the US in detail.4 However, most of potential for profit within the world market these studies do not address company activi- and believed that one day IBM’s sales abroad ties in regions of the developing world, such as would surpass its growing domestic business. Latin America.5 Chile, a slender South Amer- In 1949, he created the IBM World Trade ican country bordered by the Pacific Ocean on Corporation to coordinate the company’s one side and the Andean cordillera on the activities outside the US and appointed his other, offers a rich site for studying IBM younger son, Arthur K.
    [Show full text]
  • The Evolution of Ibm Research Looking Back at 50 Years of Scientific Achievements and Innovations
    FEATURES THE EVOLUTION OF IBM RESEARCH LOOKING BACK AT 50 YEARS OF SCIENTIFIC ACHIEVEMENTS AND INNOVATIONS l Chris Sciacca and Christophe Rossel – IBM Research – Zurich, Switzerland – DOI: 10.1051/epn/2014201 By the mid-1950s IBM had established laboratories in New York City and in San Jose, California, with San Jose being the first one apart from headquarters. This provided considerable freedom to the scientists and with its success IBM executives gained the confidence they needed to look beyond the United States for a third lab. The choice wasn’t easy, but Switzerland was eventually selected based on the same blend of talent, skills and academia that IBM uses today — most recently for its decision to open new labs in Ireland, Brazil and Australia. 16 EPN 45/2 Article available at http://www.europhysicsnews.org or http://dx.doi.org/10.1051/epn/2014201 THE evolution OF IBM RESEARCH FEATURES he Computing-Tabulating-Recording Com- sorting and disseminating information was going to pany (C-T-R), the precursor to IBM, was be a big business, requiring investment in research founded on 16 June 1911. It was initially a and development. Tmerger of three manufacturing businesses, He began hiring the country’s top engineers, led which were eventually molded into the $100 billion in- by one of world’s most prolific inventors at the time: novator in technology, science, management and culture James Wares Bryce. Bryce was given the task to in- known as IBM. vent and build the best tabulating, sorting and key- With the success of C-T-R after World War I came punch machines.
    [Show full text]
  • Treatment and Differential Diagnosis Insights for the Physician's
    Treatment and differential diagnosis insights for the physician’s consideration in the moments that matter most The role of medical imaging in global health systems is literally fundamental. Like labs, medical images are used at one point or another in almost every high cost, high value episode of care. Echocardiograms, CT scans, mammograms, and x-rays, for example, “atlas” the body and help chart a course forward for a patient’s care team. Imaging precision has improved as a result of technological advancements and breakthroughs in related medical research. Those advancements also bring with them exponential growth in medical imaging data. The capabilities referenced throughout this document are in the research and development phase and are not available for any use, commercial or non-commercial. Any statements and claims related to the capabilities referenced are aspirational only. There were roughly 800 million multi-slice exams performed in the United States in 2015 alone. Those studies generated approximately 60 billion medical images. At those volumes, each of the roughly 31,000 radiologists in the U.S. would have to view an image every two seconds of every working day for an entire year in order to extract potentially life-saving information from a handful of images hidden in a sea of data. 31K 800MM 60B radiologists exams medical images What’s worse, medical images remain largely disconnected from the rest of the relevant data (lab results, patient-similar cases, medical research) inside medical records (and beyond them), making it difficult for physicians to place medical imaging in the context of patient histories that may unlock clues to previously unconsidered treatment pathways.
    [Show full text]
  • Cell Broadband Engine Spencer Dennis Nicholas Barlow the Cell Processor
    Cell Broadband Engine Spencer Dennis Nicholas Barlow The Cell Processor ◦ Objective: “[to bring] supercomputer power to everyday life” ◦ Bridge the gap between conventional CPU’s and high performance GPU’s History Original patent application in 2002 Generations ◦ 90 nm - 2005 ◦ 65 nm - 2007 (PowerXCell 8i) ◦ 45 nm - 2009 Cost $400 Million to develop Team of 400 engineers STI Design Center ◦ Sony ◦ Toshiba ◦ IBM Design PS3 Employed as CPU ◦ Clocked at 3.2 GHz ◦ theoretical maximum performance of 23.04 GFLOPS Utilized alongside NVIDIA RSX 'Reality Synthesizer' GPU ◦ Complimented graphical performance ◦ 8 Synergistic Processing Elements (SPE) ◦ Single Dual Issue Power Processing Element (PPE) ◦ Memory IO Controller (MIC) ◦ Element Interconnect Bus (EIB) ◦ Memory IO Controller (MIC) ◦ Bus Interface Controller (BIC) Architecture Overview SPU/SPE Synergistic Processing Unit/Element SXU - Synergistic Execution Unit LS - Local Store SMF - Synergistic Memory Frontend EIB - Element Interconnect Bus PPE - Power Processing Element MIC - Memory IO Controller BIC - Bus Interface Controller Synergistic Processing Element (SPE) 128-bit dual-issue SIMD dataflow ○ “Single Instruction Multiple Data” ○ Optimized for data-level parallelism ○ Designed for vectorized floating point calculations. ◦ Workhorses of the Processor ◦ Handle most of the computational workload ◦ Each contains its own Instruction + Data Memory ◦ “Local Store” ▫ Embedded SRAM SPE Continued Responsible for governing SPEs ◦ “Extensions” of the PPE Shares main memory with SPE ◦ can initiate
    [Show full text]
  • Cheryl Watson's
    Cheryl Watson’s 1998, No. 6 TUNING Letter A PRACTICAL JOURNAL OF S/390 TUNING AND MEASUREMENT ADVICE Inside this issue... This Issue: My heart attack is important news, at least from my point of view. See page 2. But I'm feeling really great these days, thanks to A Note From Cheryl & Tom .2 modern medicine. Management Issues #20 ......3 Class Schedule .....................4 Upgrading a processor is the focus of this issue. How to size, S/390 News how to understand the difference between speed and capacity, and GRS Star .............................5 how to avoid typical problems after an upgrade are all covered MXG & CA-MICS.................5 starting on page 14. CMOS & Compression ......5 Y2K ......................................6 Java and Component Broker are featured in two articles pro- Java .....................................6 vided by Glenn Anderson of IBM Education (page 6). Component Broker ............7 LE.........................................9 IBM is now recommending that multi-system sites be at WSC Flashes ....................10 OS/390 R5 before freezing systems for Y2K. See WSC Flash The Net..............................11 98044 on page 10 for this important item. Pubs & APARs .................12 Focus: Processor Upgrades A known integrity exposure in ISPF has existed for the five Sizing Processors............14 years since ISPF V4, but new customers keep running into the Migration Issues...............23 problem. See page 29. Upgrade Problems...........26 Cheryl's Updates New Web links and important new manuals and books are Most Common Q&As.......28 listed in our S/390 News starting on page 12. TCP/IP................................28 ISPF Exposure..................29 Don't go to OS/390 R5 without checking with your TCP/IP WLM Update .....................30 vendors or you could be in serious trouble.
    [Show full text]
  • Watson Machine Learning for Z/OS — Jamar Smith Data Scientist, North America Z Hybrid Cloud [email protected]
    Watson Machine Learning for z/OS — Jamar Smith Data Scientist, North America z Hybrid Cloud [email protected] 1 Goal Demonstrate the value of enterprise analytics on the IBM Z platform. 2 Agenda Enterprise Analytics Strategy Machine Learning Overview Value of Analytics in Place IBM Cloud Pak 4 Data 3 Enterprise Analytics Strategy 4 Current trends in analytics The need for Pervasive Analytics is increasing in almost every industry Real time or near real time analytic results are necessary Need to leverage all relevant data sources available for insight Ease demands on highly sought-after analytic skill base Embrace rapid pace of innovation 5 Data gravity key to enterprise analytics Performance matters Core transactional for variety of data on systems of record are and off IBM Z on IBM Z Predominance of data Real-time / near real- originates on IBM Z, time insights are z/OS (transactions, valuable member info, …) Data volume is large, distilling data Data Security / data privacy provides operational needs to be preserved efficiencies Gravity Podcast: http://www.ibmbigdatahub.com/podcast/making-data-simple-what-data-gravity 6 IBM Z Analytics Keep your data in place – a different approach to enterprise analytics • Keep data in place for analytics • Keep data in place, encrypted and secure • Minimize latency, cost and complexity of data movement • Transform data on platform • Improve data quality and governance • Apply the same resiliency to analytics as your operational applications • Combine insight from structured & unstructured data from z and non-z data sources • Leverage existing people, processes and infrastructure 7 Machine Learning Overview 8 What is Machine Learning? Computers that learn without being explicitly programmed.
    [Show full text]
  • An Introduction to the Cell Broadband Engine Architecture
    IBM Systems & Technology Group An Introduction to the Cell Broadband Engine Architecture Owen Callanan IBM Ireland, Dublin Software Lab. Email: [email protected] 1 Cell Programming Workshop 5/7/2009 © 2009 IBM Corporation IBM Systems & Technology Group Agenda About IBM Ireland development Cell Introduction Cell Architecture Programming the Cell – SPE programming – Controlling data movement – Tips and tricks Trademarks – Cell Broadband Engine and Cell Broadband Engine Architecture are trademarks of Sony Computer Entertainment, Inc. 2 Cell Programming Workshop 5/7/2009 © 2009 IBM Corporation IBM Systems & Technology Group Introduction - Development in IBM Ireland Lotus Software Research Tivoli Industry Assets & Models RFID Center of Competence High Performance Computing 3 Cell Programming Workshop 5/7/2009 © 2009 IBM Corporation IBM Systems & Technology Group Introduction – HPC in IBM Dublin Lab Blue Gene team Deep Computing Visualization (DCV) Dynamic Application Virtualization (DAV) Other areas also – E.g. RDMA communications systems 4 Cell Programming Workshop 5/7/2009 © 2009 IBM Corporation IBM Systems & Technology Group Cell Introduction 5 Cell Programming Workshop 5/7/2009 © 2009 IBM Corporation IBM Systems & Technology Group Cell Broadband Engine History IBM, SCEI/Sony, Toshiba Alliance formed in 2000 Design Center opens March 2001 ~$400M Investment, 5 years, 600 people February 7, 2005: First technical disclosures January 12, 2006: Alliance extended 5 additional years YKT, EFK, BURLINGTON, ENDICOTT ROCHESTER
    [Show full text]
  • Watson Health Intro 170612
    Empowering Heroes, Transforming Health Introduction to Watson Health IBM's statements regarding its plans, directions and intent are subject to change or withdrawal without notice at IBM's sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. Watson Health © IBM Corporation 2017 2 Time matters in healthcare, cognitive technologies, and machine learning. Watson Health © IBM Corporation 2017 3 IBM Leadership in Augmented Intelligence and Health: Over a Decade in Development 7,000 employees IBM enables an 10,000+ “evidence-based clients & partners Healthcare Memorial Sloan eco-system” Kettering Cancer Center Mayo Clinic Cognitive test Cleveland Clinic case results in creation of Watson 2005 2008 2010 2011 2012 2014 2014 2015 2016 Watson Health © IBM Corporation 2017 4 Global Shifts Drive Momentum R&D, Access, Delivery & Engagement Data Dynamic Value vs Efficient, Explosion Delivery Volume Effective R&D 150+ exabytes 50% $47 trillion 1 in 10 of healthcare Expected alternative payments Estimated global economic clinical Trials in cancer data today1 from Centers of Medicare & impact of chronic disease by are shut down from lack of Medicaid by 20184 20307 participation10 Over 230K 75%+ $3 trillion $2B active clinical trials2 of patients expected to use Estimated U.S.
    [Show full text]
  • Predictive Analytics in Value-Based Healthcare
    White paper Predictive Analytics in Value-Based Healthcare: Forecasting Risk, Utilization, and Outcomes Delivering effective value-based healthcare requires identifying Consider a model predicting the chance of a hospital and mitigating risk by anticipating and preventing adverse readmission. The input training dataset might contain events and outcomes. Predictive models have been a part of thousands of hospitalization events. Each hospital admission healthcare practice for decades. However, more advanced would include the information on a patient’s medical history, analytics have started to take shape to provide better visibility like chronic conditions and prior utilization. The dataset into characterizing a patient’s current state and future risk. would also contain the “answer” as to whether that particular With the use of big data, it is possible to build models around hospitalization resulted in a readmission within 30 days. predicting future events and outcomes, utilization, and overall Data scientists typically use specialized software (or write risk. These predictive models can be: their own) to build models that “learn” from the results of a – incorporated into a clinical workflow to facilitate care readmission to improve predictive accuracy. The model is then management and identify individuals at risk evaluated using the testing dataset, where a similar set of data – used to perform risk adjustment on quality measures is provided without the “answer” and the model’s prediction to account for patient severity is compared to the correct result (whether the patient was – employed to understand the treatment pathway with truly readmitted or not). The accuracy of a model on a testing the greatest chance of success dataset is typically what is presented as the true performance Predictive models have many potential uses, but no of the model.
    [Show full text]
  • IBM Db2 on Cloud Solution Brief
    Hybrid Data Management IBM Db2 on Cloud A fully-managed, relational database on IBM Cloud and Amazon Web Services with elastic scaling and autonomous failover 1 IBM® Db2® on Cloud is a fully-managed, SQL cloud database that can be provisioned on IBM Cloud™ and Amazon Web Services, eliminating the time and expense of hardware set up, software installation, and general maintenance. Db2 on Cloud provides seamless compatibility, integration, and licensing with the greater Db2 family, making your data highly portable and extremely flexible. Through the Db2 offering ecosystem, businesses are able to desegregate systems of record and gain true analytical insight regardless of data source or type. Db2 on Cloud and the greater Db2 family support hybrid, multicloud architectures, providing access to intelligent analytics at the data source, insights across the business, and flexibility to support changing workloads and consumptions cases. Whether you’re looking to build cloud-native applications, transition to a fully-managed instance of Db2, or offload certain workloads for disaster recovery, Db2 on Cloud provides the flexibility and agility needed to run fast queries and support enterprise-grade applications. Features and benefits of Db2 on Cloud Security and disaster recovery Cloud databases must provide technology to secure applications and run on a platform that provides functional, infrastructure, operational, network, and physical security. IBM Db2 on Cloud accomplishes this by encrypting data both at rest and in flight, so that data is better protected across its lifecycle. IBM Db2 on Cloud helps restrict data use to only approved parties with user authentication for platform services and resource access control.
    [Show full text]
  • POV – Watson Privacy, Compliance, & Security
    POV – Watson Privacy, Compliance, & Security “Every organization that develops or uses AI, or hosts or processes data, must do so responsibly and transparently. Companies are being judged not just by how we use data, but by whether we are trusted stewards of other people’s data. Society will decide which companies it trusts.” -Ginni Rometty, IBM Chairman, President, and CEO Table of contents: • Scope • Introduction • Data privacy: What is it and how to do it • Compliance & regulations • Security • Frequently asked questions Scope The scope of this document is the IBM Watson services running on IBM Cloud. The document does not address other IBM or client offerings with Watson in their names (e.g., Watson Health), or services that are not in the AI part of the IBM Cloud catalog. Nor does it address any third-party services showing up under AI in the IBM Catalog. Introduction The ability of artificial intelligence (AI) to transform vast amounts of complex, ambiguous information into insights has the potential to reveal long-held secrets and solve some of the world’s most enduring problems. It can help doctors treat disease, predict the weather, and manage the global economy. It is an undeniably powerful tool. And like all powerful tools, great care must be taken in its development and deployment. To reap the societal benefits of artificial intelligence, we will first need to trust it. We have created a system of best practices that guide the management of Watson; a system that includes contracts and disclosures that help foster full transparency; a strategy that reflects our compliance efforts with existing legislation and policy; third-party certifications and security testing by third parties to validate the best practices; and a framework that provides for privacy and personal data protection.
    [Show full text]