Feb 2020 @qualcomm_tech Making AI ubiquitous Qualcomm Technologies, Inc. Devices, machines, and things are becoming more intelligent 2 Reasoning Learn, infer context, Offering new and anticipate capabilities to enrich our lives Perception Action Hear, see, monitor, Act intuitively, and observe interact naturally, and protect privacy 3 A world where virtually everyone and everything is intelligently connected 4 Distributed autonomy Privacy/security New experiences Immediacy The intelligent Processing over 5G Efficiency Edge cloud wireless edge On-device On-device AI, processing, sensing, Customized/local value vision,… augmented by edge cloud Reliability Process data closest to the source to scale for massive amount of data and Private/public networks connected things Personalization 5 Process data at the source to scale AI and make sense of a digitized world Past Today Cloud-centric AI Partially-distributed AI AI training and AI inference Power-efficient in the central cloud on-device AI inference On-device Future Fully-distributed AI With lifelong on-device learning On-device Privacy intelligence is Reliability paramount Low latency Process data closest to the source, complement the cloud Efficient use of network bandwidth On-device intelligence is quickly gaining momentum Key segments are expected to see full AI attach rates by 2025 10% 100% AI attach rate AI attach rate 2018 2025 PCs / Smart Mobile Automotive XR Tablets speakers Source: Tractica, 2019 Mobile is becoming the pervasive AI platform ~7.8 Billion Cumulative smartphone unit shipments forecast between 2018–2022 Source: IDC Aug. ‘18 9 Mobile computing Smart cities Mobile scale changes everything Smart homes Automotive Bringing AI Smartphones to the masses Healthcare Industrial IoT Networking Wearables Extended reality Rapid replacement cycles Superior scale Integrated/optimized technologies 10 AI offers enhanced experiences and new capabilities for smartphones True personal assistance Superior photography Extended battery life Natural user interfaces Enhanced connectivity Enhanced security A new development paradigm where things repeatedly improve 11 AI will drive transformation across industries 12 Boundless mobile XR experiences 13 Shaping the future of transportation Personalized driver settings Driver awareness monitoring Greater autonomous capabilities 14 Powering the factory of the future 15 Autonomous manufacturing Smart security for home Smart displays Smarter and robotics and enterprise and speakers agriculture More efficient use Home hubs and Sustainable cities Digitized logistics of energy and utilities smart appliances and infrastructure and retail IoT AI for IoT across the home, industrial/enterprise, and Smart Cities 16 Power and thermal efficiency are essential for on-device AI The challenge of Constrained mobile AI workloads environment Very compute intensive Must be thermally efficient for sleek, ultra-light designs Large, complicated neural network models Requires long battery life for all-day use Complex concurrencies Storage/Memory bandwidth limitations Real-time Always-on 17 Making power efficient AI pervasive Focusing on high performance HW/SW and optimized network design Efficient Algorithmic Software hardware advancements tools Developing heterogeneous compute to Algorithmic research that benefits from Software accelerated run-time run demanding neural networks at low state-of-the-art deep neural networks for deep learning power and within thermal limits Optimization for space and SDK/development frameworks Selecting the right compute runtime efficiency block for the right task 18 Our AI leadership Over a decade of cutting-edge AI R&D, speeding up commercialization and enabling scale Research face MWC demo Collaboration Brain Corp Qualcomm® Artificial Power efficiency gains Mobile AI Enablement detection with deep showcasing photo with Google on raises $114M Intelligence Research through compression, Center in Taiwan to Deep-learning learning sorting and hand TensorFlow initiated quantization, and open based AlexNet wins writing recognition compilation ImageNet competition Research in spiking Research artificial Opened Qualcomm Announced Qualcomm Gauge equivariant neural networks neural processing Research Netherlands Facebook Technologies architectures Caffe2 support ships ONNX CNNs supported by Microsoft, Facebook, Amazon 2009 2013 2015 2016 2017 2018 2019 2020 Acquired Qualcomm EuVision 2007 Opened joint Technologies Investment and research lab researchers Qualcomm Research collaboration with Completed Brain with University Acquired win best paper initiates first AI project Brain Corp Corp joint research of Amsterdam Scyfer at ICLR ® Qualcomm Snapdragon Qualcomm® 3rd Gen Snapdragon Consistent AI R&D investment is Neural 660 Vision Snapdragon 665, 730, ® Processing Snapdragon Intelligence Automotive 730G Qualcomm SDK 630 Platform Cockpit Snapdragon the foundation for product leadership Ride Platform nd rd Qualcomm Artificial Intelligence Research is an initiative of Qualcomm Technologies, Inc. 2 Gen AI Engine 3 Gen AI Engine Snapdragon Qualcomm® 4th Gen AI 5th Gen AI Qualcomm Snapdragon, Qualcomm Neural Processing SDK, Qualcomm Vision Intelligence (Snapdragon 835) (Snapdragon 845) 710 Cloud AI 100 Engine Engine Platform, Qualcomm AI Engine, Qualcomm Cloud AI, Qualcomm Snapdragon Ride, and st ® 1 Gen Qualcomm AI Engine (Snapdragon Qualcomm® (Snapdragon Qualcomm QCS400I are products of Qualcomm Technologies, Inc. and/or its subsidiaries. ® 19 (Qualcomm Snapdragon™ 820 855) QCS400 865) Mobile Platform) (First audio SoC) Leading research and development across the entire spectrum of AI Fundamental Applied research research Deep Neural Graph and Machine Deep G-CNN transfer network kernel learning learning for learning compression optimization training tools graphics Bayesian Deep Neural Compute Source CV DL for combinatorial generative network Voice UI in memory compression new sensors optimization models quantization Hybrid Video Bayesian Hardware- Power reinforcement Fingerprint recognition distributed aware management learning learning deep learning & prediction 20 Can we apply foundational mathematics of physics, like quantum field theory, to deep learning? 21 G-CNN Video 22 Today’s deep learning Tomorrow’s deep learning Traditional CNNs Gauge Equivariant CNNs Produce state-of-the art results but… No matter how you rotate or move the object, do not generalize input like rotations the generalized model will still identify the object Applying Translation Rotated objects and foundational images applicable to works drones, robots, cars, mathematics fisheye-lens cameras. of physics VR, AR,.. Like quantum field theory, to deep learning Rotation doesn’t work (Convolutional neural networks would need to be retrained with (Generalized CNNs (G-CNN): Gauge equivariant CNN, Group, and Steerable CNN new rotated images to determine new set of parameters—like filter weights) pioneered by Qualcomm AI Research do not need to be retrained) Advancing fundamental AI research, such as generalized CNNs 23 Unifying framework Equivariance Gauge equivariant CNN unify special cases like No matter how you rotate or move the Group CNNs and Steerable CNNs, all pioneered object, it will still be identified by Qualcomm AI Research G-CNN can generalize models for different Robust performance, faster training, and fewer symmetries — traditional CNNs must training examples required be retrained Broad societal benefits Generalized geometry Use cases like drones, robots, cars, XR, fisheye Traditional CNNs work well on narrow field-of-view lenses, 3D gaming, … cameras, but fail on e.g. fish-eye cameras But also areas like state-of-the-art accuracy on G-CNN can analyze image data on any curved climate pattern segmentation space, from flat to spherical Pioneering deep learning research in generalized CNNs 24 Trained neural network model New Inference input data output Compression Quantization Compilation Learning to prune model while Learning to reduce bit-precision Learning to compile AI models for keeping desired accuracy while keeping desired accuracy efficient hardware execution Applying AI to optimize AI model through automated techniques Hardware AI Acceleration Acceleration research awareness (scalar, vector, tensor) Such as compute-in-memory Advancing AI research to increase power efficiency 25 Trained neural network model New Inference input data output Compression Quantization Compilation Learning to prune model while Learning to reduce bit-precision Learning to compile AI models for keeping desired accuracy while keeping desired accuracy efficient hardware execution Applying AI to optimize AI model through automated techniques Compression with Perf. per watt Performance Recent less than 1% loss improvement from improvement over 3x in accuracy1 >4x savings in memory 4x TensorFlow Lite3 examples and compute2 Advancing AI research to increase power efficiency 26 1: With both Bayesian compression and spatial SVD with ResNet18 as baseline. 2: For a quantized INT8 model vs a FP32 model that is not quantized. 3: On average improvement of tested AI models. Qualcomm® Artificial Intelligence Engine The hardware and software components for efficient on-device machine learning Mobile Apps NN Frameworks Cognitive Toolkit Runtime Software Frameworks Qualcomm® Neural Processing TensorFlow Lite Google NN API 5th Gen SDK AI Engine Libraries Qualcomm® Math Libraries OpenCL Hexagon NN Cores Qualcomm® Hexagon™ DSP Qualcomm® Kryo™ CPU Qualcomm® Adreno™ GPU Scalar Vector Tensor Qualcomm