Machine learning is going mobile
By David Schatsky
ACHINE learning—the process by which Signals Mcomputers can get better at perform- ing tasks through exposure to data, rather • Google has introduced language translation than through explicit programming—requires software, using small neural networks opti- massive computational power, the kind usu- mized for mobile phones, which can per- ally found in clusters of energy-guzzling, form well without an Internet connection.1 cloud-based computer servers outfitted with specialized processors. But an emerging trend • Lenovo announced a mobile phone that promises to bring the power of machine learn- uses multiple sensors, high-speed image ing to mobile devices that may lack or have processing hardware, and specialized only intermittent online connectivity. This will Google software to support capabili- give rise to machines that sense, perceive, learn ties such as indoor wayfinding, precision from, and respond to their environment and measuring, and augmented reality even their users, enabling the emergence of new when offline.2 product categories, reshaping how businesses engage with customers, and transforming how • NVIDIA, a maker of graphics processing work gets done across industries. technology, introduced an embeddable module for computer vision applications Machine learning is going mobile
in devices such as drones and autono- systems run in the cloud on powerful serv- mous vehicles that the company says ers, processing data such as digitized voice or consumes one-tenth the power of a photos that users upload. competing offering.3 Until recently, a typical smartphone lacked the power to perform such tasks without con- • Qualcomm introduced a new processor and necting to the cloud, except in limited ways. software platform that support machine For instance, some mobile phone software can learning tasks such as image classification, recognize a single face—the owner’s—in order speech recognition, and anomaly detection to unlock the phone, or a small set of predeter- without a connection to a network.4 mined words such as “OK Google.” But offline support for increasingly powerful perception • Drone maker DJI recently introduced tasks is coming to mobile devices. a consumer-oriented drone that uses advanced computer vision hardware to Pushing machine learning enable it to follow a moving object while onto mobile devices automatically avoiding obstacles.5 Firms are starting to outfit smartphones, drones, and cars with chips based on new Machine learning is the designs that can run neural networks efficiently keystone cognitive technology while consuming 90 percent less power than 11 Emerging technologies rarely get as big a previous generations. Research efforts at MIT publicity boost as machine learning recently and IBM suggest that we will soon see more saw, when Google software defeated one of the chips on the market that excel at running neu- world’s top players of Go, one of the most com- ral networks at high speed, in small spaces and 12 plex board games ever created, in a best-of-five at low power. Because of this, mobile devices series of matches.6 The international headlines are becoming increasingly capable of perform- confirmed that machine learning—the pro- ing sophisticated feats that take advantage of cess by which fresh data can teach computers neural networks, such as computer vision and to better perform tasks—is one of the hottest speech recognition, once reserved for powerful domains within the field of artificial intel- servers running in the cloud. ligence, and that this cognitive technology is It is not only progress in hardware that is progressing rapidly.7 bringing machine learning to mobile devices. Neural networks—computer models Tech vendors are also finding ways to create designed to mimic aspects of the human compact neural networks capable of running brain’s structure and function, with elements tasks such as speech recognition and language representing neurons and their interconnec- translation on conventional mobile phones tions—are an increasingly popular way of with no connection to a server required. implementing machine learning. They are For instance, Google has introduced mobile particularly well suited for performing percep- language-translation software using small tual tasks such as computer vision and speech neural networks optimized for smartphones 13 recognition. Familiar examples of applications that can perform well even offline. And that employ neural networks for such tasks Google researchers recently published a paper include Google’s voice search,8 Facebook’s sys- describing an Internet-independent speech tem for tagging people in photos,9 and Google recognition system that performs well on a 14 Photos, which uses a neural network-based commercial mobile phone. image recognition system to automatically Mobile devices are acquiring the power to classify photos by their contents.10 All of these perform sophisticated perceptual tasks with- out dependence on connectivity to the cloud,
2 Machine learning is going mobile
Perceptual interfaces bringing greater accuracy, reliability, and and interactivity responsiveness while strengthening user pri- In media and entertainment, we will likely vacy. This should greatly expand the number of see mobile devices—both general-purpose applications of perceptual computing coming ones such as mobile phones and special-pur- to market—and not only on mobile phones. pose ones such as augmented-reality head- Mobile machine learning and perceptual com- sets—offering ever more realistic and engaging puting will power a wide range of devices, from augmented and virtual reality for games and mobile sensors to phones, tablets, drones, cars, filmed entertainment. and new types of devices as yet unimagined, Ultralow power processors designed creating significant opportunities for business. for machine learning will likely help con- sumer and industrial devices and machines Many industries will see new understand and respond to the environ- and improved applications ment around them, and find their way into It’s impossible to enumerate all of the Internet-independent voice-controlled wear- applications we will see for mobile devices able devices, household appliances, and capable of performing sophisticated percep- industrial machinery. tual tasks involving vision, speech, or other Navigation and motion control sensory input. But they are likely to be found in every industry and have one or more of the Low-power chips with powerful com- following capabilities: puter vision support are bringing impressive capabilities to unmanned aerial vehicles, also • Analysis or diagnosis of sensory data known as drones, which already have applica- tions in many industries, from real estate and • Perceptual interfaces or interactivity construction to agriculture, energy, aero- space, and defense. Drone maker DJI recently • Navigation and motion control introduced a consumer-oriented aerial vehicle A few examples follow. able to follow a moving object while automati- cally avoiding obstacles.16 Analysis or diagnosis New, powerful mobile computer vision In health care, we envision a wide range of modules that use deep learning are helping diagnostic applications, including some aimed advanced driver assistance systems to “address at consumers. Imagine, for instance, a smart- the challenges of everyday driving, such as phone app that can diagnose skin conditions unexpected road debris, erratic drivers and 17 and insect bites by analyzing digital photos construction zones.” without transmitting the image data over Indoor navigation apps that use computer a network. vision to precisely locate a user, track her We imagine mobile architecture and motion, and guide her in interior spaces will design applications that use computer vision to find use in museums, train stations, airports, generate accurate 3D models of interior spaces malls, and retail stores, opening up new adver- quickly and easily. tising and commerce opportunities without the An ever more powerful and resilient need to deploy beacons or other connectivity- Internet of Things will include self-monitor- based approaches. ing industrial equipment that uses machine And on the horizon are applications not learning to predict maintenance needs and yet imagined, from wearable to pocketable to self-diagnose failures.15 portable, that can sense, analyze, and respond to sensory inputs including sound, video,
3 Machine learning is going mobile
and biometrics—all enabled by low-power from household appliances to personal robots chips designed to support neural networks for to industrial equipment. machine learning. Marketing leaders should explore how a new generation of perceptive devices could Implications help cultivate closer and more responsive rela- Compact, efficient, low-power, high- tionships with customers. performance, mobile machine learning. New Operations executives should evaluate how products. New human-computer interfaces. such devices—including the evolving crop of Powerful new ways of engaging with and serv- augmented-reality tools for industry—could ing customers. The trend described here has help their people deliver an efficiency and implications for companies and professionals quality edge.18 across industries. Cyber risk professionals should explore Makers of mobile devices and mobile apps how mobile machine learning may present should begin to familiarize themselves with new ways of detecting and mitigating threats the potential of a new generation of devices targeting mobile devices. Qualcomm is already capable of offline machine learning. promoting its Snapdragon 820 processor’s User-experience designers should begin to machine learning-based malware detection explore the kinds of experiences made possible capabilities; there will surely be other offerings by these technologies. Tech vendors are releas- that take advantage of machine learning for ing development kits, such as Google’s Project this purpose.19 Tango development kit and NVIDIA’s Jetson The arrival on mobile devices of machine TX1 Developer Kit, to encourage designers learning and enhanced perceptual computing and developers to do so. is likely to have a big impact on a wide range of Product strategists and engineers working products, applications, and business practices on consumer and industrial products should over the next 18–24 months. It is time to start consider the value that mobile perceptual preparing for this era of mobile machine learn- computing could bring to products ranging ing and perceptual computing.
Endnotes
1. Otavio Good, “How Google Translate 3. Dustin Franklin, “NVIDIA Jetson TX1 squeezes deep learning onto a phone,” supercomputer-on-module drives next Google Research blog, July 29, 2015, http:// wave of autonomous machines,” NVIDIA, googleresearch.blogspot.com/2015/07/ November 11, 2015, https://devblogs. how-google-translate-squeezes-deep.html NVIDIA.com/parallelforall/NVIDIA-jetson- 2. Lenovo, “Lenovo and Google partner on new tx1-supercomputer-on-module-drives- Project Tango device,” January 7, 2016, http:// next-wave-of-autonomous-machines/. news.lenovo.com/news-releases/lenovo-and- 4. Snapdragon blog, “Live from New York, google-partner-on-new-project-tango-device. it’s Snapdragon 820: Prepare for an im- htm; Project Tango, www.google.com/atap/ mersive dive into mobile experience,” project-tango/, accessed March 20, 2016. November 10, 2015, www.qualcomm. com/news/snapdragon/2015/11/10/ live-new-york-its-snapdragon-820-prepare- immersive-dive-mobile-experience.
4 Machine learning is going mobile
5. DJI, “DJI launches new era of intelligent ing-capabilities-in-next-gen-devices. Farshid flying cameras,” March 2, 2016, www.dji. Sabet, chief business officer at Movidius, told us com/newsroom/news/DJI-Launches-New- its latest processor consumed one-tenth to one- Era-of-Intelligent-Flying-Cameras. twentieth of the power of a contemporary mo- 6. Youkyung Lee, “Go-playing program bile system on a chip for comparable computer AlphaGo defeats human champion 4:1,” vision tasks. (Interview on March 7, 2016.) Associated Press, March 15, 2016, http:// 12. Larry Hardesty, “Energy-friendly chip can hosted2.ap.org/APDEFAULT/f70471f764144b- perform powerful artificial-intelligence tasks,” 2fab526d39972d37b3/Article_2016-03-15-AS- MIT News, February 3, 2016, http://news.mit. SKorea-Game-Human-vs-Computer/ edu/2016/neural-chip-artificial-intelligence- id-bb42c197629048e1b2de71f16ef86b4b. mobile-devices-0203; interview with IBM’s 7. For an overview of cognitive technologies, Dharmendra Modha, March 17, 2016. see David Schatsky, Craig Muraskin, and 13. Good, “How Google Translate squeezes Ragu Gurumurthy, Demystifying artificial deep learning onto a phone”; Taran- intelligence, Deloitte University Press, No- tola, “Microsoft Translator gets of- vember 4, 2014, http://dupress.com/articles/ fline and photo-based features.” what-is-cognitive-technology/?coll=12201. 14. See Ian McGraw et al., “Personalized speech 8. Haşim Sak et al., “Google voice search: recognition on mobile devices,” Google, March faster and more accurate,” Google Re- 11, 2016, http://arxiv.org/pdf/1603.03185.pdf. search blog, September 24, 2015, http:// 15. This is an extension of the idea of edge googleresearch.blogspot.com/2015/09/ analytics described in David Schatsky google-voice-search-faster-and-more.html. and Avinav Trigunait, Internet of Things: 9. Yaniv Taigman et al., “DeepFace: Closing Dedicated networks and edge analytics will the gap to human-level performance in face broaden adoption, Deloitte University Press, verification,” Research at Facebook, June 24, January 21, 2016, http://dupress.com/articles/ 2014, https://research.facebook.com/publica- internet-of-things-iot-adoption-edge-analytics- tions/deepface-closing-the-gap-to-human- wireless-communication-networks/?per=4005. level-performance-in-face-verification/. 16. DJI, “DJI launches new era of in- 10. Chuck Rosenberg, “Improving photo telligent flying cameras.” search: A step across the semantic gap,” 17. NVIDIA, “NVIDIA boosts IQ of self- Google Research blog, June 12, 2013, http:// driving cars with world’s first in-car artificial googleresearch.blogspot.com/2013/06/ intelligence supercomputer,” January improving-photo-search-step-across.html. 4, 2016, http://NVIDIAnews.NVIDIA. 11. See, for instance, Samir Kumar, “Qualcomm com/news/NVIDIA-boosts-iq-of-self- Zeroth is advancing deep learning in devices,” driving-cars-with-world-s-first-in-car- Qualcomm OnQ blog, March 2, 2015, www. artificial-intelligence-supercomputer. qualcomm.com/news/onq/2015/03/02/ 18. To learn more about how companies are qualcomm-zeroth-advancing-deep-learning- already employing augmented and virtual devices-video, with power efficiency described reality, see Nelson Kunkel, Steve Soechtig, Jared in Hubert Nguyen, “Qualcomm Snapdragon Miniman, and Chris Stauch, Augmented and 820 speed & features,” Ubergizmo, January virtual reality go to work, Deloitte University 14, 2016, www.ubergizmo.com/articles/ Press, February 24, 2016, http://dupress.com/ snapdragon-820-speed-features/. See also articles/augmented-and-virtual-reality/. Movidius, “Google and Movidius to enhance deep learning capabilities in next-gen devices,” 19. Qualcomm Protective Intelligence, www. January 27, 2016, www.movidius.com/news/ qualcomm.com/products/snapdragon/secu- google-and-movidius-to-enhance-deep-learn- rity/smart-protect, accessed March 20, 2016.
5 Machine learning is going mobile About the author
David Schatsky is a senior manager at Deloitte LLP. He tracks and analyzes emerging technology and business trends, including the growing impact of cognitive technologies, for the firm’s leaders and its clients.
David Schatsky Senior manager Deloitte LLP +1 646-582-5209 [email protected]
Acknowledgements
The author would like to acknowledge the contributions ofSherean Ali, Ragu Gurumurthy, Craig Muraskin, and Michael Tam of Deloitte LLP; Elliott Torres and Phil Wilson of Deloitte Consult- ing LLP; Patrick Duessel, Abhi Singh, and Craig Wigginton of Deloitte & Touche LLP; Jeff Bier of the Embedded Vision Alliance; Dharmendra Modha of IBM Corp.; Farshid Sabet of Movidius Inc.; and Edgar Auslander of Menlo Business Partners.
Deloitte Digital is creating a new model for a new age—we’re a creative digital consultancy. That means we bring together all the creative and technology capabilities, business acumen, and industry insight needed to help transform our clients’ businesses with digital. With our end-to-end capabilities, clients can bring us their biggest challenges, knowing we’ve got what it takes to bring a new business vision to life. Let us show you how. www.deloittedigital.com
6 Follow @DU_Press Sign up for Deloitte University Press updates at DUPress.com.
About Deloitte University Press Deloitte University Press publishes original articles, reports and periodicals that provide insights for businesses, the public sector and NGOs. Our goal is to draw upon research and experience from throughout our professional services organization, and that of coauthors in academia and business, to advance the conversation on a broad spectrum of topics of interest to executives and government leaders. Deloitte University Press is an imprint of Deloitte Development LLC.
About this publication This publication contains general information only, and none of Deloitte Touche Tohmatsu Limited, its member firms, or their related entities (collectively the “Deloitte Network”) is, by means of this publication, rendering professional advice or services. Before making any decision or taking any action that may affect your finances or your business, you should consult a qualified professional adviser. No entity in the Deloitte Network shall be responsible for any loss whatsoever sustained by any person who relies on this publication.
About Deloitte Deloitte refers to one or more of Deloitte Touche Tohmatsu Limited, a UK private company limited by guarantee (“DTTL”), its network of member firms, and their related entities. DTTL and each of its member firms are legally separate and independent entities. DTTL (also referred to as “Deloitte Global”) does not provide services to clients. Please see www.deloitte.com/about for a more detailed description of DTTL and its member firms. Deloitte provides audit, tax, consulting, and financial advisory services to public and private clients spanning multiple industries. With a globally connected network of member firms in more than 150 countries and territories, Deloitte brings world-class capabilities and high-quality service to clients, delivering the insights they need to address their most complex business challenges. Deloitte’s more than 200,000 professionals are committed to becoming the standard of excellence. © 2016. For information, contact Deloitte Touche Tohmatsu Limited.