Is Believing DIRECTIONAL, 3D SOUND WILL KICK VIRTUAL REALITY up to a NEW LEVEL of VIVID REALISM P

THE HEALING WHITE-HAT BOTS THE SMART INDIA’S MASSIVE FIELDS TO THE RESCUE STETHOSCOPE IoT EXPERIMENT Electrical aids Can AI automate Using machine learning A vast network aims to speed skin repair software security? to detect pneumonia improve quality of life P. 07 P. 30 P. 36 P. 42

FOR THE TECHNOLOGY INSIDER | 02.19 Hearing Is Believing DIRECTIONAL, 3D SOUND WILL KICK VIRTUAL REALITY UP TO A NEW LEVEL OF VIVID REALISM P. 24 starting at AWGs $4,980 per channel 2.4 GSa/s, 16 bit, 750 MHz 4, 8 and more channels <50 ns trigger latency Typical Applications Semiconductor testing, quantum computing, phased-array radar design & test, Boost your lidar, spectroscopy, NMR lab's performance Impedance starting at Analyzers $11,270 DC to 5 MHz, 1 mΩ to 1 TΩ 0.05% basic accuracy Compensation Advisor and Confidence Indicators Typical Applications High-Q dielectrics, capacitive sensors, supercapacitors, PV materials, component characterization

Lock-in starting at Amplifiers $6,210 Up to 600 MHz Scope, FFT, FRA, Sweeper, Imaging tool Optional: AWG, PID, PLL, Boxcar, Counter, AM & FM Typical Applications AFM, LVP, CARS, SRS, SNOM, graphene, optical PLL, THz, pump-probe, RFID, MEMS, NEMS, gyros, NDT, MRFM LabOne® Software All instruments are equipped with LabOne®, the Zurich Instruments control software, providing a wealth of features, efficient workflows and ease of use. Access your instruments from any web browser or integrate it into your LabVIEW, MAT- LAB, Python, C, and .NET programs.

Intl. +41 44 515 0410 USA 855-500-0056 (Toll Free) [email protected] www.zhinst.com

Zurich Let's discuss your application Instruments Start the conversation today FEATURES_02.19

24 30 36 42 The White- The The India VR FOR Hat Hacking Stethoscope of Things Machine Gets Smart The world’s largest Internet of Things YOUR EARS Meet Mayhem, winner Engineers are upgrad- network will A complete virtual-reality experience of the DARPA contest ing the classic clinical transform India. to find and repair tool with adaptive By V.S. Shridhar requires dynamic 3D audio technology. software vulnerabilities. acoustics and AI. By Mathias Johansson By David Brumley By Mounya Elhilali & James E. West

On the cover Illustration for IEEE Spectrum by Eddie Guy DIRAC RESEARCH DIRAC

SPECTRUM.IEEE.ORG | NORTH AMERICAN | FEB 2019 | 01 DEPARTMENTS_02.19DEPARTMENTS_01.13

07 16 06 Online News Resources Opinion spectrum.ieee.org Electroceuticals to Help SIGSALY’s WWII Secrets What Would Doug Engelbart Do? Consumer Electronics You Heal Re-creating a critical component of A pioneer in modern computing Hall of Fame Gentle pulses of electricity can aid the first digital voice scrambler. and organizational-knowledge Hundreds of IEEE members recovery and fend off bacteria. By Jon D. Paul management could help solve our shared stories about their By Charles Q. Choi greatest global problems. favorite gadgets in our Consumer 18 Geek Life: By Tekla S. Perry Electronics Hall of Fame Member 09 Low-Power Tech Cools The Pinball Renaissance Experience. See all of them, Data Centers 19 Careers: Planning to Get Into 03 Back Story including the prizewinners, at 10 Medical Implants That the C-Suite 04 Contributors https://spectrum.ieee.org/ Remember Their Shape 20 Tools & Toys: Consumer 22 Internet of Everything: Do You member-experience 12 A New Charging Standard for EVs Electronics Memories Need a Smart Microwave? 14 The Big Picture: The Big Bad 52 Past Forward: 23 Numbers Don’t Lie: Robo-Wolf A Phone Fit for a Princess The Vacuum Tube’s Power Law

ADDITIONAL RESOURCES

Tech Insider / Webinars The Institute Available at spectrum.ieee.org/webinars Available at theinstitute.ieee.org

On-Demand: Spacecraft Charging and What It Means to Your INTERNET OF ANIMALS Learn how artificial intelligence, data Space System analytics, and other technologies are being used to protect endangered On-Demand: Retirement, on Your Terms species and track livestock. On-Demand: Simulation of Ultrasonic Borehole Imaging & Flow Measurement IEEE MILESTONE Francisco Salvá’s electric telegraph, invented in 1804, will be honored this month. He invented a new method of telegraphy by combining the generation of an electric current using the recently White Papers invented voltaic pile with detection by water electrolysis. Available at spectrum.ieee.org/whitepapers MEMBER ACHIEVEMENT Read about this year’s recipient of the IEEE Remove Barriers to Efficiency with High-Performance ZVS Medal for Innovations in Healthcare Technology, Member Kamil Ugurbil. He Buck Regulators did pioneering work on ultra-high-field MRI technology used for biomedical HPE Servers Powers ABB’s Performance and Reliability and brain research.

IEEE SPECTRUM (ISSN 0018-9235) is published monthly by The Institute of Electrical and Electronics Engineers, Inc. All rights reserved. © 2019 by The Institute of Electrical and Electronics Engineers, Inc., 3 Park Avenue, New York, NY 10016-5997, U.S.A. Volume No. 56, Issue No. 2, North American edition. The editorial content of IEEE Spectrum magazine does not represent official positions of the IEEE or its organizational units. Canadian Post International Publications Mail (Canadian Distribution) Sales Agreement No. 40013087. Return undeliverable Canadian addresses to: Circulation Department, IEEE Spectrum, Box 1051, Fort Erie, ON L2A 6C7. Cable address: ITRIPLEE. Fax: +1 212 419 7570. INTERNET: [email protected]. ANNUAL SUBSCRIPTIONS: IEEE Members: $21.40 included in dues. Libraries/institutions: $399. POSTMASTER: Please send address changes to IEEE Spectrum, c/o Coding Department, IEEE Service Center, 445 Hoes Lane, Box 1331, Piscataway, NJ 08855. Periodicals postage paid at New York, NY, and additional mailing offices. Canadian GST #125634188. Printed at 120 Donnelley Dr., Glasgow, KY 42141-1060, U.S.A. IEEE Spectrum circulation is audited by BPA Worldwide. IEEE Spectrum is a member of the Association of Business Information & Media Companies, the Association of Magazine Media, and

Association Media & Publishing. IEEE prohibits discrimination, harassment, and bullying. For more information, visit http://www.ieee.org/web/aboutus/whatis/policies/p9-26.html. GLUEKIT PAUL; D. JON UNIVERSITY; NORTHWESTERN LEFT: FROM

02 | FEB 2019 | NORTH AMERICAN | SPECTRUM.IEEE.ORG JOHNS HOPKINS UNIVERSITY “This is thefirsttimein career is my that I’veto save“This had anopportunity THE IMPROVER indicated at the bottom of each page. Both have the same same the have Both page. each of bottom the at indicated Spectrum page CITING ARTICLES IN IEEE IN ARTICLES CITING T 2001, hecontinued2001, to invent andimprove, working insuch areas as was inducted intowas theNational inducted Academy in1998,the ofEngineering diagnose pneumonia,atop killerofchildren inthe developing world. electret microphone,electret anow-ubiquitous usedinhearing technology experimentation invention, ledto his withGerhard Sessler, ofthe experiments, which weren’t of ableto pulses transmitultraprecise Elhilali describe in this issue, is an AI-enabled stethoscope to help anAI-enabled is issue, inthis Elhilali describe Medal ofTechnology andInnovation in2006. National Inventor’s Hall ofFame in 1999, andwas awarded theNational architectural andteleconferencing. acoustics West, an IEEELife Fellow, aids, computers, andmobilephones. find asolution to amajor problem oftheirday.” lives,” West says. “Whoknows, maybe we oneofthebabies save will later at Johns Hopkins University, inBaltimore, which hejoinedin sound. West builtanewwithtransducer made kindofheadphones summer internship Telephone at Bell Laboratories. to comebackover break to seewhat Christmas Icould do.” Further thingsworkthese very well, butonly for alimited time.’ SoIagreed from a solid dielectric material. Thenhewentfrom backto asoliddielectric college. “In November, Igot acall,” West remembers. said,‘Gee whiz, “They After graduating in1957, West staff. There and Labs joinedtheBell His latest invention,His which heandJohns Hopkins Mounya colleague His supervisors asked supervisors His himto work usedinacoustics onheadphones BACK STORY_ n umbers may differ. In citations, you should should you citations, In differ. may umbers , V IEEE Spectrum IEEE in or 48, p. February 2019, 2(INT), no. ol. 56, where segregationlaws racial family deniedhis many he invention that made James James made that invention he every step forwardIn1953, education. inhis heentered Buthechallengedopportunities. quo thestatus with Philadelphia’s Temple University, andhesoonlandeda He grew theGreat upduring Depression inrural Virginia, his conviction thathis hecould make thingsbetter. West was borninto aworld indire needof improvement.

S PECTRUM PECTRUM

IEEE Spectrum IEEE i nclude the issue designation. For For designation. issue the nclude e publishes an international and a ditorial content, but because of of because but content, ditorial E . W , Vol. est

56, no. 2 (NA), February 2019, p. 52. p. February 2019, 2(NA), no. 56, famous came out of famous came n e

IEEE IEEE in is Forward Past xample, N d ifferences in advertising, advertising, in ifferences orth American edition, as

02.19 IEEE. gateway to Your mobile APP: IEEE THE your fingertips. at and getIEEE Download now CONTRIBUTORS_

EDITOR IN CHIEF ADVERTISING PRODUCTION MANAGER David Brumley Susan Hassler, [email protected] Felicia Spagnoli, [email protected] EXECUTIVE EDITOR SENIOR ADVERTISING PRODUCTION COORDINATOR Brumley spent five years working as Glenn Zorpette, [email protected] Nicole Evans Gyimah, [email protected] a chief information security officer, EDITORIAL DIRECTOR, DIGITAL EDITORIAL ADVISORY BOARD “chasing people who broke into my systems,” he Harry Goldstein, [email protected] MANAGING EDITOR Susan Hassler, Chair; David C. Brock, Sudhir Dixit, Limor says. “I wanted to do something about it, and so Elizabeth A. Bretz, [email protected] Fried, Robert Hebner, Joseph J. Helble, Grant Jacoby, Leah I’ve spent the past 15 years at Carnegie Mellon SENIOR ART DIRECTOR Jamieson, Jelena Kovacevic, Deepa Kundur, Norberto University trying to find an autonomous solution.” Mark Montgomery, [email protected] Lerendegui, Steve Mann, Allison Marsh, Jacob Østergaard, His system finds and patches vulnerabilities that SENIOR EDITORS Umit Ozguner, Thrasos N. Pappas, H. Vincent Poor, John Rogers, bad guys might exploit. Brumley’s article, in this Stephen Cass (Resources), [email protected] Jonathan Rothberg, Umar Saif, Takao Someya, Maurizio Vecchione, Yu Zheng, Kun Zhou, Edward Zyszkowski issue [p. 30], describes how the system won the Erico Guizzo (Digital), [email protected] Jean Kumagai, [email protected] 2016 DARPA Cyber Grand Challenge. Samuel K. Moore, [email protected] MANAGING DIRECTOR, PUBLICATIONS Michael B. Forster Tekla S. Perry, [email protected] EDITORIAL CORRESPONDENCE Philip E. Ross, [email protected] IEEE Spectrum, 3 Park Ave., 17th Floor, David Schneider, [email protected] New York, NY 10016-5997 Charles Q. Choi DEPUTY ART DIRECTOR TEL: +1 212 419 7555 FAX: +1 212 419 7570 Brandon Palacio, [email protected] BUREAU Palo Alto, Calif.; Tekla S. Perry +1 650 752 6661 Choi is a freelance science journalist PHOTOGRAPHY DIRECTOR Randi Klett, [email protected] based in New York City. In his spare ASSOCIATE ART DIRECTOR Erik Vrielink, [email protected] DIRECTOR, BUSINESS DEVELOPMENT, SENIOR ASSOCIATE EDITOR time, he’s working on a science fiction series MEDIA & ADVERTISING Mark David, [email protected] that he describes as “cosmic in scale.” In this Eliza Strickland, [email protected] issue, Choi reports on “electroceuticals,” medical NEWS MANAGER Amy Nordrum, [email protected] ADVERTISING INQUIRIES ASSOCIATE EDITORS devices that use electrical stimulation to promote NAYLOR ASSOCIATION SOLUTIONS Willie D. Jones (Digital), [email protected] Erik Henson +1 352 333 3443, [email protected] healing [p. 7]. One expert he interviewed said that Michael Koziol, [email protected] astronauts’ suits may someday use electrical fields SENIOR COPY EDITOR Joseph N. Levine, [email protected] REPRINT SALES +1 212 221 9595, ext. 319 to disinfect themselves. That far-out proposal COPY EDITOR Michele Kogon, [email protected] REPRINT PERMISSION / LIBRARIES Articles may be could be fodder for Choi’s sci-fi series. EDITORIAL RESEARCHER Alan Gardner, [email protected] photocopied for private use of patrons. A per-copy fee must ADMINISTRATIVE ASSISTANT be paid to the Copyright Clearance Center, 29 Congress Ramona L. Foster, [email protected] St., Salem, MA 01970. For other copying or republication, CONTRIBUTING EDITORS Evan Ackerman, Mark Anderson, contact Managing Editor, IEEE Spectrum. Robert N. Charette, Peter Fairley, Tam Harbert, Mark Harris, Mounya Elhilali David Kushner, Robert W. Lucky, Prachi Patel, Morgen E. Peck, COPYRIGHTS AND TRADEMARKS IEEE Spectrum is a Elhilali directs Johns Hopkins University’s Richard Stevenson, Lawrence Ulrich, Paul Wallich registered trademark owned by The Institute of Electrical DIRECTOR, PERIODICALS PRODUCTION SERVICES Peter Tuohy and Electronics Engineers Inc. Reflections, Spectral Laboratory for Computational Audio EDITORIAL & WEB PRODUCTION MANAGER Roy Carubia Lines, and Technically Speaking are trademarks of IEEE. Perception, where she studies how human brains SENIOR ELECTRONIC LAYOUT SPECIALIST Bonnie Nani Responsibility for the substance of articles rests upon the interpret sounds in noisy environments. She PRODUCT MANAGER, DIGITAL Shannan Dunlap authors, not IEEE, its organizational units, or its members. worked with colleague James E. West to design a WEB PRODUCTION COORDINATOR Jacqueline L. Parker Articles do not represent official positions of IEEE. Readers smart stethoscope [p. 36] that detects the telltale MULTIMEDIA PRODUCTION SPECIALIST Michael Spector may post comments online; comments may be excerpted for sounds of pneumonia. The artificially intelligent ADVERTISING PRODUCTION +1 732 562 6334 publication. IEEE reserves the right to reject any advertising. tool uses signal-processing tricks from Elhilali’s neuroscience research. “It was a great opportunity to try out the theories we’d developed,” she says.

IEEE BOARD OF DIRECTORS CHIEF MARKETING OFFICER Karen L. Hawkins Mathias Johansson PRESIDENT & CEO José M.F. Moura, [email protected] +1 732 562 3964, [email protected] Johansson is cofounder and CEO of Dirac +1 732 562 3928 FAX: +1 732 465 6444 CORPORATE ACTIVITIES Donna Hourican PRESIDENT-ELECT Toshio Fukuda +1 732 562 6330, [email protected] Research, which develops algorithms to TREASURER Joseph V. Lillie SECRETARY Kathleen A. Kramer MEMBER & GEOGRAPHIC ACTIVITIES Cecelia Jankowski improve automotive sound systems, home audio PAST PRESIDENT James A. Jefferies +1 732 562 5504, [email protected] equipment, smartphones, and headphones. The VICE PRESIDENTS Witold M. Kinsner, Educational Activities; Hulya Kirkici, STANDARDS ACTIVITIES Konstantinos Karachalios work is based on research he did as a student at Publication Services & Products; Francis B. Grosz Jr., Member +1 732 562 3820, [email protected] Uppsala University, in Sweden. In “VR for Your & Geographic Activities; K.J. “Ray” Liu, Technical Activities; EDUCATIONAL ACTIVITIES Jamie Moesch Ears” [p. 24], he describes the company’s work in Robert S. Fish, President, Standards Association; Thomas M. +1 732 562 5514, [email protected] Coughlin, President, IEEE-USA creating 3D audio for virtual reality. Johansson is a CHIEF FINANCIAL OFFICER & DIVISION DIRECTORS ACTING CHIEF HUMAN RESOURCES OFFICER guitarist as well as an audiophile, and he recently Renuka P. Jindal (I); David B. Durocher (II); Sergio Benedetto performed at a smartphone vendor’s conference. (III); John P. Verboncoeur (IV); John W. Walz (V); Manuel Castro Thomas R. Siegert +1 732 562 6843, [email protected] (VI); Bruno Meyer (VII); Elizabeth L. “Liz” Burd (VIII); Alejandro TECHNICAL ACTIVITIES Mary Ward-Callan “Alex” Acero (IX); Ljiljana Trajkovic (X) +1 732 562 3850, [email protected] REGION DIRECTORS MANAGING DIRECTOR, IEEE-USA Chris Brantley Babak Dastgheib-Beheshti (1); Wolfram Bettermann (2); +1 202 530 8349, [email protected] V.S. Shridhar Gregg L. Vaughn (3); David Alan Koehler (4); Robert C. Shapiro (5); Keith A. Moore (6); Maike Luiken (7); Magdalena IEEE PUBLICATION SERVICES & PRODUCTS BOARD Shridhar is senior vice president Salazar-Palma (8); Teófilo J. Ramos (9); Hulya Kirkici, Chair; Derek Abbott, John Baillieul, Sergio and head of the Internet of Things Akinori Nishihara (10) DIRECTOR EMERITUS Theodore W. Hissey Benedetto, Ian V. “Vaughan” Clarkson, Eddie Custovic, Samir division at Tata Communications, in Chennai. M. El-Ghazaly, Ron B. Goldfarb, Larry Hall, Ekram Hossain, In “The India of Things” [p. 42], he describes IEEE STAFF W. Clem Karl, Om P. Malik, Aleksander Mastilovic, Carmen the immense opportunities that large-scale IoT EXECUTIVE DIRECTOR & COO Stephen Welby S. Menoni, Paolo Montuschi, Lloyd A. “Pete” Morley, Sorel infrastructure can bring to India. “Food, clean +1 732 562 5400, [email protected] Reisman, Gianluca Setti, Gaurav Sharma, Maria Elena Valcher, CHIEF INFORMATION OFFICER Cherif Amirat John Vig, Steve Yurkovich, Bin Zhou, Reza Zoughi water, electricity, access to health care—things we +1 732 562 6017, [email protected] IEEE OPERATIONS CENTER take for granted in developed countries may be a GENERAL COUNSEL & CHIEF COMPLIANCE OFFICER 445 Hoes Lane, Box 1331 struggle in the developing nations, and IoT can help Jack S. Bailey, +1 212 705 8964, [email protected] PUBLICATIONS Michael B. Forster Piscataway, NJ 08854-1331 U.S.A. tremendously,” Shridhar said in a recent interview. +1 732 562 3998, [email protected] Tel: +1 732 981 0060 Fax: +1 732 981 1721

04 | FEB 2019 | NORTH AMERICAN | SPECTRUM.IEEE.ORG IoT calls for fast communication between sensors.

Visualization of the normalized 3D far-field pattern of a slot-coupled microstrip patch antenna array.

Developing the 5G mobile network may not be the only step to a fully functioning Internet of Things, but it is an important one — and it comes with substantial performance requirements. Simulation ensures optimized designs of 5G-compatible technology, like this phased array antenna. The COMSOL Multiphysics® software is used for simulating designs, devices, and processes in all fields of engineering, manufacturing, and scientific research. See how you can apply it to 5G and IoT technology designs. comsol.blog/5G SPECTRAL LINES_ 02.19

LONG-RANGE THINKERS: Paul Saffo [left], Erika Gregory, Ben Rattray, and Erika Woolsey discuss how Doug Englebart’s approach of using technology to augment human capabilities can help address today’s toughest problems.

the limiting factor in effecting change; it’s translating scientific discovery into understanding and action. To push understanding forward, Woolsey cofounded The Hydrous. Ini- tially, the organization focused on creating open-access 3D models of coral reefs; it is now moving on to developing ocean experiences for virtual reality. “You can’t get people to protect something they haven’t experienced,” she said. For Engelbart, an important step in the development of tech tools was the tech WHAT WOULD prototype demo. Woolsey would like to do a demo of her own: “I’d take people in landlocked states, where they don’t have DOUG ENGELBART DO? any access to the ocean, on virtual dives.” The man who helped pioneer modern computing and Nuclear proliferation. NSquare man- organizational-knowledge management could also help us aging director Erika Gregory reminded solve our greatest global problems attendees that the way Americans think of nuclear weapons was locked in more FIFTY YEARS AGO LAST DECEMBER, Doug Engelbart than 70 years ago, when the United States first devel- demonstrated some of the computing tools we all still use oped its nuclear capability. Those early nuclear strat- today—the mouse, hypertext, and windows. His goal wasn’t egies shaped politics and culture, she said, and have to revolutionize computing per se; it was to give people bet- “actually hurt [our democracy]; there is not a less ter tools for collaboration, which he hoped would lead to better ways democratic process than [that for] nuclear weapons. to fix the real problems of society. • “He thought the single greatest We created an insular group of people and ceded existential challenge of our time was to raise the curve on our ability to responsibilities for decisions to them.” collectively solve problems,” said Christina Engelbart, Doug Engelbart’s Now, she said, that era’s reliance on deterrence is daughter, who spoke at a symposium honoring her father at the Computer breaking down, “as nonstate actors are getting nuclear History Museum in Mountain View, Calif., in December. Ms. Engelbart is material and a guy dropping a socket down a missile the executive director and cofounder of the Engelbart Institute. • How silo can nearly destroy a chunk of the nation.” are we doing in terms of meeting that challenge? According to Christina, Not accepting the idea that nuclear weapons are her father would have said, “Great effort, guys, but there’s one problem.” the purview of only a few, Gregory is building a That problem is that technology evolves at a much faster pace than the network of diverse teams to work on the problem. organizations that manage them. “The explosion of unchecked digital She already had her demo—last November, at the technology is a threat multiplier, and that gap is a cause for concern,” Rhode Island School of Design. It was a proof of she said. She remains hopeful that organizations can catch up. • To get concept, showing that collaborative teams work- there, said futurist Paul Saffo, “we need more Engelbarts, more people ing across sectors, with experts in the room, could who are long-distance thinkers.” He then brought some experts on stage produce better results than experts alone. who are working on some of the most challenging problems of our time— We think Doug would have been pleased. using Engelbartian approaches. —Tekla S. Perry Here’s a sampler: A version of this article appears in our View From Climate change. Stanford marine biologist Erika Woolsey has researched the Valley blog. coral reefs for a long time. Documenting the dramatic bleaching of the

Great Barrier Reef, Woolsey began thinking that knowledge wasn’t ↗ POST YOUR COMMENTS at https://spectrum.ieee.org/engelbart0219 FAIRBAIRN/CHM DOUGLAS

06 | FEB 2019 | NORTH AMERICAN | SPECTRUM.IEEE.ORG 43: PAIRS OF NERVES IN THE PERIPHERAL NERVOUS SYSTEM, WHICH CONNECTS THE BRAIN AND SPINAL CORD TO THE REST OF THE BODY.

Electricity helps the ON YOUR NERVES: This implant heart beat, the mus- stimulates peripheral nerves for HEAL FASTER up to two weeks before it breaks cles twitch, and the body down and is absorbed by the body. communicate with the brain. WITH ELECTRICITY Now scientists are increasingly using electricity to promote healing. Unlike previous inventions that were often quite Electric bandages and biodegradable bulky, new “electroceutical” devices are easier to wear, and electronic implants could ward off some can even biodegrade inside the body. Here are a few infections and speed recovery designs that could someday aid patients in recovery.

Electric Bandage The body naturally generates electrical fields during healing. “Once there is a wound, cells will push ions through their membranes to generate an electric field,” which helps the cells align and grow in the direction of the wound, says materials scientist Xudong Wang at the University of

NORTHWESTERN UNIVERSITY NORTHWESTERN Wisconsin–Madison.

SPECTRUM.IEEE.ORG | NORTH AMERICAN | FEB 2019 | 07 Knowing this, Wang and his colleagues developed a bandage that con- verts mechanical energy emitted by a patient’s body motions into electricity. “We’re looking to apply electric fields to mimic what nature does,” he says. Specifically, the bandage relies on the phenomenon known as triboelectricity, the most common cause of static electricity. When two substances repeatedly touch and then separate, the surface of one material can steal electrons from the other—which is why rubbing your feet on a carpet builds up electric charge. The bandage consists of a Teflon strip that slides back and forth over a copper-coated plastic layer. When looped around the torsos of rats, the bandages generated electrical pulses when- ever the rodents breathed and reduced the time it took for an incision to heal to just 3 days, compared with up to 12 days for the normal healing process. Scientists have known for years that electricity could help repair skin, but most electrotherapy devices today administer intense shocks. Wang says that the new bandage’s gentler pulses reduced the production of reactive oxygen species—chemicals that could potentially harm tissue—by nearly a factor of five. The bandages are made of relatively that can continuously deliver electri- ELECTROCEUTICALS: Xudong Wang common materials and are simple to cal pulses to nerves, and breaks down [top, right] applies an electric bandage that accelerates healing to graduate student Yin fabricate, suggesting they should not when it is no longer needed. The device Long. A patch by a different research group cost much more than a regular bandage, is roughly as wide as a small coin and as [bottom] disrupts bacteria that infect wounds. Wang says. The scientists now plan to test thick as a sheet of paper, and is flexible their devices on pig skin, which more enough to wrap around an injured nerve. oughly the rodents recovered nerve closely imitates human tissue, he adds. It’s powered and controlled wirelessly by signaling and muscle strength. The a transmitter outside the body, and can researchers noticed no negative side Biodegradable Implant electrically stimulate a nerve for about effects from the device or its absorption. Research has found that electric fields two weeks before the body absorbs it. The scientists now plan to test their can accelerate the repair of damaged In experiments with rats that have implants on larger animals as the next nerves, likely by triggering the release of injured sciatic nerves, which control step toward someday testing on humans. nerve-regenerating chemicals by neurons the hamstrings and muscles of the lower They also want to see if stimulating an and nearby cells. However, such therapy legs and feet, the devices provided 1 hour injured nerve for longer periods of time is currently applied only during surgery, of electrical stimulation per day for one, provides even more benefits. says materials scientist John Rogers at three, or six days. When the scientists Northwestern University, in Evanston, Ill. monitored the rats for 10 weeks, they Beating Bacteria Now Rogers and his colleagues have found that the more stimulation the rats As many as 80 percent of all wound

developed a biodegradable implant received, the more quickly and thor- infections involve microbes that have CENTER MEDICAL WEXNER UNIVERSITY STATE OHIO THE BOTTOM: MILLION-WEAVER; SAM TOP:

08 | FEB 2019 | NORTH AMERICAN | SPECTRUM.IEEE.ORG constructed slimy fortresses known as biofilms to shelter the germs from anti- A COOLER CLOUD biotics and the body’s natural defend- ers. Electric bandages may be able to NEWS A new type of thermal conductor cools data fight against biofilm infections, said centers without water, refrigerants, chillers, Chandan Sen, who developed one such or server fans bandage when he was director of Ohio State University’s center for regenerative medicine and cell-based therapies. Data centers are hun- The computer equipment in a Scientists had previously tried prevent- gry, hot, and thirsty. typical data center runs at about ing biofilms with drugs, but bacteria The approximately 15 megawatts, devoting 1 MW of quickly evolve to resist those drugs. (In 3 million data centers that power to server fans. But August, Sen became director of a new in the United States consume bil- such a data center would require regenerative medicine center at Indi- lions of liters of water and about an additional 7 MW (for a total load ana University.) 70 billion kilowatt-hours of elec- of 22 MW) to power other cooling Sen and his colleagues noted that bac- tricity per year, or nearly 2 percent equipment, and it would need teria rely heavily on electrical charges of the nation’s total electricity use. 500 million liters of water per year. to stick to surfaces and to communicate. About 40 percent of that energy At a time when data-center traffic “Bacteria have an electrical ecosystem, runs air conditioners, chillers, is expected to double every two which, if you perturb, takes away their server fans, and other equipment years, the industry’s appetite for ability to adhere and to communicate to keep computer chips cool. electricity and water could soon with each other,” Sen says. Now, Forced Physics, a com- reach unsustainable levels. Sen’s team developed bandages pany based in Scottsdale, Ariz., According to Forced Physics’ that have silver and zinc printed onto has developed a low-power sys- chief technology officer, David them, and generate a weak electric tem that it says could slash a data Binger, the company’s conduc- field when moistened with bodily flu- center’s energy requirements for tor can help a typical data cen- ids such as sweat or blood. In experi- cooling by 90 percent. The com- ter eliminate its need for water ments on pigs with burn wounds, the pany’s JouleForce conductor is or refrigerants and shrink its bandages prevented biofilms from form- a passive system that uses ambi- 22-MW load by 7.72 MW, which ing when applied within 2 hours after ent, filtered, nonrefrigerated air translates to an annual reduction the scientists infected the burns with to whisk heat away from computer of 67.6 million kWh. That data bacteria. The bandages could also dis- chips. In February, Forced Phys- center could also save a total of rupt biofilms that had formed when ics plans to launch its first on-site US $45 million a year on infrastruc- applied seven days after infection, help- pilot test at a commercial facility in ture, operating, and energy costs ing the pig’s white blood cells attack Chandler, Ariz., owned by H5 Data with the new system, according to the infections. Centers. There, a rack of 30 conduc- Binger. “We are solving the prob- Vomaris Innovations, in Tempe, tors will cool IT equipment con- lem that electrons create,” he said. Ariz., is now commercializing this suming 36 kilowatts, as sensors In today’s data centers, cir- technology. Sen, who holds a stake track airflow, temperature, power cuit boards and other electronic in the company, says this research is usage, and air pressure. Informa- also currently in a U.S. Department of tion gleaned from the one-year test Defense–funded clinical trial on burn will be used to demonstrate per- victims. He and his colleagues are inde- formance to potential customers. pendently investigating whether these bandages can fightm ultidrug-resistant bacteria. “I think these ‘electroceuticals’ make up a field that can grow in a very big way in the near future,” Sen says. —Charles Q. Choi

↗ POST YOUR COMMENTS at https://spectrum.ieee.org/

FORCED PHYSICS FORCED electricbandage0219

COOL CHIPS: Tiny aluminum “fins” inside this metal box whisk away the heat generated by computer-server electronics.

SPECTRUM.IEEE.ORG | NORTH AMERICAN | FEB 2019 | 09 components are enclosed in a comb, neatly orienting air mole- metal containers about the size cules to point in the same direction SHAPE- of pizza boxes. Forty boxes are and arranging them into columns. stacked vertically into racks. Row At the same time, heat from the after row of these racks, arranged attached electronics transfers into SHIFTING side by side in narrow aisles, fill the aluminum fins. When air mole sprawling one-story buildings. cules contact the fins’ surfaces, ELECTRODES An elaborate ventilation network they pick up heat from the metal, blows chilled air onto the front of which prompts them to accelerate the racks. Small fans at the rear out the back. FOR THE of each box draw the cool air The hotter the air is as it exits over the electronics inside. Then, the conductor, the better. In doz- BRAIN larger fans at the back of the rack ens of lab tests with ambient air suck out the heated air. temperatures between 21 °C and Materials that have memory The JouleForce conductor is a 49 °C, the air exiting the JouleForce could make medical narrow box that looks like it could conductor measured around 65 °C— implants easier to place hold a couple of long-stem roses. which is 27 °C hotter than with con- Circuit boards are not housed ventional cooling systems. inside but rather are attached to “It’s very efficient,” says Richard Getting neurons to commu- the outside. It takes 40 conductors, Madzar, head of critical systems at nicate with electronics has made entirely of aluminum, to fill a H.F. Lenz Co., a firm that designs always been hard—hard on standard rack. There are no small data centers. Madzar, who is not the neurons, that is. Arrays server fans—instead, four large fans affiliated with Forced Physics, has of rigid metal electrodes implanted in at the rear of the rack suck ambient seen demonstrations of the con- the brain pierce blood vessels and dis- air through the conductors. The ductor in laboratories under sim- lodge support cells, causing the body to fans are the only moving parts ulated conditions. “It eliminates cover up the array with an insulating scar, in the system, and the air itself is the server fans and requires less which prevents many incoming signals never chilled. For the average data power than they would have oth- from getting through. Engineers now center, these large fans need just erwise consumed,” he says. He also think shape-memory materials could do 0.28 MW—a fraction of the 1 MW likes that the conductor is reusable, the job much better, because they can that server fans generally use. modular, and recyclable. be programmed to snake around blood Inside the conductor, 3,000 very How data-center managers will vessels and climb nerves like a vine. thin aluminum fins are lined up in respond to this new approach is One of these materials, an alloy of two rows that form a V. The V shape unclear. Their business models are nickel and titanium called nitinol, eas- moves air easily and quickly out the based on cooling technologies that ily moves between shapes depending back. As a fan sucks air through a are already widely available, says on the temperature. A target shape can conductor, low air pressure draws Madzar. They may have heard of be programmed by bending cool n itinol it into the microchannels between other kinds of conductive cooling into position, heating it to a critical tem- the fins. The fins work like teeth in that rely on water or refrigerants, perature, and holding it at that temper- for instance, but these systems are ature for a period of time. Thereafter, primarily used in supercomput- even if you bend it into another shape, ers and tend to be more expensive heating the alloy will transform it back than conventional cooling technol- into the programmed shape. “We are solving ogies. If the JouleForce conductor Researchers in Duygu Kuzum’s lab at the problem is to move into everyday use, it will the University of California, San Diego, have to be more widely available figured that electrodes made from nitinol that electrons and competitively priced. “I need might be a good fit for long-term brain create” to see it in a catalog,” says Madzar. implants. Today’s standard is an array of —Tracy Staedter spikelike microelectrodes that is pressed into brain tissue, where it connects with — David Binger, ↗ POST YOUR COMMENTS at https://spectrum.ieee.org/

Forced Physics staycool0219 and records signals from nearby neu- (2) DIEGO SAN CALIFORNIA, OF UNIVERSITY

10 | FEB 2019 | NORTH AMERICAN | SPECTRUM.IEEE.ORG The technology is a long way from use in humans, Kuzum says. But her team is exploring the idea with surgeons at UCSD as a possible improvement to electrodes for deep-brain stimulation, which are now implanted to treat Parkinson’s disease. In China, a separate group of engineers—at Tsinghua University, in Beijing, and at Zhejiang University, in Hangzhou—is seeking a less-damaging, easier-to-implant electrode for nerves rons. This type has been successfully rather than the brain. Their project used in brain-computer interfaces that was inspired by the way vines climb have let people control machines with poles and trees, Yingchao Zhang from their minds. Tsinghua University told engineers at However, the implantation procedure IEDM. Instead of nitinol, the team tears at the brain’s vasculature, stresses used a shape-memory polymer. The its astrocyte support cells, and causes substance works in a similar way, but other damage, as UCSD graduate student the change in shape is triggered by Ruoyu Zhao explained to engineers in body temperature. December at the IEEE Electron Devices Zhang and his colleagues constructed Meeting (IEDM), in San Francisco. In a 100-μm-thick substrate from a shape- response, the body walls off the area memory polymer and deposited an near the electrode array with scar tissue, even thinner pattern of gold onto it to which “leads to a dense and electrically conduct nerve signals. Then, they pro- inactive site around the implant,” he says. SNAKES IN A BRAIN: Shape-memory alloy grammed the polymer to bend itself microelectrodes [top] would be programmed to into a tight corkscrew. So the number of neurons the implant bend their way around the brain’s blood vessels. can sense drops off rapidly over time. In one experiment, they implanted the Kuzum and her group figured that if trodes through a 210-μm-wide needle. device on a rabbit’s vagus nerve. When the microelectrodes could steer around They tested the system by inserting it the unwound polymer encountered the blood vessels, the body’s response would into a 3D-printed model of a mouse animal’s body heat, it snaked its way be much less severe. The idea is to take brain. As the injector pushed the elec- around the nerve. Zhang and his col- a high-resolution CT scan of the implant trodes in, it heated them with current. leagues then tested the connection by area, map a safe path for the electrodes, This set the electrodes at their pro- stimulating the nerve to alter the rab- and program microelectrodes to form grammed shapes, bending them around bit’s heart rate. the shape of that path as they’re inserted the model’s blood vessels. Companies have explored vagus nerve into the brain. In a test in real mouse brains, the elec- stimulation to treat heart failure, though The team started by making a bun- trodes were able to distinguish the sig- so far without commercial success. How- dle of 16 nitinol electrodes, each with a nals of individual neurons and caused ever, it is currently a treatment for epi- diameter of just 23 micrometers. (The minimal damage at the implant site. lepsy and drug-resistant depression, and diameter of the smallest form of human However, the UCSD team has not yet is under trial for enhancing stroke recov- neuron is about 4 μm.) The group then shown that the implants maintain their ery, and for treating Crohn’s disease, fitted these electrodes into a mold in the ability to pick up neural signals better migraines, and other maladies. shape of the path the electrodes must than standard microarrays do over a Electrodes that slither through your travel through a model of the brain. Run- long period. Kuzum says the team will do brain and body might seem a bit creepy, ning 155 milliamperes through each elec- one-year implant experiments in mice but at least they will adhere to the old trode for 10 seconds heated it sufficiently in the future. Their more immediate physician’s axiom: First, do no harm. to program the shape. goal is to make the electrodes even nar- —Samuel K. Moore Kuzum’s group built an injection rower, perhaps smaller than the neural ↗ POST YOUR COMMENTS at https://spectrum.ieee.org/

device that feeds a bundle of microelec- cell bodies themselves. neuralelectrodes0219

NEWS

SPECTRUM.IEEE.ORG | NORTH AMERICAN | FEB 2019 | 11 CHINA AND JAPAN DRIVE A GLOBAL EV CHARGING EFFORT

The new standard will be backward compatible NEWS RECHARGING: A Nissan Leaf fills up at with select charging stations a CHAdeMO charging station.

Two leading industry groups, chargers for electric vehicles: 22,647 The new EV charger standard is code- Japan’s CHAdeMO and the China units operating in 71 countries as of named ChaoJi. Like its predecessors, the Electricity Council, announced last year September. This includes more than ChaoJi standard will use the Controller they would codevelop an ultrafast charg- 2,900 in North America and 7,900 Area Network (CAN) bus to coordinate ing protocol for electric vehicles. Now, in Europe. communications between the electronic the partners, under the supervision of CHAdeMO’s Chinese counterpart, control units for features such as airbags the Japanese and Chinese governments, the China Electricity Council (CEC), and audio systems, without requiring are inviting other countries to join them. counts 270,000 chargers using its GB/T dedicated wiring. Their goal is to develop a global standard standard installed in China and India. Maximum power for the new stan- for all types of EVs by 2020. Together, the two groups account for dard is tentatively set at 900 kW CHAdeMO, a consortium of automo- more than 90 percent of the installed EV (1,500 × 600 amperes), which will be tive, power-generation, and IT compa- fast-charger market—that is, DC charg- capable of quickly charging large vehi- nies (including Nissan and Volvo), has ing stations up to 120 kilowatts that con- cles such as earth-moving equipment, the largest global installation of DC nect directly to the battery. buses, trucks, and helicopters. Whereas

Expand Your Professional Network With IEEE With over 430,000 members in over 160 countries, IEEE makes it easy for you to connect with colleagues who share your expertise or interests. Become involved in our various societies, affinity and special interest groups and watch your professional network grow. PHOTONICS SPIE. WEST Visit www.ieee.org/join and start connecting today. FEBRUARY 5-7, 2019 BOOTH #4482 NORTH HALL

YOUR OPTICAL COATING PARTNER alluxa.com NISSAN

12 | FEB 2019 | NORTH AMERICAN | SPECTRUM.IEEE.ORG Alluxa_IEEE_Quarter.indd 1 1/14/19 3:45 PM a 50-kW c harger takes about 30 minutes magnetic compatibility. To reduce this Just how many countries take up that to charge a 25-kilowatt-hour battery today, burden, Sagawa says, the new standard invitation will likely determine whether a 900‑kW charger of the future could should maintain a close affinity with the the new effort creates a truly global charge a 450‑kWh battery in 30 minutes. existing standards. standard or merely adds to the present Before that happens, though, Meanwhile, CHAdeMO and the CEC confusion. —John Boyd CHAdeMO must modify its specifications are pushing ahead. Yoshida says that, for upcoming chargers to handle 350 kW provided there is agreement on back- An extended version of this article appears and 500 kW-plus in the new ChaoJi stan- ward compatibility and the CAN bus, “we in our Energywise blog. dard. The specifications will also require welcome countries who are interested to ↗ POST YOUR COMMENTS at https://spectrum.ieee.org/ manufacturers to include a new con- participate in developing the standard.” evcharging0219 nector design and liquid cooling cables. Makoto Yoshida, secretary general of the CHAdeMO Association and general manager at Nissan, said in a recent press New Version! briefing in Tokyo that the new protocol will be backward compatible with the present CHAdeMO and GB/T standards. Yoshida added that the partners are also thinking of developing a standard that covers vehicles such as scooters, fork- lifts, and lightweight cars. Such a charger would be rated at between 2 and 20 kW. Despite the current dominance of the CHAdeMO and GB/T standards, a third standard, developed by SAE Inter national, could pose a long-term threat to ChaoJi. Named Combined Charging System (CCS), it’s backed by BMW, Ford, General Motors, Volkswagen, and others. Although CCS was introduced in 2014, (five years after CHAdeMO’s debut), its acceptance is reportedly growing fast. And a fourth proprietary standard, devised by Tesla, is also gaining ground. Confusion caused by incompatible charger standards is one obstacle pre- venting faster EV adoption. Other hurdles include high price, limited charging infrastructure, driving-range anxiety, and slow charging times. Should CHAdeMO and the CEC gain support for an industry-wide standard for ultrafast charging, it could eliminate some of these hurdles. But the introduc- Over 75 New Features & Apps in Origin 2019! tion of another standard will likely be a For a FREE 60-day major headache for hardware develop- Over 500,000 registered users worldwide in: evaluation, go to ◾ ers, says Kouhei Sagawa, an assistant 6,000+ Companies including 20+ Fortune Global 500 OriginLab.Com/demo ◾ 6,500+ Colleges & Universities and enter code: 8547 professor in electrical engineering at ◾ 3,000+ Government Agencies & Research Labs Tokai University, in Tokyo. “This will lead to an increase in the development period and in the cost,” he says. Sagawa points out that manufacturers 25+ years serving the scientific & engineering community need to change the electrical design of cars for each new standard, swap out hardware mountings, and test electro-

SPECTRUM.IEEE.ORG | NORTH AMERICAN | FEB 2019 | 13 14 | FEB 2019 | NORTH AMERICAN | SPECTRUM.IEEE.ORG PHOTOGRAPH BY The Yomiuri Shimbun/AP BIG BAD WOLF

EVEN WITH TODAY’S modern implements, teasing a healthy crop out of the soil requires a huge commitment of time and energy. It’s easy to see why a Japanese agricultural cooperative was looking for a way to protect local rice and chestnut crops from wild boars that were treating rows of plantings like an open- air buffet. Last spring, the group introduced the solar-powered Super Monster Wolf. The robot has a sensor that causes its head to swivel, its flashing red LED eyes to light up, and its “voice” to emit a loud howl when it detects an animal nearby. The electronic sentries, which sell for 514,000 yen (about US $4,700), are said to have dramatically lowered crop losses within a kilometer of wherever they’ve been placed.

THETHE BIG BIG PICTUREPICTURE newsNEWS

SPECTRUM.IEEE.ORG | NORTH AMERICAN | FEB 2019 | 15 RE-CREATING THE SIGSALY QUANTIZER THIS 1943 ANALOG-TO-DIGITAL CONVERTER GAVE THE ALLIES AN UNBREAKABLE SCRAMBLER

N THE YEARS BEFORE WORLD WAR II, GERMAN INTELLIGENCE COULD DECODE BAND-SCRAMBLED U.S. I radiotelephone conferences. After Pearl Harbor, an unbreakable speech scrambler was developed with top priority, and by 1943, it was deployed. Known as SIGSALY, the device pioneered many advances critical to modern digital media technologies, including spread-spectrum communications and the first use of pulse-code modulation (PCM) to transmit speech. • SIGSALY was top secret, so even today information about the details of its construction are hard to come by. I’ve spent 20 years re- searching the history of digital technology and digital media, especially SIGSALY. I searched IEEE and U.S. National Security Agency (NSA) journals, and Bell Telephone Laboratories patents. Finally, I found Lieut. Donald Mehl, a WWII SIGSALY technician, who gave me invaluable assistance. In 2015, I realized that it might be possible to re-create a key element of SIGSALY—the quantizer—using vintage parts. • SIGSALY was unbreakable because, unlike earlier analog systems, it scrambled voices by using a one-time random digital encryption key. Before a digital key can be applied, a speaker’s voice must first be converted from analog to digital, thus the RESOURCES_HANDS ON quantizer. • Modern analog-to-digital conversion makes it easy to capture the entire audio spectrum. With the technology available at the time, the creators of SIGSALY used 12 speech parameters that best encoded speech. SIGSALY used vocoders to analyze incoming speech and classify them into 10 frequency sub-bands; a pitch parameter; and a bit that indicated whether the speech sound was voiced or unvoiced. • SIGSALY had 72 identical quantizers to digitize the 12 vocoder parameters and the encryption key (stored as a vinyl recording of noise). The digitized voice and key were enciphered by modulo six addition. Each quantizer em-

ployed fiveVT-109/2051 thyratrons. Thyratrons are a type of electronic tube, but they differ from conventional ones in that they NSA (3); PAUL D. JON LEFT: FROM CLOCKWISE

16 | FEB 2019 | NORTH AMERICAN | SPECTRUM.IEEE.ORG collected since the 1950s. I searched eBay for original VT-109/2051 thyratrons, and porcelain octal tube sockets. I used other 1940s-era parts (such as Allen-Bradley car- bon composition resistors) for the quantizer, while I used some modern components (like dual primary power transformers) for supporting electronics not present in the original quantizer. To display the components, I built the quantizer into a 49- by 17- by 8-centimeter clear plastic chassis. The overall cost was US $1,250. Design and construction extended over three years, and the debugging was very challenging, especially with 350 V present! Con- struction finished and I started debugging just an hour before departing from California DIVIDE AND CONQUER: The complete to Paris, where the quantizer was to be dem- SIGSALY voice scrambler was a huge machine [bottom] that relied on many onstrated at several crypto and InfoSec con- analog-to-digital converters. The heart ferences. After I added power resistors to of the converter was a resister divider solve some problems with filament voltages ladder [top photo and schematic, middle] connected to thyratrons. In my re-creation, and fixed a few wiring errors, the quantizer be- I have added neon indicator lights. gan to work a bit, but only two of the five thyratrons triggered, the microphone preamp had full-scale oscillations once a second (“motor- boating”), and the anode power was heavily overloaded as each thyratron fired. I had to stop debugging and depart. I packed the quantizer with a few tools and don’t respond in a linear way: They are off until conversion, and five neon lamps that light up in spare parts, and Air France kindly obliged the grid voltage exceeds a trigger; then an arc logarithmic sequence. One input is a dynam- me with special treatment for the delicate forms and they “latch up” and conduct until ic microphone: It’s plugged into a preamp that machine. In Paris, I had to finish debugging the anode current falls to zero, resetting them. uses a 6SL7GT dual triode to amplify its audio without a laboratory or instruments, save for The five thyratrons formed a flash converter, signal by 60 decibels. A slowly increasing ramp a voltmeter. I scrounged parts and tools at in which an incoming signal is compared to a signal is an alternate input. the Marché aux Puces (Paris flea markets), reference voltage at a number of tap points in My design uses the same five-tap logarith- and finally finished debugging at an old the circuit. In SIGSALY, the analog input volt- mic ladder and the same type of thyratron friend’s ham-radio shack! Many of the prob- age is fed into a five-tap logarithmic resistor that SIGSALY used. Instead of SIGSALY’s lems arose because the thyratrons varied divider ladder, whose taps drive the thyratron 20-ms pulsed sampling, I sample the input at considerably in grid trigger voltages. (The grids. As the input voltage increases, the five the zero-voltage crossings of the AC power. same problems of thyratron inconsistency thyratrons thus trigger in sequence (produc- The full details of the quantizer’s construc- were tackled by the original SIGSALY engi- ing not binary but a logarithmic “thermometer tion would take more room to explain than is neers and technicians.) code”). These five outputs were sampled ev- available here, but a complete description is Now, the finished unit works reliably. Ac- ery 20 milliseconds to form a six-level quan- available as a supplement to this article on- cording to the NSA and the National Cryp- tized output signal (it’s a six-level output, not line. One critical difference between thyra- tologic Museum curators, this is the first five, because zero is one level). trons and their modern descendants—silicon attempt ever to re-create any piece of In reconstructing the SIGSALY quantiz- controlled rectifiers andTRIACs —is that the SIGSALY. But best of all, old and young, tech- er I wanted to make something that could be thyratron filaments must be preheated be- nical and nontechnical audiences alike are used in practical demonstrations and also fore applying any anode voltage to prevent impressed and immediately comprehend double as a wonderful museum piece. The damage. A thermal time-delay relay gives the the underlying principles and the close con- design includes a 120/240-volt power sup- filaments 15 seconds to heat up. nections to today’s digital age. —JON D. PAUL ply, a microphone preamp, a ramp generator, For some parts, I turned to my own labora- POST YOUR COMMENTS at https://spectrum.ieee.org/ a panel meter displaying the analog-to-digital tory, which has vintage electronic parts I’ve sigsaly0219

SPECTRUM.IEEE.ORG | NORTH AMERICAN | FEB 2019 | 17 RESOURCES_GEEK LIFE

accompanying a renaissance in game de- PINBALL’S HIGH-TECH REBIRTH sign. “Pinball isn’t just back, it’s better,” he ENTHUSIASTS DELIGHT IN says. For the older machines, Zahler’s goal is ITS ELECTROMECHANICAL to make them playable again. “Depending COMPLEXITY on the condition, it takes a month or two,” he says. Sometimes plastic parts will have warped over time, but a heat gun can tease them back into shape. Other times, replacement parts are needed that are no longer available, but “3D printing is a miracle come true for pinball repairers,” says Zahler. “It’s a rewarding feeling, bringing [the machines] back from the dead.” That’s a feeling shared by Zahler’s appren- tice, Beth Senturia. A year ago, Senturia left her career as an insurance agent and retirement planner to become Modern Pinball’s manager on one condition: that Zahler teach her how to repair pinball machines. “It’s like working at Disney World, only better,” Senturia says of her job, which involves fixing older machines as they break down. This can happen up to three times a day and can mean rebuilding a flipper or diagnosing and solder- ing a broken connection. “Every day I get to see people having fun, plus I get to work with f you think pinball—the original SOUL OF THE MACHINE: Keeping a roomful of my hands and take something that was bro- electric arcade game—is dead, pinball machines running—some of them decades ken and make it not be broken,” she says. I old—is the job of Beth Senturia. A year ago Senturia you’d be wrong. Although the ar- switched careers to learn how to diagnose and Senturia was introduced to pinball by her rival of video games pushed pinball into the repair these electromechanical puzzles. father when she was a small child. When background, it’s seeing something of a glob- Modern Pinball opened, Senturia started al resurgence, with fans enjoying both clas- Designing a new machine takes about playing in tournaments (she is currently in sic and sophisticated new machines. 14 months. A typical game takes 3.5 days to 63rd place in the women’s world ranking of That was evident in December at Modern assemble and contains about 3,500 parts pinball players) and then became a museum Pinball in New York City. The occasion was and 400 meters of wire. The Beatles game is volunteer before being offered the manager’s the unveiling of a new Beatles-themed game controlled by a 1-gigahertz, dual core, i.MX 6 job. Senturia says she leaped at the chance. from Stern Pinball. Stern employs over 300 Arm-based processor. She’d always had an interest in things that people in a 10,000-square-meter plant near At the unveiling, this complexity stood in were electronic or mechanical, but didn’t Chicago. According to CEO Gary Stern, sales stark contrast to a machine just a few me- pursue it when she was younger because are brisk: “We had 40 percent sales growth ters away, the 1974 space-themed Skylab “as a middle-class girl in the Midwest, you’re in 2016, and 40 percent sales growth in game. Skylab represents the last genera- not going to do that,” she says. “But now, 2017. Coming off those years, growth has tion of pinball machines before the advent of 35 years later, I had an opportunity to learn. been a little slower, with 25 percent in 2018.” solid-state electronic control: Its “CPU” is a It was finally time to do what I wanted to do.” His company exports most of its in ventory, motor-driven shaft fitted with a row of cams Senturia looks over the twin rows of glowing, but has seen domestic sales growth as well, that open and close a bank of relay switches. flashing, ringing machines at ModernPinball helped by increasing numbers of family en- Skylab is a permanent attraction at Mod- and smiles: “I have never been happier.” tertainment centers. More of these indoor ern Pinball, which occupies a narrow store- —STEPHEN CASS amusement parks are opening up in the front. Besides being an event space and United States because finding suitable real showroom for new games, Modern Pinball A version of this article appears in our Tech estate is easier. “Empty big-box stores are is also a recognized museum. It was found- Talk blog. available because of retail’s shift to online ed in 2013 by Steve Zahler, who agrees POST YOUR COMMENTS at https://spectrum.ieee.org/

storefronts,” says Stern. there is a resurgence of interest in pinball, pinball0219 KLETT RANDI

18 | FEB 2019 | NORTH AMERICAN | SPECTRUM.IEEE.ORG RESOURCES_CAREERS

and liaise between engineering and nontech- THE STEPS TO THE C-SUITE nical teams. The point, both experts say, is to HOW TO LEAD AT EVERY position yourself as a standout dedicated to helping the company run more smoothly. By STAGE OF YOUR CAREER the end of this decade, you should have solid experience in leading projects.

Years 15–25: Help your managers achieve company goals. “Corporate stewardship is critically important,” Verhalen says. “You want to connect with the leadership team, demonstrating to the supe- riors you work for, and work with, that you are a respectful leader.” You might move to titles like supervisor, manager, and director during this time. So at the start of this stage, Verhalen says, EEs should devote themselves to setting up projects effectively, including data gathering and resource planning. “This requires not only deep understanding of the project but also where the strengths and weaknesses lie within yourself and your team,” she adds. By the middle to the end of this phase, you might have three to 10 managers who report to you, “and now, instead of being the per- hile everyone’s journey is dif- Kodi Verhalen, a past president of the Na- son who was learning everything, you’re the W ferent, many freshly minted EEs tional Society of Professional Engineers, in person teaching,” Fasano says. Delegating dream of leading major projects, or Alexandria, Va. Verhalen gives an example from tasks can be especially challenging for detail- ascending to the C-suite of a large company. her work with power companies: “For large oriented EEs, Fasano says. “Some managers If that’s your goal, engineering career experts capital management projects, a lot goes into aren’t great at this, and it can end up being the agree there are some common milestones to making the case for a project. You can volunteer fork in the road.” aim for on the way up. to put together slides, coordinate with lawyers, and generally help to get something built. The Year 25+: Make a play for senior manage- Years 1–5: Find a good place to build your point is that you’re working across the company, ment (or not). technical foundation. learning about budgets and execution.” The path to a position like CEO will be highly This stage requires patience from the striv- Some companies have hierarchies filled dependent on the industry and the company ing future leader. “When you’re fresh out of with traditional job titles; those at other firms you work for, but EEs seeking these roles school, it’s 99 percent technical work, and may vary. So Verhalen recommends asking should be authoring research papers, writ- that’s good,” says Anthony Fasano, founder of for the written requirements for these roles. ing thought-leadership articles, and speak- the career training platform Engineering Man- “Sometimes it’s an unspoken rule that every- ing at conferences. agement Institute, in Ridgewood, N.J. It’s also one at the VP level or above has an advanced “You need to recognize this is a high-level helpful to remind yourself that the best engi- degree, for example,” she notes. strategy position,” Fasano says. “You’re neering managers can jump in on the techni- To Fasano, the best way EEs can position removing yourself from any details. It is very- cal side, Fasano says. He recommends that themselves for leadership is to develop spe- big-picture, and it’s not for everyone.” as you gain expertise, you should signal your cific expertise. “I worked for a manager who “Maybe you realize you want to manage the interest in learning how projects work. Ask a was a storm-water expert, and everyone in- design team, or that the VP level is as much re- colleague or superior about how they build cluding senior management knew he was the sponsibility as you want,” concludes Verhalen. budgets, or offer to collect data for a project. go-to for this topic,” he says. “Those are the “It’s about finding over the years what draws people who are thought of first for leadership.” you in, because that’s where you’ll excel.” Years 5–15: Discover your niche, and take Perhaps you might dedicate yourself to know- —JULIANNE PEPITONE on responsibility. ing everything about one kind of semiconduc- POST YOUR COMMENTS at https://spectrum.ieee.org/

GILLIAN BLEASE/GETTY IMAGES BLEASE/GETTY GILLIAN Grab opportunities to stretch yourself, says tor. Or you might take public speaking classes stepstothetop0219

SPECTRUM.IEEE.ORG | NORTH AMERICAN | FEB 2019 | 19 RESOURCES_TOOLS & TOYS

s we highlighted last month in YOUR FAVORITE GADGETS A Resources, IEEE Spectrum has THE BEST OF OUR launched a Consumer Electronics Hall of Fame to celebrate some of the great- CONSUMER ELECTRONICS est gadgets of the last 50 years. IEEE members HALL OF FAME CONTEST could also win prizes, such as drones or smart watches, by sharing their memories of their favorite gadget. Here are some selections from the winning entries as of press time (selections have been edited and condensed for clarity).

Amiga 500: “It is the device that at the age of 8 made me decide to become an engineer. I wanted to be a part of the computer revolution. The graphics, sound, game play, and ease of use was extraordinary. There is nothing I have owned since that I have the same passion toward or fond memories of. If it was not for that device, I most probably would not have become an engineer!” —SASHA NIKOLIC

HP-25: In 1975, “I acquired a rare item, the Hewlett-Packard HP-25, which was a scientific calculator! Such things were not available in Iceland, but one of my uncles knew a sailor that was on a freighter sailing to Boston. He under- took to find a store that sold the HP-25. I paid just over US $300 for it.” —EDVARD GUDNASON

Epson HX-20: “The first personal comput- INSPIRING ELECTRONICS: For one of our Consumer Electronics Hall of Fame contest er I owned was an Epson HX-20, complete with winners, the Amiga computer was the device that put him on the path to engineering. 300-baud acoustic- | CONTINUED ON PAGE 48 BILL BERTRAM/WIKIPEDIA BILL

UA801x Series Dual Inductors for Class-D Output Filters • Ultra-low total losses and excellent L vs. I linearity up to 10 Amps for low distortion and improved sound quality • Shielded package contains both coils • Very low magnetic coupling coefﬁ cient (K<0.001) • Designed for 100 Watts into 4 Ohm load Free Samples @ coilcraft.com/Class-D

20 | FEB 2019 | NORTH AMERICAN | SPECTRUM.IEEE.ORG OP2 Courtesy of The World’s ROBOTIS Best ROBOTS GUIDE Is Here! ROBOTS.IEEE.ORG

IEEE Spectrum’s new ROBOTS site features more than 200 robots from around the world.

• Spin, swipe and tap to make • Rate robots and check • Play Faceoff, an interactive robots move. their ranking. question game. • Read up-to-date robotics • View photography, videos news. and technical specs. Check out Robots.ieee.org on your desktop, tablet, or phone now! INTERNET OF EVERYTHING_BY STACEY HIGGINBOTHAM OPINION

and one that can tell you what it saw. Add- ing face recognition and computer vision to that camera turns a product that pes- ters you with useless notifications into something actually helpful. The kitchen is a great place to see the growing usefulness of smart devices. My connected June oven has a camera and a graphics chip inside, so it can track what food is in the oven and recognize how it needs to be cooked. But true intelligence goes beyond just computer vision. With a connected device, manufacturers can embed intelligence into its accompanying app so that the user doesn’t have to think about it. For example, the Joule Sous Vide cooker doesn’t have an interface: Every- thing is embedded in the app to help cooks take the guesswork out of cooking. The cook tells the app what meat or veg- etable is in the bag and its approximate thickness, and from there the Joule sets the temp and timer on the user’s behalf. This abdication of thought to the device is why voice has been so essential in making products smarter—and more useful— even if at first glance it seems superfluous. DO YOU NEED A SMART Take the new Alexa-enabled US $60 microwave launched last year by Ama- zon as an example. When the company MICROWAVE? launched the oven, people reacted with confusion: Why give a microwave voice control if you have to put the food inside OVER THE HOLIDAYS, DROVES OF CONSUMERS bought video door- it anyway? bells, connected lights, and smart outlets that work with Amazon’s Alexa, In this case, the voice control offers an Apple’s HomeKit, or Google Home. Plenty of people unwrapped con- intuitive way to interact with the artificial nected speakers and image-processing cameras on Christmas morning. intelligence that provides the cook times • Many of these purchases will get returned. Or they’ll be thrown away after one too and settings for various foodstuffs. At the many updates or a security scare. Perhaps luckier devices will find homes with tech- end of the day, the microwave can offer a savvy friends. But most will be abandoned, in one way or another, because most of better result than if you just punched in the smart devices on the market are stupid. • Over the six years I’ve covered smart 60 seconds—even if you still have to put home devices, they’ve presented their owners with four real problems: First, the the food inside the oven yourself. devices were expensive. They also didn’t offer much functionality beyond remote That’s when the high price of a con- control from an app. Even more frustrating, getting devices from different vendors nected gadget becomes justifiable. Now to play nice together was tough. But perhaps the biggest problem is that consumers the challenge is to explain why these had no idea what to do with these devices. • Thankfully, that’s changing: Now there devices are worthwhile. Given how many are more meaningful uses for smart devices because smart devices are finally living people mocked the Alexa microwave as up to their name. Companies are now designing products that use artificial intelli- silly, it seems manufacturers haven’t suc- gence. Alongside that intelligence, the growth of voice as a user interface can now ceeded in that last bit yet. n provide effortless interactions. To see how important intelligence is, consider a • ↗ POST YOUR COMMENTS at https://spectrum.ieee.org/ camera. There’s a big difference between a camera that can tell you it saw something microwave0219

22 | FEB 2019 | NORTH AMERICAN | SPECTRUM.IEEE.ORG ILLUSTRATION BY James Steinberg NUMBERS DON’T LIE_BY VACLAV SMIL OPINION

to the second power—(MW) (GHz)2—in Progression of power density for major device types (MW)(GHz) 2 the Proceedings of the IEEE, May 1999. 107 The researchers demonstrated successive waves of innovation: The power Source: V.L. Granatstein, R.K. densities of the early tubes were first 105 Parker, and C.M. Armstrong, Proceedings of the IEEE, overtaken by the densities of magne- Vol. 87, No. 5, May 1999 trons, then by those of klystrons, and 103 finally, in the 1970s, by those of gyro- Moore’s Law Klystron tron oscillators and free-electron lasers. 10 Each family of devices followed a logistic Magnetron and crossed- ield ampli ier curve as it approached its performance 10–1 Free-electron limits before yielding to the next family. laser Between the mid-1930s and the late 1960s, the maximum power density 10–3 of gridded tubes (triode and higher) increased by four orders of magnitude. 10–5 During the same time, the power density 1930 1940 1950 1960 1970 1980 1990 2000 Year of cavity magnetrons and crossed-field amplifiers rose by five orders of magnitude; between 1944 and 1974, the maximum power density of klystrons rose by six orders. The same improvement came THE VACUUM TUBE’S for gyrotrons and free-electron lasers between the 1960s and 2000. POWER LAW If you plot the entire sequence of logistic curves on a semilogarithmic graph, the envelope of the curves forms a IN THE 48 YEARS SINCE THE INTRODUCTION of the first micro- straight line that gains nearly 1.5 orders processor, in 1971, the number of electronic components that can be of magnitude per decade. As soon as I crammed onto a given area on a chip has increased seven orders of came across the graph, I realized that magnitude. That corresponds to a doubling about every two years the ascent must be very close to the [see “Moore’s Curse,” IEEE Spectrum, April 2015]. • You might think that the per- growth rate dictated by Moore’s Law, formance of previous vacuum-tube electronics could not possibly compare with and a simple calculation confirmed the that record of improvement. Not so. It’s just that the key metric of improvement rate. Between 1935 and 2000 the aver- is different. • The diode, the simplest vacuum tube, was invented in 1904 by age annual rise of the linear envelope line John A. Fleming; three years later came Lee de Forest’s triode, and tetrodes and indicates that the growth of maximum pentodes followed in the 1920s. These “gridded” tubes use the voltage in a grid to power densities of vacuum electronics modulate current from an electron source. Work on magnetrons—another type of was almost exactly 35 percent—virtually vacuum tube that generates microwaves by squeezing electrons through a mag- identical to the mean annual rate of netic field—led to the first patent in 1935 and to the first deployment in radar, in the growth for the post-1965 crowding of United Kingdom in 1940. The klystron (used in radar, later in satellite communi- transistors on a chip. cations and in high-energy physics) was patented in 1937, and gyrotrons (generat- To be sure, the trend lines of vacuum ing power in the millimeter wavelengths, used for heating materials and plasmas) tubes and of integrated circuits involve were first introduced in the Soviet Union in the mid-1960s. • These successive different figures of merit. But it is cer- generations of tubes improved by leaps and bounds in their power density—that tainly noteworthy that the first family is, in the maximum power that can be transported through a device—which is of electronic devices improved as fast in proportional to the cross-sectional area of the circuit, which in turn is inversely its domain as the second family did in its proportional to the operating frequency. In 1960 Leon Nergaard, at that time the different domain. A kind of Moore’s Law director of RCA Microwave Research Laboratory, proposed average power den- was in effect in electronics long before sity as a figure of merit for comparing the performance growth of these diverse Gordon Moore set it down, in 1965. n devices. Four decades later Victor L. Granatstein, Robert K. Parker, and Carter ↗ POST YOUR COMMENTS at https://spectrum.ieee.org/ M. Armstrong estimated the numbers as the product of megawatts and gigahertz vacuumtube0219

SPECTRUM.IEEE.ORG | NORTH AMERICAN | FEB 2019 | 23

For Your Ears

VRDYNAMIC 3D AUDIO IS KEY TO THE IMMERSIVE EXPERIENCE BY MATHIAS JOHANSSON • ILLUSTRATION BY EDDIE GUY

SPECTRUM.IEEE.ORG | NORTH AMERICAN | FEB 2019 | 25 UT ON YOUR VIRTUAL-REALITY HEADSET AND BE TRANSPORTED to a distant planet, ducking the crossfire in a battle between alien species. Laser rifle shots whiz by your head; military shuttles hover before you; the frantic calls of comrades hail from all directions. ¶ Change the channel. ¶ Now you are courtside at a basketball game. You hear players trash- talking on the court in front of you and coaches yelling from the bench to your left. Turn your head and the sound spins with you; the announcers in the broadcast booth are in front of you, the court sounds behind. ¶ Change the channel. ¶ Now you’re at the Gothenburg Concert Hall, whoseP acoustics are ranked among the best in the world. From your front-row seat, the 109-person orchestra before you simmers quietly at first, then roars to life, its sound enveloping you. Turn your head to the left to hear the violins more strongly; turn it to the right and hear the cellos and brass section a little above the rest.

Today the technology to create the visual component of these world, you can move, and as you do so your aural virtual-reality (VR) experiences is well on its way to becoming experience changes noticeably. widely accessible and affordable. But to work powerfully, virtual And though these sophisticated surround-sound reality needs to be about more than visuals. Unless what you are setups are definitely better than the stereo that pre- hearing convincingly matches the visuals, the virtual experience ceded them, for most listeners today, evolution has gone breaks apart. backward. Modern portable music devices have moved Take that basketball game. If the players, the coaches, the people away from speakers and into headphones. That’s announcers, and the crowd all sound like they’re sitting midcourt, a step away from realistic sound because today’s head- you may as well watch the game on television—you’ll get just as phones can’t even do stereo properly, much less sur- much of a sense that you are “there.” round sound or anything approaching 3D. Unfortunately, today’s audio equipment and our widely used Here’s why. With speakers, a sound played only recording and reproduction formats are simply inadequate to through a left speaker will be heard by your left ear— the task of re-creating convincingly the sound of a battlefield on and also by your right ear, at an almost imperceptibly a distant planet, a basketball game at courtside, or a symphony later instant and with a slight attenuation. Your brain as heard from the first row of a great concert hall. processes this slight delay and attenuation and gives Sure, a stereo recording played through headphones might you an instantaneous impression of the direction of place the sound of a sports announcer in your right ear and the and distance to the sound. But when you use head- coaches’ chatter in your left. But there they would remain, no phones, the left channel reaches only your left ear. The matter how much you move around in the virtual environment. experience is artificial, and it causes some odd percep- For a lifelike experience, engineers need to duplicate the precise tions. For example, when a voice speaks equally loudly directionality and position of every sound—from above and below, in the left and the right channels of headphones, it far and near, behind and ahead—and update it dynamically as the will seem to be emanating from inside your head, not user moves within the virtual world. from some position in front of you. That’s why musi- It’s a big challenge, but not at all insurmountable. Some cal experiences can sometimes seem somewhat odd virtual-reality producers have already begun using limited, first- when heard through headphones. How do we get from generation 3D audio technology to improve on two-dimensional there to a fully three-dimensional, interactive, virtual- stereo and surround sound. And developments currently in audio experience? research laboratories, including mine here at Dirac Research, in With just two isolated channels to work with, such a Uppsala, S weden, hold out the promise of truly lifelike virtual- goal may seem unattainable. But in principle it’s not: reality audio in just a few years. Here’s how we think we’ll be able The human auditory system uses just two ears to dis- to up the ante in virtual reality. tinguish between front and back, up and down, and everything in between, so engineers should be able to create a 3D audio experience by carefully controlling TODAY’S MOST WIDELY USED AUDIO FORMAT is the timing, volume, resonance, and echo characteris- two-channel, or stereo, sound. A stereo system records two tics of each sound as it reaches the ear. It would take a signals, left and right; the listener plays them back through lot of high-speed calculations to adjust, on the fly, the a pair of loudspeakers or headphones—again, one for the left, sound coming through each channel, but fortunately, and one for the right. Surround-sound systems go beyond ste- high-speed calculations are something we can do. reo by adding a center front speaker, two or four rear speakers, Indeed, researchers throughout the audio industry and a subwoofer for dedicated bass output. Newer approaches, are working on this approach. One fortunate aspect: like Auro-3D from Auro Technologies, add speakers at different Engineers can better control the audio emerging from heights. These start to give a 3D illusion but can’t create a vir- headphones than from open-air speakers, because tual world because the speaker positions are fixed. In the real they do not need to compensate for the room’s shape

26 | FEB 2019 | NORTH AMERICAN | SPECTRUM.IEEE.ORG or objects in it, or deal with background or other stray noise or sound reflections that could distract from the virtual environment.

THE SIMPLEST WAY TO CREATE a 3D audio recording is to place microphones in some- one’s ears and record the scene in stereo from a specific position. In practice, sound engineers usually use a dummy head to do this. This is known as binaural recording, and the process has been around for a long time. Some radio stations have broadcast binaurally recorded concerts, but the technique never caught on widely. Binaural recordings have several limitations. But for virtual reality, the main problem is the fixed position of both the listener and the sound sources. Nevertheless, we can learn a lot from binaural recordings and apply what we learn to creating Shifting Sounds interactive audio for virtual reality. A binaural Head-related transfer functions (HRTFs) are measurements recording captures the varying sound intensities of how ears respond to sounds; they are affected by the shape and times of arrival at two ears from each sound of the head and ears, the position of the ears in relation to other body parts, and the position of the sound source. For this source, and it also captures changes to the sound sonogram, which demonstrates how HRTFs for the left ear that occur due to reflections and shadows caused change with the orientation of the head, researchers measured by the shape of the head, ears, and torso. Scien- six test subjects and averaged the responses. tists call these latter effects head-related transfer functions (HRTFs). A newer approach is the object-based format. Instead of assum- With a powerful computer and digital-signaling a certain playback system, an object-based recording encodes processing software, along with headphones equipped the sound field by tagging sound sources. For example, a cello, a with a position-tracking device, we can create a stan- piano, and a vocalist are identified with information about their dardized HRTF, using a dummy head or live models, positions, intensities, and other data. This method then relies on and then adjust it in real time during playback accord- smart playback devices to interpret the tags according to their ing to the listener’s head orientation and the original capabilities and emit the sounds in a manner consistent with the direction of the sound sources. tags. Dolby Atmos, introduced in 2012, and DTS:X, introduced in 2015, both use this approach. The object-based formats were initially created to improve the ere’s where we run into a speed home theater experience. Dolby Atmos–enabled home theater bump in the development pro- receivers, for example, incorporate ceiling loudspeakers. But the cess, however. Both stereo and formats can potentially be adapted for virtual audio. surround sound are what audio A third approach is the scene-based format. Ambisonics, devel- engineers call channel-based for- oped in the 1970s by Peter Fellgett, Michael Gerzon, and other mats: They encode the audio for researchers sponsored by the National Research Development a certain speaker configuration Corp., in the United Kingdom, takes this approach. Scene-based Hfor playback. Ordinary stereo recordings just have basic encoding creates a spatial representation of the recorded sound left/right information. They don’t contain detailed field as seen from a specific position. In its basic configuration, an directional information about the recorded sound. Sur- Ambisonics recording uses four microphone capsules arranged round sound does a bit better; the most well-known in a tetrahedral pattern (higher-resolution recordings use more). surround-sound format, 5.1 sound, allows the mixing Think of scene-based encoding as sorting the sound surrounding engineer to position sound relative to five different the listener into a number of preset directional bins, compared reference locations and adds a low-frequency channel with object-based formats, which don’t predetermine the bins but played back through a subwoofer. The format assumes instead let each sound object be positioned at any arbitrary spot. that speakers are positioned to the front left, front cen- Current Ambisonics technology has significant weaknesses, ter, front right, rear left, and rear right of the listener. particularly for use in real-world recording. Spatial resolution is This scheme gives us more information than plain stereo low, and the recording microphones tend to blur the directional-

DIRAC RESEARCH DIRAC does, but it still isn’t good enough for convincing VR. ity somewhat. But it is a convenient way to record. And because

SPECTRUM.IEEE.ORG | NORTH AMERICAN | FEB 2019 | 27 it is an open-source format that is readily available, it is being oration does not completely vanish, though the loca- used by players big and small. These include Facebook, which tion accuracy is much improved over a generic HRTF; incorporates it in the company’s 360-degree videos, and Google, it’s likely that the techniques used to measure an HRTF which uses it in its VR audio technology. are at least partly to blame. To date, two major commercial audio companies have released Researchers are currently testing different measur- encoding formats that support 3D audio incorporating some of ing ideas for creating personalized HRTFs. Some are these techniques. The MPEG-H 3D Audio System, developed by the using microphones placed in a listener’s ears, record- Fraunhofer Institute for Integrated Circuits, in Erlangen, G ermany, ing test signals played from various directions; this is supports object-based, channel-based, and scene-based audio, a time-consuming and error-prone method. Others as well as combinations thereof. Dolby AC-4 supports object- are trying to model individual ears using scanning based and channel-based audio. While all of these schemes have and computer-graphics analysis tools, like ray tracing, found some success, none of them has risen above the pack, and to code how they reflect sound from various angles. it is not clear if one of them will eventually dominate. And these While personalized HRTFs might go far to fix some approaches, to date, have been focused on encoding audio for of the current difficulties with VR sound, they have reproduction through speakers; moving to headphones presents other problems. For one thing, they assume a static bigger challenges. head and torso that move in tandem, while VR users All of this activity is good news for those of us trying to create true frequently turn their heads without moving their bod- virtual audio. However, though this work has established a good ies. But the biggest difficulty is simply the time and cost foundation, it’s unlikely that any one of the existing approaches of creating them, which means they aren’t going to be is going to develop into a robust 3D audio technology. A fresh practical for the average VR user. approach is needed.

HERE’S WHERE MY COMPANY’S RESEARCH RESEARCHERS ARE GETTING CLOSER. If we combine comes in. For the last 10 years, Dirac, of which I am CEO object- and scene-based encoding with HRTF processing, we the- and cofounder, has investigated various approaches oretically should be able to render 3D audio over headphones to improve HRTF processing. Our researchers found for head-mounted VR, and adjust it interactively as the listener that head movements, in particular, have a substantial moves through virtual worlds. impact on the HRTFs. But, to date, applying this technology has been a struggle. A key To understand why, imagine tilting your head toward shortcoming of HRTF playback is front/back confusion. Here’s the one of your shoulders. As your ear nears your shoulder, problem. The placement of a human’s ears means that a sound pro- the reflection of sound from the shoulder arrives at the duced dead center in front of or behind (or above or below) the ear more quickly, while the corresponding reflection listener has the same time of arrival and intensity at each ear. So at the other ear gets additional attenuation and delay. in order to determine the position of the sound, the brain exploits Acting on this observation, we have built a set of tiny changes caused by the shape of the ear, head, and torso. The what we are calling dynamic HRTFs. We based these on fine details in this anatomy, the shadows and reflections they create, measurements of 30 people. We oriented the listeners’ preferentially amplify certain frequencies in relation to others, heads in a variety of yaw, pitch, and roll positions in depending on the direction from which the sound comes. These relation to their torsos, with one degree of resolution details vary from person to person, mostly because of the distance in three dimensions, and tested sounds played from in between our ears but also because of other anatomical differences. front of the listeners, from both sides, from above and Many researchers believe that the only way to solve the front/ below, and from behind. We ended up making several back problem is to use individualized HRTFs, that is, personal- hundred measurements for each subject. (The actual ized acoustic body maps. More on that in a moment. number of HRTFs measured depended on each sub- Another challenge is that the sound processed by HRTFs often ject’s range of motion.) sounds unnatural. Certain pieces of the audio spectrum are inevi- To avoid having to individualize the model, and the tably amplified too much or too little. These inconsistencies can expense that would incur, we focused on the common be easily perceived by a trained listener. A casual listener might aspects of the HRTFs. If a certain ridge or valley in the not identify these colorations directly, but would hear something frequency response of an HRTF was common (within amiss, like the difference between a cheap audio speaker and an a tolerance limit) to all people tested, we made it part expensive one. of our generic model; if a characteristic was uncom- The reason for this second problem is unclear. Some argue mon, our algorithms made sure that this HRTF left no that the ways we measure HRTF are flawed, causing our existing sonic trace on the processed sound. HRTF databases to be inadequate. Others believe the coloration This approach won’t be perfect for every person. But is inevitable unless HRTFs are individualized. The reality is likely we have studied it and believe that there are a few strong a mix of both: Even when measuring an individual HRTF, the col- shadowing effects and a few strong reflections having to

28 | FEB 2019 | NORTH AMERICAN | SPECTRUM.IEEE.ORG

PHOTOGRAPH BY Firstname Lastname Moving Through a Virtual Audioscape In the real world, sounds come from all directions. person moves within the virtual world, while the heard most strongly by the left ear, red dots most To simulate that effect for virtual reality, the sound sources maintain their virtual positions. strongly by the right ear, and purple dots equally sounds heard by each ear must change as the In this illustration, blue dots indicate sounds by both ears.

do with the head orientation in relation to the body that, Then, on playback, we have to consider both the virtual room if modeled well enough, capture the essential informa- and the actual listener, passing not just each sound but each reflection needed by a human auditory system to determine tion of a sound wave—off the floor, the ceiling, and other objects— the direction of a sound. And, as long as the HRTFs cap- and through the appropriate HRTF. This procedure quickly ture a motion of the head or audio object in a consistent becomes hugely complex and computationally intensive. way, small discrepancies between a generic model and Right now, for interactive applications like games, developers an individual’s HRTF will be ignored by the listener’s simplify the acoustic information. They separate the sound into auditory system. As a result, the audio experience will a set of directional sound sources along with a combined ambient be realistic enough for most listeners. For those whose sound field, rather than simulating the acoustic characteristics of individual HRTF for a certain direction differs dramati- an entire scene. The directional sounds can then be processed by cally from the model, the sound will still seem reasonably HRTFs, while the ambient sound is assumed to come with equal natural, though not 100 percent directionally accurate. intensity from all directions. For most people, this technique pro- Our first commercial implementation of our dynamic duces reasonably convincing 3D sound in some virtual environ- HRTF technology, Dirac VR, will start appearing in prod- ments. Eventually, more-realistic acoustic simulations of virtual ucts by gaming headset manufacturers later this year. rooms will evolve, improving the authenticity of the audio experience in a wider range of challenging environments. I expect that within a few years researchers will create con- ixing the dynamic HRTF problem vincing 3D audio experiences for VR streams of, say, a basketball doesn’t get us to a truly realistic vir- game or a concert. Then the big challenge will be fine-tuning the tual sound experience, however. The HRTF algorithms to get the computational and memory require- HRTFs give us a way to mimic sounds ments down to the point where they can run on portable, battery- from any direction, but sounds are operated devices. Once this final barrier is overcome, immersive also affected by more than just the 3D audio for virtual reality will be ready for mass adoption. physical characteristics of the listener. In less than a decade, 3D audio over headsets with head-tracking AF person talking to you in an open field sounds very dif- capabilities will allow us to have remote meetings in which you can ferent from a person speaking in a room. Even within a move about an actual room, having sidebar discussions with one room, the location of walls and other objects strongly colleague or another as you huddle close or step away. We will be affect sound. able to have the experience of sitting courtside at the NBA finals. So, for virtual environments, we have to take into And we’ll be able to enjoy the music of Johann Strauss Jr. at the best account how the shape of the virtual room and the seat in the Vienna Musikverein. For me, that last possibility alone objects within it—or the road, cliff, or battlefield— is worth the investments required in solving the engineering chal- affect sounds. That involves modeling reflections and lenges that remain to fulfill the promise of 3D audio. n standing waves, the diffusive properties of walls, and

ISTOCKPHOTO the effects of interior objects when we create the sounds. ↗ POST YOUR COMMENTS at https://spectrum.ieee.org/3daudio0219

SPECTRUM.IEEE.ORG | NORTH AMERICAN | FEB 2019 | 29

PHOTOGRAPH BY Firstname Lastname @font-face { font-family: ‘Google Sans’; font-style: normal; font-weight: 400; src: local(‘Google Sans Regular’), local(‘GoogleSans-Regular’), url(//fonts.gstatic.com/s/googlesans/v9/4UaGrENHsxJlGDuGo1OIlL3Kwp5eKQtGBlc.woff2) format(‘woff2’); unicode-range: U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116; } @font-face { font-family: ‘Google Sans’; font-style: normal; font-weight: 400; src: local(‘Google Sans Regular’), local(‘GoogleSans-Regular’), url(//fonts.gstatic. com/s/googlesans/v9/4UaGrENHsxJlGDuGo1OIlL3Nwp5eKQtGBlc.woff2) format(‘woff2’); unicode-range: U+0370-03FF; } @font-face { font-family: ‘Google Sans’; font-style: normal; font-weight: 400; src: local(‘Google Sans Regular’), local(‘GoogleSans- Regular’), url(//fonts.gstatic.com/s/googlesans/v9/4UaGrENHsxJlGDuGo1OIlL3Awp5eKQtGBlc.woff2) format(‘woff2’); unicode- range: U+0100-024F, U+0259, U+1E00-1EFF, U+2020, U+20A0-20AB, U+20AD-20CF, U+2113, U+2C60-2C7F, U+A720-A7FF; } @font-face { font-family: ‘Google Sans’; font-style: normal; font-weight: 400; src: local(‘Google Sans Regular’), local(‘GoogleSans- Regular’), url(//fonts.gstatic.com/s/googlesans/v9/4UaGrENHsxJlGDuGo1OIlL3Owp5eKQtG.woff2) format(‘woff2’); unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, {{ The White-Hat Hacking Machine }} U +2122, U +2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD; } @font-face { font-family: ‘Google Sans’; font-style: normal; font-weight: 500; src: local(‘Google Sans Medium’), local(‘GoogleSans-Medium’), url(//fonts.gstatic.com/s/googlesans/v9/4UabrENHsxJlGDuG o1OIlLU94Yt3CwZsPF4oxIs.woff2) format(‘woff2’); unicode-range: U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116; } @font-face { fo nt-fa m i ly: ‘G o o gle S a n s’; fo nt- st yle: n o r m a l; fo nt- w e ig ht: 500; sr c: lo c a l(‘G o o gle S a n s M e d iu m’), lo c a l(‘G o o gle S a n s - Medium’), url(//fonts.gstatic.com/s/googlesans/v9/4UabrENHsxJlGDuGo1OIlLU94YtwCwZsPF4oxIs.woff2) format(‘woff2’); unicode- range: U+0370-03FF; } @font-face { font-family: ‘Google Sans’; font-style: normal; font-weight: 500; src: local(‘Google Sans Medium’), local(‘GoogleSans-Medium’), url(//fonts.gstatic.com/s/googlesans/v9/4UabrENHsxJlGDuGo1OIlLU94Yt9CwZsPF4 oxIs.woff2) format(‘woff2’); unicode-range: U+0100-024F, U+0259, U+1E00-1EFF, U+2020, U+20A0-20AB, U+20AD-20CF, U+2113, U +2C60 -2C7F, U +A720 -A7F F; } @fo nt-fa c e { fo nt-fa m i ly: ‘G o o gle S a n s’; fo nt- st yle: n o r m a l; fo nt- w e ig ht: 500; sr c: lo c a l(‘G o o gle Sans Medium’), local(‘GoogleSans-Medium’), url(//fonts.gstatic.com/s/googlesans/v9/4UabrENHsxJlGDuGo1OIlLU94YtzCwZsP F4o.woff2) format(‘woff2’); unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+2000- 206F, U+2074, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD; } @font-face { font-family: ‘Product Sans’; font-style: normal; font-weight: 400; src: local(‘Product Sans’), local(‘ProductSans-Regular’), url(//fonts.gstatic. com/s/productsans/v9/pxiDypQkot1TnFhsFMOfGShVGdeOYktMqlap.woff2) format(‘woff2’); unicode-range: U+0100-024F, U+0259, U+1E00-1EFF, U+2020, U+20A0-20AB, U+20AD-20CF, U+2113, U+2C60-2C7F, U+A720-A7FF; } @font-face { font-family: ‘Product Sans’; font-style: normal; font-weight: 400; src: local(‘Product Sans’), local(‘ProductSans-Regular’), format(‘woff2’); unicode-range: [[ How Mayhem won a DARPA challenge to find and patch software vulnerabilities ]] U +2074, U +20A C, U +2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD; } @font-face { font-family: ‘Roboto’; font-style: italic; font-weight: 400; src: local(‘Roboto Italic’), local(‘Roboto-Italic’), url(//fonts.gstatic.com/s/roboto/v18/KFOkCnqEu92Fr1Mu51xFIzIXK- M ny rY k.w o ff2) for m at(‘w o ff2’); u nico d e -ra nge: U+0460-052F, U+1C80-1C88, U+20B4, U+2DE0-2DF F, U+A640-A69F, U+F E2E-F E2F; }@ font-face { font-family: ‘Roboto’; font-style: italic; font-weight: 400; src: local(‘Roboto Italic’), local(‘Roboto- Italic’), url(//fonts.gstatic.com/s/roboto/v18/KFOkCnqEu92Fr1Mu51xMIzIXKMnyrYk.woff2) format(‘woff2’); unicode-range: U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116; } @font-face { font-family: ‘Roboto’; font-style: italic; font-weight: 400; src: local(‘Roboto Italic’), local(‘Roboto-Italic’), url(//fonts.gstatic.com/s/roboto/v18/KFOkCnqEu92Fr1Mu51xEIzIXK- M ny rY k.w o ff 2) fo r m at(‘w o ff 2’); u n ic o d e -ra n g e: U +1F00 -1F F F; } @fo nt-fa c e { fo nt-fa m i ly: ‘R o b o t o’; fo nt- st yle: it a l ic; font- weight: 400; src: local(‘Roboto Italic’), local(‘Roboto-Italic’), url(//fonts.gstatic.com/s/roboto/v18/ KFOkCnqEu92Fr1Mu51xLIzIXKMnyrYk.woff2) format(‘woff2’); unicode-range: U+0370-03FF; } @font-face { font-family: ‘Roboto’; font-style: italic; font-weight: 400; src: local(‘Roboto Italic’), local(‘Roboto-Italic’), url(//fonts.gstatic.com/s/ roboto/v18/KFOkCnqEu92Fr1Mu51xHIzIXKMnyrYk.woff2) format(‘woff2’); unicode-range: U+0102-0103, U+0110-0111, U+1EA0-1EF9, U+20AB; } @font-face { font-family: ‘Roboto’; font-style: italic; font-weight: 400; src: local(‘Roboto Italic’), local(‘Roboto-Italic’), url(//fonts.gstatic.com/s/roboto/v18/KFOkCnqEu92Fr1Mu51xGIzIXKMnyrYk.woff2) format(‘woff2’); unicode-range: U+0100-024F, U+0259, U+1E00-1EFF, U+2020, U+20A0-20AB, U+20AD-20CF, U+2113, U+2C60-2C7F, U+A720-A7FF; } @font- face { font-family: ‘Roboto’; font-style: italic; SPECTRUM.IEEE.ORG | North American | FEB 2 0 1 9 | 3 1 local(‘Roboto-Italic’), url(//fonts.gstatic.com/s/roboto/v18/KFOkCnqEu92Fr1Mu51xIIzIXKMny.woff2) format(‘woff2’); unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+2000-206F, U+2074, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD; } @font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 400; src: local(‘Roboto Regular’), // B y D avi d B r u m le y // url(//fonts.gstatic.com/s/roboto/v18/KFOmCnqEu92Fr1Mu72xKKTU1Kvnz.woff2) format(‘woff2’); unicode-range: U+0460-052F, U+1C80-1C88, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F; } @font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 400; src: local(‘Roboto Regular’), local(‘Roboto-Regular’), url(//fonts. gstatic.com/s/roboto/v18/KFOmCnqEu92Fr1Mu5mxKKTU1Kvnz.woff2) format(‘woff2’); unicode-range: U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116; } @font-face ++ Photo-illustration by Sean McCa be ++ normal; font-weight: 400; src: local(‘Roboto Regular’), local(‘Roboto-Regular’), url(//fonts.gstatic.com/s/roboto/v18/KFOmCnqEu92Fr1Mu7mxKKTU1Kvnz.woff2) format(‘woff2’); u n ic o d e -ra n g e: U +1F00 -1F F F; } @fo nt-fa c e { fo nt-fa m i ly: ‘R o b o t o’; fo nt- st yle: n o r m a l; fo nt- w e ig ht: 400; sr c: lo c a l(‘R o b o t o Regular’), local(‘Roboto-Regular’), url(//fonts.gstatic.com/s/roboto/v18/KFOmCnqEu92Fr1Mu4WxKKTU1Kvnz.woff2) format(‘woff2’); u n ic o d e -ra n g e: U +0370 - 03F F; } @fo nt-fa c e { fo nt-fa m i ly: ‘R o b o t o’; fo nt- st yle: n o r m a l; fo nt- w e ig ht: 400; sr c: lo c a l(‘R o b o t o Regular’), local(‘Roboto-Regular’), url(//fonts.gstatic.com/s/roboto/v18/KFOmCnqEu92Fr1Mu7WxKKTU1Kvnz.woff2) format(‘woff2’); unicode-range: U+0102-0103, U+0110-0111, U+1EA0-1EF9, U+20AB; } @font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 400; src: local(‘Roboto Regular’), local(‘Roboto-Regular’), url(//fonts.gstatic.com/s/roboto/v18/KFOmCnqEu- 92Fr1Mu7GxKKTU1Kvnz.woff2) format(‘woff2’); unicode-range: U+0100-024F, U+0259, U+1E00-1EFF, U+2020, U+20A0-20AB, U+20AD- 20CF, U+2113, U+2C60-2C7F, U+A720-A7FF; } @font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 400; src: local(‘Roboto Regular’), local(‘Roboto-Regular’), url(//fonts.gstatic.com/s/roboto/v18/KFOmCnqEu92Fr1Mu4mxKKTU1Kg.woff2) format(‘woff2’); unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+2000-206F, U+2074, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD; } @font-face { font-family: ‘Roboto’; font-style: normal; @font-face { font-family: ‘Google Sans’; font-style: normal; font-weight: 400; src: local(‘Google Sans Regular’), local(‘GoogleSans-Regular’), url(//fonts.gstatic.com/s/googlesans/v9/4UaGrENHsxJlGDuGo1OIlL3Kwp5eKQtGBlc.woff2) format(‘woff2’); unicode-range: U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116; } @font-face { font-family: ‘Google Sans’; font-style: normal; font-weight: 400; src: local(‘Google Sans Regular’), local(‘GoogleSans-Regular’), url(//fonts.gstatic. com/s/googlesans/v9/4UaGrENHsxJlGDuGo1OIlL3Nwp5eKQtGBlc.woff2) format(‘woff2’); unicode-range: U+0370-03FF; } @font-face { font-family: ‘Google Sans’; font-style: normal; font-weight: 400; src: local(‘Google Sans Regular’), local(‘GoogleSans- Regular’), url(//fonts.gstatic.com/s/googlesans/v9/4UaGrENHsxJlGDuGo1OIlL3Awp5eKQtGBlc.woff2) format(‘woff2’); unicode- range: U+0100-024F, U+0259, U+1E00-1EFF, U+2020, U+20A0-20AB, U+20AD-20CF, U+2113, U+2C60-2C7F, U+A720-A7FF; } @font-face { font-family: ‘Google Sans’; font-style: normal; font-weight: 400; src: local(‘Google Sans Regular’), local(‘GoogleSans- Regular’), url(//fonts.gstatic.com/s/googlesans/v9/4UaGrENHsxJlGDuGo1OIlL3Owp5eKQtG.woff2) format(‘woff2’); unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, {{ The White-Hat Hacking Machine }} U +2122, U +2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD; } @font-face { font-family: ‘Google Sans’; font-style: normal; font-weight: 500; src: local(‘Google Sans Medium’), local(‘GoogleSans-Medium’), url(//fonts.gstatic.com/s/googlesans/v9/4UabrENHsxJlGDuG o1OIlLU94Yt3CwZsPF4oxIs.woff2) format(‘woff2’); unicode-range: U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116; } @font-face { fo nt-fa m i ly: ‘G o o gle S a n s’; fo nt- st yle: n o r m a l; fo nt- w e ig ht: 500; sr c: lo c a l(‘G o o gle S a n s M e d iu m’), lo c a l(‘G o o gle S a n s - Medium’), url(//fonts.gstatic.com/s/googlesans/v9/4UabrENHsxJlGDuGo1OIlLU94YtwCwZsPF4oxIs.woff2) format(‘woff2’); unicode- range: U+0370-03FF; } @font-face { font-family: ‘Google Sans’; font-style: normal; font-weight: 500; src: local(‘Google Sans Medium’), local(‘GoogleSans-Medium’), url(//fonts.gstatic.com/s/googlesans/v9/4UabrENHsxJlGDuGo1OIlLU94Yt9CwZsPF4 oxIs.woff2) format(‘woff2’); unicode-range: U+0100-024F, U+0259, U+1E00-1EFF, U+2020, U+20A0-20AB, U+20AD-20CF, U+2113, U +2C60 -2C7F, U +A720 -A7F F; } @fo nt-fa c e { fo nt-fa m i ly: ‘G o o gle S a n s’; fo nt- st yle: n o r m a l; fo nt- w e ig ht: 500; sr c: lo c a l(‘G o o gle Sans Medium’), local(‘GoogleSans-Medium’), url(//fonts.gstatic.com/s/googlesans/v9/4UabrENHsxJlGDuGo1OIlLU94YtzCwZsP F4o.woff2) format(‘woff2’); unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+2000- 206F, U+2074, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD; } @font-face { font-family: ‘Product Sans’; font-style: normal; font-weight: 400; src: local(‘Product Sans’), local(‘ProductSans-Regular’), url(//fonts.gstatic. com/s/productsans/v9/pxiDypQkot1TnFhsFMOfGShVGdeOYktMqlap.woff2) format(‘woff2’); unicode-range: U+0100-024F, U+0259, U+1E00-1EFF, U+2020, U+20A0-20AB, U+20AD-20CF, U+2113, U+2C60-2C7F, U+A720-A7FF; } @font-face { font-family: ‘Product Sans’; font-style: normal; font-weight: 400; src: local(‘Product Sans’), local(‘ProductSans-Regular’), format(‘woff2’); unicode-range: [[ How Mayhem won a DARPA challenge to find and patch software vulnerabilities ]] U +2074, U +20A C, U +2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD; } @font-face { font-family: ‘Roboto’; font-style: italic; font-weight: 400; src: local(‘Roboto Italic’), local(‘Roboto-Italic’), url(//fonts.gstatic.com/s/roboto/v18/KFOkCnqEu92Fr1Mu51xFIzIXK- M ny rY k.w o ff2) for m at(‘w o ff2’); u nico d e -ra nge: U+0460-052F, U+1C80-1C88, U+20B4, U+2DE0-2DF F, U+A640-A69F, U+F E2E-F E2F; }@ font-face { font-family: ‘Roboto’; font-style: italic; font-weight: 400; src: local(‘Roboto Italic’), local(‘Roboto- Italic’), url(//fonts.gstatic.com/s/roboto/v18/KFOkCnqEu92Fr1Mu51xMIzIXKMnyrYk.woff2) format(‘woff2’); unicode-range: U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116; } @font-face { font-family: ‘Roboto’; font-style: italic; font-weight: 400; src: local(‘Roboto Italic’), local(‘Roboto-Italic’), url(//fonts.gstatic.com/s/roboto/v18/KFOkCnqEu92Fr1Mu51xEIzIXK- M ny rY k.w o ff 2) fo r m at(‘w o ff 2’); u n ic o d e -ra n g e: U +1F00 -1F F F; } @fo nt-fa c e { fo nt-fa m i ly: ‘R o b o t o’; fo nt- st yle: it a l ic; font- weight: 400; src: local(‘Roboto Italic’), local(‘Roboto-Italic’), url(//fonts.gstatic.com/s/roboto/v18/ KFOkCnqEu92Fr1Mu51xLIzIXKMnyrYk.woff2) format(‘woff2’); unicode-range: U+0370-03FF; } @font-face { font-family: ‘Roboto’; font-style: italic; font-weight: 400; src: local(‘Roboto Italic’), local(‘Roboto-Italic’), url(//fonts.gstatic.com/s/ roboto/v18/KFOkCnqEu92Fr1Mu51xHIzIXKMnyrYk.woff2) format(‘woff2’); unicode-range: U+0102-0103, U+0110-0111, U+1EA0-1EF9, U+20AB; } @font-face { font-family: ‘Roboto’; font-style: italic; font-weight: 400; src: local(‘Roboto Italic’), local(‘Roboto-Italic’), url(//fonts.gstatic.com/s/roboto/v18/KFOkCnqEu92Fr1Mu51xGIzIXKMnyrYk.woff2) format(‘woff2’); unicode-range: U+0100-024F, U+0259, U+1E00-1EFF, U+2020, U+20A0-20AB, U+20AD-20CF, U+2113, U+2C60-2C7F, U+A720-A7FF; } @font- face { font-family: ‘Roboto’; font-style: italic; SPECTRUM.IEEE.ORG | North American | FEB 2 0 1 9 | 3 1 local(‘Roboto-Italic’), url(//fonts.gstatic.com/s/roboto/v18/KFOkCnqEu92Fr1Mu51xIIzIXKMny.woff2) format(‘woff2’); unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+2000-206F, U+2074, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD; } @font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 400; src: local(‘Roboto Regular’), // B y D avi d B r u m le y // url(//fonts.gstatic.com/s/roboto/v18/KFOmCnqEu92Fr1Mu72xKKTU1Kvnz.woff2) format(‘woff2’); unicode-range: U+0460-052F, U+1C80-1C88, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F; } @font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 400; src: local(‘Roboto Regular’), local(‘Roboto-Regular’), url(//fonts. gstatic.com/s/roboto/v18/KFOmCnqEu92Fr1Mu5mxKKTU1Kvnz.woff2) format(‘woff2’); unicode-range: U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116; } @font-face ++ Photo-illustration by Sean McCa be ++ normal; font-weight: 400; src: local(‘Roboto Regular’), local(‘Roboto-Regular’), url(//fonts.gstatic.com/s/roboto/v18/KFOmCnqEu92Fr1Mu7mxKKTU1Kvnz.woff2) format(‘woff2’); u n ic o d e -ra n g e: U +1F00 -1F F F; } @fo nt-fa c e { fo nt-fa m i ly: ‘R o b o t o’; fo nt- st yle: n o r m a l; fo nt- w e ig ht: 400; sr c: lo c a l(‘R o b o t o Regular’), local(‘Roboto-Regular’), url(//fonts.gstatic.com/s/roboto/v18/KFOmCnqEu92Fr1Mu4WxKKTU1Kvnz.woff2) format(‘woff2’); u n ic o d e -ra n g e: U +0370 - 03F F; } @fo nt-fa c e { fo nt-fa m i ly: ‘R o b o t o’; fo nt- st yle: n o r m a l; fo nt- w e ig ht: 400; sr c: lo c a l(‘R o b o t o Regular’), local(‘Roboto-Regular’), url(//fonts.gstatic.com/s/roboto/v18/KFOmCnqEu92Fr1Mu7WxKKTU1Kvnz.woff2) format(‘woff2’); unicode-range: U+0102-0103, U+0110-0111, U+1EA0-1EF9, U+20AB; } @font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 400; src: local(‘Roboto Regular’), local(‘Roboto-Regular’), url(//fonts.gstatic.com/s/roboto/v18/KFOmCnqEu- 92Fr1Mu7GxKKTU1Kvnz.woff2) format(‘woff2’); unicode-range: U+0100-024F, U+0259, U+1E00-1EFF, U+2020, U+20A0-20AB, U+20AD- 20CF, U+2113, U+2C60-2C7F, U+A720-A7FF; } @font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 400; src: local(‘Roboto Regular’), local(‘Roboto-Regular’), url(//fonts.gstatic.com/s/roboto/v18/KFOmCnqEu92Fr1Mu4mxKKTU1Kg.woff2) format(‘woff2’); unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+2000-206F, U+2074, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD; } @font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 500; src: local(‘Roboto Medium’), local(‘Roboto-Medium’), url(//fonts.gstatic.com/s/roboto/v18/KFOlCnqEu92Fr1MmEU9fCRc4AMP6lbBP.woff2) format(‘woff2’); unicode-range: U+0460-052F, U+1C80-1C88, U+20B4, U+2DE0-2DFF, U+A640- A69F, U+FE2E-FE2F; } @font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 500; src: local(‘Roboto Medium’), local(‘Roboto-Medium’), url(//fonts.gstatic.com/s/roboto/v18/KFOlCnqEu92Fr1MmEU9fABc4AMP6lbBP.woff2) format(‘woff2’); unicode-range: U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116; } @font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 500; src: local(‘Roboto Medium’), local(‘Roboto-Medium’), url(//fonts.gstatic.com/s/roboto/v18/KFOl- CnqEu92Fr1MmEU9fCBc4AMP6lbBP.woff2) format(‘woff2’); unicode-range: U+1F00-1FFF; } @font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 500; src: local(‘Roboto Medium’), local(‘Roboto-Medium’), url(//fonts.gstatic.com/s/ roboto/v18/KFOlCnqEu92Fr1MmEU9fBxc4AMP6lbBP.woff2) format(‘woff2’); unicode-range: U+0370-03FF; } @font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 500; src: local(‘Roboto Medium’), local(‘Roboto-Medium’), url(//fonts. gstatic.com/s/roboto/v18/KFOlCnqEu92Fr1MmEU9fCxc4AMP6lbBP.woff2) format(‘woff2’); unicode-range: U+0102-0103, U+0110-0111, U+1EA0-1EF9, U+20AB; } @font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 500; src: local(‘Roboto

Back in 2011, when the venture capi- nearly 10 years building technology The cybersecurity battleground is talist Marc Andreessen said that “soft- that would make software safe, auto- populated by hackers who are techni- ware is eating the world,” it was still a matically. Then, in 2012, we founded cally skilled and, at the highest levels, fresh idea. Now it’s obvious that soft- ForAllSecure to bring our product to creative in exploiting weaknesses in ware permeates our lives. From com- the world. The one thing we needed software to penetrate an organization’s plex electronics like medical devices was a way to prove that we could do defenses. The criminals who do this for and autonomous vehicles to simple what we said we could do, and we got their own gain are commonly called objects like Internet-connected light- it in the form of a prize competition. black hats, and they often create tools bulbs and thermometers, we’re sur- that legions of amateur “script kiddies” {{{}}} rounded by software. can use to unleash havoc, like the IoT And that means we’re all more Fast-forward to 2016: My team botnets that in 2016 launched a massive exposed to attacks on that software is huddled in a hotel ballroom in attack on the Internet after gaining con- than ever before. Las Vegas, chewing our fingernails trol over minicams and video recorders Every year, 111 billion lines are added and fairly sure that we had just lost a in people’s homes. In contrast, “white to the mass of software code in exis- competition we’d spent thousands of hats” use their skills to thwart such tence, and every line presents a poten- hours preparing for. It was the DARPA attacks. But there simply aren’t enough tial new target. Steve Morgan, founder Cyber Grand Challenge (CGC), one of white-hat hackers to protect all the soft- and editor in chief at the research firm several such events—like the one for ware proliferating in the commercial Cybersecurity Ventures, predicts that self-driving vehicles back in the early world, let alone the common infrastruc- system break-ins made through a pre- 2000s—staged by the U.S. Defense ture and the military platforms vital to viously unknown weakness—what the Advanced Research Projects Agency to national and global security. industry calls “zero-day exploits”—will stimulate technological breakthroughs In 2014, DARPA announced the Cyber average one per day in the United States for national security. The CGC grew Grand Challenge as a two-year project by 2021, up from one per week in 2015. out of DARPA’s recognition that the with the goal of testing whether it was It was to solve this problem that my United States might one day find itself possible to develop AI systems that could colleagues and I at Carnegie Mellon without the manpower or the tools to find, verify, and patch software weak- University (CMU), in Pittsburgh, spent fend off cyberthreats. nesses. In 2015, some 100 teams entered the prequalification stage. In 2016, the top seven advanced to the grand champi- onship finale, where they’d need to enter THE MAGNIFICENT SEVEN: Competing computers a full c yber-reasoning system—one that glow as they glower at the audience attending the finals of the Cyber Grand would not merely notice a problem but Challenge, held in Las Vegas in 2016. could also infer its nature. The cham- pion would win US $2 million, and the second- and third-place finishers would get $1 million and $750,000, respectively. After DARPA released details about its competition, it dawned on my colleagues and me that this was a great opportunity to demonstrate that the automated cybersecurity we’d developed was no mere theoretical game. After spinning out ForAllSecure, we’d consistently faced skepticism about how practical our solution could be. We figured that we’d better win the DARPA competition, given that we’d been working on this for a decade. Our research at CMU had begun with a simple premise: People need a way

to check the software they’re buying DARPA

32 | FEB 2019 | NORTH AMERICAN | SPECTRUM.IEEE.ORG font-weight: 500; src: local(‘Roboto Medium’), local(‘Roboto-Medium’), url(//fonts.gstatic.com/s/roboto/v18/KFOlCnqEu92Fr1MmEU9fCRc4AMP6lbBP.woff2) format(‘woff2’); unicode-range: U+0460-052F, U+1C80-1C88, U+20B4, U+2DE0-2DFF, U+A640- A69F, U+FE2E-FE2F; } @font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 500; src: local(‘Roboto Medium’), local(‘Roboto-Medium’), url(//fonts.gstatic.com/s/roboto/v18/KFOlCnqEu92Fr1MmEU9fABc4AMP6lbBP.woff2) format(‘woff2’); unicode-range: U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116; } @font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 500; src: local(‘Roboto Medium’), local(‘Roboto-Medium’), url(//fonts.gstatic.com/s/roboto/v18/KFOl- CnqEu92Fr1MmEU9fCBc4AMP6lbBP.woff2) format(‘woff2’); unicode-range: U+1F00-1FFF; } @font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 500; src: local(‘Roboto Medium’), local(‘Roboto-Medium’), url(//fonts.gstatic.com/s/ roboto/v18/KFOlCnqEu92Fr1MmEU9fBxc4AMP6lbBP.woff2) format(‘woff2’); unicode-range: U+0370-03FF; } @font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 500; src: local(‘Roboto Medium’), local(‘Roboto-Medium’), url(//fonts. gstatic.com/s/roboto/v18/KFOlCnqEu92Fr1MmEU9fCxc4AMP6lbBP.woff2) format(‘woff2’); unicode-range: U+0102-0103, U+0110-0111, U+1EA0-1EF9, U+20AB; } @font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 500; src: local(‘Roboto

TOWER OF POWER: Like its six rivals in the DARPA competition, Mayhem required water cooling. However, power and temperature statistics showed that Mayhem consistently worked the hardest of them all.

To build Mayhem, we began with the first software-analysis system we developed at CMU, which is based on the formal analysis of a program. This method can be likened to creating a mathematical formula that represents every path a software program might take, thus producing an ever-branching tree of analysis. Such a tree can quickly get too big to manage, but we have found smart ways to collapse some of the paths, pruning the tree down to just a few branches. We are then able to explore the remaining branches more deeply. Symbolic execution builds an equation to represent all the logic in a program—for example, “x + 5 = 7”—and then solve the equation. Contrast this strategy with another method of software analysis known as fuzzing, in which you and ensure that it’s safe. Coders will, of the severity of the bugs they find. But feed random permutations of data into course, make a due-diligence effort to only the most profitable companies can a program to crash it, after which you flush out security flaws, but their main afford the strongest testing of their soft- can determine the vulnerabilities that concerns are always more basic: They ware. The issue grows more complex as were at fault and how they might be have to ship their product on time and finished software includes ever more exploited in a more deliberate attack. ensure that it does what it’s supposed components from open-source proj- Fuzzing keeps putting in random data to do. The problem is that hackers will ects and other third parties. until a particular string of data makes find ways to make the software do things The system we entered in the com- the equation true, finally determining it’s not supposed to do. petition, Mayhem, automated what that x =2. Today’s state of the art for software white-hat hackers do. It not only Both approaches have their strengths, security involves using special tools pointed to possible weaknesses, it but for many years fuzzing had the to review the source code and to flag exploited them, thus proving con- advantage because it was easier to potential security weaknesses. Because clusively that they were in fact weak- implement and much faster at trying that process produces a lot of false nesses. This was also a key part of new inputs. Symbolic execution, mean- positives—flagging things that in fact are the CGC, as demonstrating a proof of while, held out a vast, untapped poten- not weaknesses—a human being must vulnerability with a working exploit tial to whoever could learn to tame it. In then go through and check every case. was part of how your machine scored the Mayhem system we started building To improve the bug-finding rate, some points. And because Mayhem was a in 2010, we were able to accomplish this companies rely on white-hat hackers machine that could be scaled up across feat by combining the two approaches. to do a one-time analysis or to partici- hundreds or thousands of nodes, the Fuzzing is like making intelligent pate in “bug bounty” programs, which analysis could proceed at a speed no guesses at lightning speed about which

THIS PAGE AND PREVIOUS PAGE, ORIGINAL PHOTO: CHELSEA MASTILAK CHELSEA PHOTO: ORIGINAL PAGE, PREVIOUS AND PAGE THIS pay them according to the number and human could match. inputs might trigger the program to

SPECTRUM.IEEE.ORG | NORTH AMERICAN | FEB 2019 | 33 Medium’), local(‘Roboto-Medium’), url(//fonts.gstatic.com/s/roboto/v18/KFOlCnqEu92Fr1MmEU9fChc4AMP6lbBP.woff2) format(‘woff2’); unicode-range: U+0100-024F, U+0259, U+1E00-1EFF, U+2020, U+20A0-20AB, U+20AD-20CF, U+2113, U+2C60-2C7F, U+A720- A7FF; } @font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 500; src: local(‘Roboto Medium’), local(‘Roboto-Medium’), url(//fonts.gstatic.com/s/roboto/v18/KFOlCnqEu92Fr1MmEU9fBBc4AMP6lQ.woff2) format(‘woff2’); unicode- range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+2000-206F, U+2074, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD; } @font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 700; src: local(‘Roboto Bold’), local(‘Roboto-Bold’), url(//fonts.gstatic.com/s/roboto/v18/KFOlCnqEu92Fr1MmWUlfCRc4AMP6lbBP.woff2) format(‘woff2’); unicode-range: U+0460-052F, U+1C80-1C88, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F; } @ font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 700; src: local(‘Roboto Bold’), local(‘Roboto-Bold’), url(//fonts.gstatic.com/s/roboto/v18/KFOlCnqEu92Fr1MmWUlfABc4AMP6lbBP.woff2) format(‘woff2’); unicode-range: U+0400- 045F, U+0490-0491, U+04B0-04B1, U+2116; } @font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 700; src: local(‘Roboto Bold’), local(‘Roboto-Bold’), url(//fonts.gstatic.com/s/roboto/v18/KFOlCnqEu92Fr1MmWUlfCBc4AMP6lbBP.

engage in some new behavior, then code, it has no functions, no local vari- down to 250 that were new and there- keeping track of those inputs that actu- ables, and no data abstractions. Binary fore deserved the highest priority. The ally do so. Symbolic execution is like ask- code has one big memory region and entire test was done in less than a week ing a mathematician to try to formally fixed-length bit vectors—a data structure by scaling Mayhem across a large num- figure out what inputs may exploit the that stores bits efficiently. You’d have to ber of servers in the Amazon cloud, with program. We found that some bugs are be a machine to work with such code, practically no human intervention. We best found by rapid guessing, others and indeed it required significant engi- submitted the more important findings by the mathematical approach. So we neering to build a machine that could to the online Debian community. One of decided to run both methods in paral- work under these constraints. the reasons we’ve spun off our research lel. Symbolic execution would reason After Mayhem identifies a vulnerabil- into a company is to be able to work at about one part of the program deeply, ity, it generates a working exploit—that is, this scale with developers as we analyze coming up with an input to trigger that code of the sort a black-hat hacker might thousands of programs with enormous region of code. The system could then use to break into a program. The point numbers of vulnerabilities. hand off that input to the fuzzing pro- is to demonstrate that the exploit can be On 3 June 2015, the 100-plus competi- gram, to rapidly hammer on that same used to obtain privileged, or root, access tors entered the qualifying round and region and shake out a vulnerability. to the operating system. The result is were given 131 unique purpose-built Another feature of Mayhem is that it that Mayhem identifies vulnerabilities challenges, each one containing soft- can work directly on binary code, as with absolute certainty, rather than ware security vulnerabilities. The seven opposed to human-coded text files— merely flagging possible problems, as teams with the highest security score that is, source code. That means the most code-analysis tools do. (based on discovering vulnerabilities system can analyze a program without In 2014 we ran a test of the Mayhem and patching them) made it into the the help of the person who developed it, technology on every program in the Cyber Grand Challenge final event—and which matters greatly for programs that Debian distribution, a popular version ForAllSecure scored more than twice incorporate third-party components for of Linux that’s used on desktops and as high as the next-best semifinalists. A which the source code may no longer servers throughout the world. Mayhem temporary moment of joy was quickly even exist. But reasoning about binary found nearly 14,000 unique vulnera- succeeded by the realization that the code is tough because, unlike source bilities, and then it narrowed that list pressure was really on now! Taking the core Mayhem technology and building a fully autonomous cyber-reasoning system was a massive THE MAYHEM TEAM: Engineers from ForAllSecure pose with undertaking. We were able to do it in their creation, Mayhem, at the closing ceremony. Author part because DARPA gave all seven final- David Brumley is in the front row, third from the left. ists enough funding for a year of development work. Our core components included a tool set that translates exe- cutable programs into a language that’s relatively easy to understand and analyze, as well as offensive tools for finding and exploiting the vulnerabilities, defensive tools for automatically patching the defective binary code, and a program to coordinate the work efficiently. In preparing for the final round, we faced two big challenges. First, although we were happy with how well Mayhem found vulnerabilities, we didn’t think the patches were efficient enough. In the competition, as in real life, you don’t want to install a patch that adds more processing power than solving that one

problem is worth. We therefore spent a DARPA

34 | FEB 2019 | NORTH AMERICAN | SPECTRUM.IEEE.ORG Medium’), local(‘Roboto-Medium’), url(//fonts.gstatic.com/s/roboto/v18/KFOlCnqEu92Fr1MmEU9fChc4AMP6lbBP.woff2) format(‘woff2’); unicode-range: U+0100-024F, U+0259, U+1E00-1EFF, U+2020, U+20A0-20AB, U+20AD-20CF, U+2113, U+2C60-2C7F, U+A720- A7FF; } @font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 500; src: local(‘Roboto Medium’), local(‘Roboto-Medium’), url(//fonts.gstatic.com/s/roboto/v18/KFOlCnqEu92Fr1MmEU9fBBc4AMP6lQ.woff2) format(‘woff2’); unicode- range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+2000-206F, U+2074, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD; } @font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 700; src: local(‘Roboto Bold’), local(‘Roboto-Bold’), url(//fonts.gstatic.com/s/roboto/v18/KFOlCnqEu92Fr1MmWUlfCRc4AMP6lbBP.woff2) format(‘woff2’); unicode-range: U+0460-052F, U+1C80-1C88, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F; } @ font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 700; src: local(‘Roboto Bold’), local(‘Roboto-Bold’), url(//fonts.gstatic.com/s/roboto/v18/KFOlCnqEu92Fr1MmWUlfABc4AMP6lbBP.woff2) format(‘woff2’); unicode-range: U+0400- 045F, U+0490-0491, U+04B0-04B1, U+2116; } @font-face { font-family: ‘Roboto’; font-style: normal; font-weight: 700; src: local(‘Roboto Bold’), local(‘Roboto-Bold’), url(//fonts.gstatic.com/s/roboto/v18/KFOlCnqEu92Fr1MmWUlfCBc4AMP6lbBP.

A COMFORTABLE WIN: Mayhem managed to build up a huge margin before suffering from a crash after the 40th round. That margin went unreported during the competition, leaving the team members in the dark until the very end.

fixed security flaws in seconds, compared with the months or years a human team would have taken. The audience numbered over 5,000, and the guest commentators—an astrophysicist and star hackers—got them riled up. We braced ourselves to see our defeat good deal of time building automated During nearly 100 rounds of compe- announced and confirmed onscreen. patching for vulnerabilities that had tition, new programs were given to the However, as we watched the scores between 0 to 5 percent overhead—in competing systems, each of which had come in with each new round, it the common case. mere minutes to analyze the code for occurred to us that Mayhem’s lead was Second, we needed a strategy for play- vulnerabilities and quickly issue patches great enough to keep it in first place, ing and winning the game. Let’s say you to protect itself. Each round was scored even though it had stopped playing find a vulnerability and make a patch for based on the machine’s ability to find after round 40. As the final rounds were it. You may not want to field the patch and prove vulnerabilities and on the announced, the weight was lifted from right away if that would mean adding so performance of the patches. our shoulders. We had won. much overhead that you slow the pro- To make the final CGC event more Mike Walker, the DARPA program gram down to a crawl. Instead, sometimes exciting for spectators, the competition director, said that the event’s demon- it’s better to wait, and patch only when organizers had decided to report the stration of autonomous cyberdefense absolutely necessary. We developed an scores only at the very end, in a play- was “just the beginning of a revolution” expert system to decide when to patch. by-play summary. That meant we didn’t in software security. He compared When our team walked into the Las really know if we were winning or los- the results to the initial flights of the Vegas ballroom for the final competi- ing, just that Mayhem was making sub- Wright brothers, which didn’t go very tion on 5 August 2016, we saw seven missions of vulnerabilities it had found. far but pointed the way to transconti- hulking racks with blinking lights sit- However, several hours into the compe- nental routes. ting atop a huge stage, below which tition, after round 40, we could tell that Right now, ForAllSecure is selling the were 180 tons of water to keep each Mayhem had simply stopped submitting. first versions of its new service to early team’s computers cool. Participants The program had crashed. adopters, including the U.S. govern- had set up the machines the previous ment and companies in the high-tech {{{}}} night, before the competition began, and aerospace industries. At this stage, and then DARPA had cut off all access to Our stomachs lurched as our worst the service mostly indicates problems them. The machines were air-gapped— nightmare seemingly came true. We that human experts then go in and they had no connections to the outside asked the organizers for a reboot, but fix. For a good while to come, systems world. All we could do was watch May- they wouldn’t allow it. With half the like Mayhem will work together with hem toiling away, observing the power competition still remaining, we began human security experts to make the usage and system temperature stats to contemplate the humiliation of defeat. world’s software safer. In the more dis- reported by each system’s rack. May- The play-by-play commentary started tant future, we believe that machine hem was consistently working the hard- as the final round wrapped up, with intelligence will handle the job alone. n est of the seven competitors—a good fancy visualizations illustrating how ↗ POST YOUR COMMENTS at https://spectrum.ieee.org/

DARPA sign, or so we hoped. each team’s machine had found and mayhem0219

SPECTRUM.IEEE.ORG | NORTH AMERICAN | FEB 2019 | 35 The Stethoscope Gets Smart

Engineers from Johns Hopkins are giving the humble stethoscope an AI upgrade

By Mounya Elhilali & James E. West

YOU WAKE UP ONE MORNING mend that you travel to a hospital in the nearest to discover that your child is ill: city? Or will she dismiss the symptoms as signs His forehead feels hot to the touch, of a common cold, and send you back home with and his rapid breathing has a your son? If she mistakenly sends you home, will wheezing sound. You live in your son survive the mistake? Malawi, where your health care options are few. Worldwide, more kids die of pneumonia and When the local clinic opens, you wait for your other lung ailments than from any other cause. turn with the solitary clinic worker. She’s not a These acute lower respiratory infections kill nearly doctor, but she’s been trained to identify and 1 million children each year worldwide, causing handle routine problems. more deaths than HIV and malaria combined. But She puts on a stethoscope and presses its chest fewer than 5 percent of people in the developing piece against your son’s front and back to carefully world have access to the X-ray imaging that’s con- listen to his lungs. Through the windows, open sidered the gold standard for pneumonia diagno- in the heat of the day, come the sounds of people ses. What’s more, many sick children never get to talking, the thrum of a generator, and the roar a clinic because they live far away from a facility, of a moped on the main road. The health worker roads are in poor condition, and transportation strains to hear. Will she detect the signs of bacte- is expensive. In sub-Saharan Africa, where most rial pneumonia, give you antibiotics, and recom- pneumonia deaths occur, only two in five children DR. ERIC D. MCCOLLUM D. ERIC DR.

36 | FEB 2019 | NORTH AMERICAN | SPECTRUM.IEEE.ORG AI ASSISTANCE: A health worker in Bangladesh listens to sounds from a boy’s lungs with the help of the Johns Hopkins smart stethoscope, which cancels out extraneous noise and identifies telltale signs of pneumonia.

SPECTRUM.IEEE.ORG | NORTH AMERICAN | FEB 2019 | 37 Pneumonia with pneumonia symptoms receive pro- GLOBAL KILLER: fessional medical care. In 2015, about 5.9 million children Other Given the global burden of these respi- died before the age 15.5% HIV 1.4% ratory ailments, the World Health Orga- of 5. Pneumonia 24.9% nization has developed strategies for accounted for Malaria 15.5 percent of those countries with limited resources. The deaths around the 5.2% guidelines for pneumonia diagnosis world, and was the minimize dependence on any tech- top cause of death in 8.9% sub-Saharan Africa. 6.1% Diarrhea nological tools, and instead rely solely Injury on observed symptoms of shortness 8.6% of breath, cough, and rapid breathing. In hopes of saving lives, the WHO rec- Congenital abnormalities 29.4% ommends antibiotic treatment for all children with these symptoms, with Preterm and birth the result that half the children who get complications treated for pneumonia don’t really have SOURCE: THE LANCET, VOLUME 388, ISSUE 10063, DECEMBER 2016 it. This approach puts unnecessary costs on communities, and the unnecessary medication contributes to the growing thereby take advantage of its low cost UR MISSION SPRANG problem of antibiotic-resistant bacteria. and ease of use. With funding from the from an ambitious global We propose a technological solution National Heart, Lung, and Blood Insti- study of pneumonia that that builds on the stethoscope, which tute, NASA, and the Bill & Melinda Gates began in 2008, with has hardly changed since its invention in Foundation, we set out to completely funding from the Gates the early 1800s. This ubiquitous clinical reengineer the stethoscope. OFoundation. The Pneumonia Etiology tool has a number of limitations. For best The project was a collaboration among Research for Child Health (PERCH) results, the user should be in a quiet envi- engineers, doctors, and public health study explored the causes of pediatric ronment, as background noise can easily experts at Johns Hopkins University, pneumonia across multiple countries in mask the subtle sounds coming from the in Baltimore. Together, we invented a Africa, Asia, and South America. lungs. The diagnostician must be well device that uses digital sensing tech- In the course of this study, a team of trained in positioning the chest piece nology for sound capture, active acous- doctors from Johns Hopkins set out to properly on the body and in interpreting tics for noise cancellation, and artificial determine just how useful lung sounds the sounds, which is something of an art intelligence (AI) to help health workers could be in diagnosing pneumonia. At form. Learning to do it well requires time make accurate pneumonia diagnoses. first, the researchers equipped local and mentorship, and even experts can We hope that our smart stethoscope will health workers with commercially disagree on what they hear. That’s why be deployed around the world to prevent available stethoscopes that they had even leading doctors prefer to comple- children from dying of pneumonia. And augmented with small digital record- ment their use of the stethoscope with as we test its capabilities, we may find ers. The health workers recorded lung other techniques, such as chest X-rays. that it’s a boon not only for the devel- sounds from the children they encoun- We wondered whether we could over- oping world but also for hospitals in the tered in rural clinics and urban hospitals, come the stethoscope’s limitations and United States and other well-off countries. then transmitted the files to expert pulmonologists at Johns Hopkins, who lis- tened to these sounds and offered more

SOUNDS OF TROUBLE: These two spectrograms show digitized audio signals from a informed medical opinions. child who’s breathing normally [left] and from a child who’s producing the high-pitched Quickly, the use of the commercial wheezing noises [right] that indicate fluid and inflammation in the lungs. stethoscope became problematic. To

Normal breathing Breathing with wheezing Hertz Hertz

800 800 20 600 600 400 400 200 200 100 0 1 2 0 1 2

Seconds Seconds UNIVERSITY HOPKINS JOHNS

38 | FEB 2019 | NORTH AMERICAN | SPECTRUM.IEEE.ORG get a decent recording from the lungs, the health worker had to position the device’s chest piece in just the right places on the child’s chest and back. Unfortunately, the local workers typically had only basic training in how to use the tool, and often missed the “sweet spots” that best transfer the lung sounds to the stethoscope head. Furthermore, they were usually making their recordings in noisy settings, nothing like the quiet doctor’s office for which the stethoscope is intended. Background sounds of people, machines, cars, and motor- cycles would mix with the lung sounds, leaving the expert with a recording of cacophony. In addition, the local workers needed Internet access to transmit their files, and connectivity limitations and glitches meant that it often took many hours or even days to get back an expert diagnosis. If they wanted to get useful recordings, the Johns Hopkins doctors decided, they’d have to entirely rethink the stethoscope. So they approached the university’s department of electrical and computer engineering, where one of us (West) is a professor, and the other (Elhi- lali) is an associate professor. As both of us specialize in audio engineering, we were eager to take up the challenge.

ERE’S HOW A CLASSIC stethoscope works. Sound is produced by an WORKING PROTOTYPES: After an early version of the smart stethoscope [top] proved organ in the body, and effective at filtering out noise and giving the listener a clear signal from the lungs, Johns Hopkins researchers developed the current version [bottom], which includes an LED dis- these acoustic waves play that gives the user a recommendation on the diagnosis. Hcause a vibration in the stethoscope’s chest piece, which acts as a resonator. This chest piece has two sides with carefully chosen shapes—a flat, disk- the user’s ears, sound levels are usu- phragm” and “bell” modes, and can like diaphragm and a hollow cup called ally quite low. often adjust the sounds that reach their a bell—that naturally oscillate with dif- More-modern electronic stetho- ears in other ways. ferent frequency ranges, and which are scopes convert the acoustic wave into The Johns Hopkins device is an elec- used for different diagnostic tasks. The electric signals that can be processed in tronic stethoscope that improves on digi- acoustic vibration travels via an air- the device to amplify the sounds. They tal devices currently on the market. The filled tube that connects to two ear- replace the double-sided chest piece upgrades start with its hardware: The pieces, thus relaying the sounds of the with one gadget that uses transducers chest piece is packed with transducer patient’s body to the listener. Although to convert acoustic signals to electric arrays to achieve a uniform sensitivity the stethoscope is designed to max- ones with minimal distortion or noise over the entire active area. This design

TOP: JOHNS HOPKINS UNIVERSITY; BOTTOM: ZEBADIAH POTLER/JOHNS HOPKINS UNIVERSITY HOPKINS POTLER/JOHNS ZEBADIAH BOTTOM: UNIVERSITY; HOPKINS JOHNS TOP: imize sound pickup and delivery to effects. Users can toggle between “dia- delivers a strong signal even when the

SPECTRUM.IEEE.ORG | NORTH AMERICAN | FEB 2019 | 39 chest piece isn’t placed in precisely SEEKING CLARITY: Traditional stethoscopes are designed for quiet doctors’ offices, the right location, which gives a major but many health clinics around the world are far from quiet. The devices are also meant to be used by experts trained in the subtle art of interpreting body sounds. The assist to untrained users. We’ve experi- smart stethoscope provides a clear audio signal by canceling out both noise from the mented with a variety of transducers, environment and irrelevant noise from inside the body. including microelectromechanical sys- Ambient noise tems (MEMS), which allow us to pack a great many microphones into a small 2 area, as well as nanofiber materials that match the acoustic properties of skin 1

to limit signal loss. Kilohertz 0 For a stethoscope to be useful in clin- 0 1 2 3 4 5 ics in the developing world, dealing with Seconds noise is of utmost importance. In clas- [1] The smart scope has an external microphone that records ambient sounds. The sic stethoscopes, environmental noise system uses active and adaptive acoustics to remove those sounds, in real time, from contaminates the signal at three points: the digitized audio signal transmitted to the user’s ears. at the chest piece, through the rubber hose, and at the ear of the user. The Noisy body sounds Johns Hopkins design mitigates noise 2 by improving the coupling between

the patient’s body and the chest piece, 1 swapping the rubber hose for an elec- Kilohertz tric cable, and employing digital noise- 0 control techniques to ensure that a 0 1 2 3 4 5 strong signal arrives at the user’s ears. Seconds Our stethoscope also has an external [2] Pulmonologists are trained to ignore other body sounds, such as the heartbeat, microphone in the chest piece to collect when they listen to the lungs. The smart scope detects those extraneous sounds and removes them, too. ambient sounds. This data can then be used to reduce noise through adaptive Enhanced signal signal analysis—as we’ll explain in the next section. 2 Our stethoscope gets its smarts from an onboard microprocessor that serves 1

as an operating system, allowing us to Kilohertz develop various applications to tailor 0 the device for different needs. So far, 0 1 2 3 4 5 Seconds our team has focused on two apps that we deemed crucial for developing world [3] What’s left is a clear audio signal containing only the sounds from the patient’s lungs. With this refined signal, both the listener and the stethoscope’s software can identify clinics. One dials down the level of noise; the signs of pneumonia. the other dials up the level of expertise.

EDUCING EXTERNAL a child is wheezing, producing high- As we designed our noise-cancellation noise is a complicated pitched whistling sounds in addition algorithm, we considered two challenges. matter. It’s relatively easy to the normal sounds of breathing, the First, the lung sounds that indicate pos- to design noise-canceling “de-noising” algorithm should not mis- sible pathologies are unpredictable and software that recognizes take this whistling for high-pitched irregular. Second, the noises in a busy Rthe signal pattern created by nor- ambient noise, such as a crying child clinic are highly variable: The exam room mal breathing and filters out every- would make. Conversely, trying to pre- could be filled with chattering people, thing else. But pathological conditions serve abnormal patterns can also pre- ringing phones, and whirring fans; the like pneumonia create abnormal serve ambient noise. Our app aims to child being examined might be crying signal patterns—and an overactive suppress undesired noise while main- out and thrashing around on the exam noise-canceling filter can remove that taining the integrity of the signal com- table; and the movements of the stetho-

critical information. For example, if ing from the patient’s lungs. scope itself can produce electronic noise. UNIVERSITY HOPKINS JOHNS

40 | FEB 2019 | NORTH AMERICAN | SPECTRUM.IEEE.ORG With those challenges in mind, our Our AI approach was inspired by the gists and gastroenterologists to collect team created a noise-cancellation algo- brain’s auditory system, which ana- training data and design algorithms. rithm that uses active and adaptive lyzes both the overall audio signal and Today’s electronic stethoscopes typ- acoustics. Instead of just setting up a fil- its specific features. The brain devotes ically cost around US $500, making ter and letting it run in a passive way, our the most processing power to the fre- them far too expensive for many health system analyzes both lung sounds and quencies that change over time and workers in the developing world. The ambient sounds recorded by the exter- those that it judges most relevant to Johns Hopkins smart scope is designed nal microphone, looking at their fre- the situation. Think about being at a to be significantly cheaper, with afford- quencies on the audio spectrum. Then noisy cocktail party, and how you’re able electronic components and low- the app adapts the degree and spectral able to ignore the background buzz of cost power and computing options, span of noise cancellation as the algo- chatter to focus on the words spoken in hopes that it can be useful for low- rithm tracks the two signals over time. by the person right in front of you. As resource communities. We validated this approach through a our app analyzes the de-noised lung Right now, our smart scope is under- blind listening test, in which we asked sound, it takes in the whole signal, but going extensive field testing. Clinic work- a panel of pediatric pulmonologists to zeroes in on the specific modulations ers in Peru, Bangladesh, and Malawi are compare the quality of sounds coming that indicate a case of pneumonia. trying out prototypes, as are doctors at from a commercial stethoscope used in We used a machine-learning approach the Johns Hopkins pediatric emergency a rural clinic with the same signals run to train the app, giving it plenty of data room. These testing and validation stud- through the Johns Hopkins algorithm. and a mandate to pick out the most ies are assessing how well it functions in The experts preferred our de-noised useful features. The algorithm trained noisy settings and how accurate its pneu- signal 95 percent of the time. on a database of recordings of about monia diagnoses are across a diverse We’ve also extended the audio soft- 1,500 patients from five African coun- population of patients. In these stud- ware to deal with extraneous sounds tries and two Asian countries. When ies, we’re evaluating the lung sounds from the body. When a health worker we tested the AI-enabled app, we found using objective signal quality metrics, is trying to listen to a patient’s lungs, that it could automatically distinguish and we’re validating the diagnoses by the sound of a beating heart can be dis- healthy people from those with pneu- comparing them with diagnoses made tracting and can mask the lung sounds. monia with an accuracy of 87 percent, via chest X-rays and the listening assess- Pulmonologists train their ears to far surpassing other automatic diag- ments of pulmonology experts. ignore the heartbeat and focus on the nosis methods. Our team is continuing While we continue our work in aca- breathing patterns—and we gave our to work on this algorithm, aiming to demia, some of our colleagues have algorithm similar training. Our app improve its performance with machine- founded a startup called Sonavi Labs to cancels out the sound of the heartbeat learning techniques. quickly turn our prototype into a com- to provide the user with the pure and Health workers in rural clinics could mercially viable product. The mission unadulterated signal from the lungs. use our smart stethoscope even if they of Sonavi Labs is to develop modern don’t have access to the Internet; the medical devices that effectively use onboard technology does all the pro- lung sounds for the diagnosis of dis- O SAVE THE LIVES OF cessing and provides an instant recom- ease. Their first products, digital stetho- children throughout the mendation on diagnosis using a small scopes called the Feelix and FeelixPro, developing world, we wish built-in LED display. We envision that are set to launch in early 2019. we could send expert physi- the final version of the smart scope By enabling on-the-spot screenings cians to staff every remote will connect via Bluetooth to the user’s with a cheap device that can be handled Tclinic and overcrowded hospital. So we phone or tablet, where the user can get by local health workers, we hope our did the next-best thing and developed more information about the diagnosis, reengineered scope will make a differ- an app to send physician expertise to visualize the lung-sound signal, and ence in the global health crisis of child- these settings. play back previously recorded sounds. hood pneumonia. Viewed on the macro This second app, the one that makes As our smart stethoscope has a pro- level, we hope to scale up our efforts so the Johns Hopkins stethoscope truly grammable platform, we’re now work- that hundreds of thousands of lives can smart, automatically screens for cases ing to develop additional apps that be saved. But on the micro level, we feel of pneumonia by distinguishing nor- would make it a multipurpose tool, use- that if even one parent sees her child mal from abnormal breathing patterns, ful for many diseases and scenarios. We recover because of early diagnosis, all looking in particular for the wheezing think it could be useful in diagnosing our hard work will have been worth it. n or crackling sounds that can indicate heart failure and various gut disorders, ↗ POST YOUR COMMENTS at https://spectrum.ieee.org/ liquid and inflammation in the lungs. and we’re collaborating with cardiolo- stethoscope0219

SPECTRUM.IEEE.ORG | NORTH AMERICAN | FEB 2019 | 41

The India of Things

Tata Communications’ countrywide IoT network aims to improve traffic, manufacturing, and health care

By V.S. SHRIDHAR

HE 20 MILLION RESIDENTS OF MUMBAI SLOG through monstrous traffic jams during monsoon season. T The heavy rains that last from June until August frequently choke off the flow of traffic through India’s most populous city, leaving millions of Mumbaikars seething in their cars after a workday. • Though it’s particularly bad during monsoon season, Mumbai’s traffic is awful even on sunny days. And meanwhile, the number of vehicles in the city continues to grow. In 2012, there were just 2 million cars, scooters, and auto-rickshaws in Mumbai. Four years later, at the end of 2016 (the most recent year for which data is available), there were 3 million. • City planners are struggling to keep up. The city’s total road length— about 2,000 kilometers—has remained stagnant over the years. More vehicles, crowded onto the same length of road, means that there are now more than 1,500 vehicles per kilometer of

ILLUSTRATION BY Dan Page SPECTRUM.IEEE.ORG | NORTH AMERICAN | FEB 2019 | 43 road. That’s even worse than Los Angeles—- vide large-scale IoT applications to ease not be able to feel their impact while infamous for its bad traffic—which, the growing pains faced by India—and you’re walking down the street today, despite having far more cars (almost potentially by other developing coun- but you will in some cities within a cou- 6.5 million), has only about 600 vehi- tries as well. ple of years. cles per kilometer. We’re already seeing how connected Mumbai has seen its population more devices can improve services in more than double since the early 1990s. The developed economies. For example, rapidly urbanizing city needs new ways UMBAI IS ONE OF MANY some cities in Europe now have con- to manage its traffic but also to cope with cities rushing to implement nected garbage trucks and garbage bins. energy usage, waste management, and M networks of connected Using IoT-enabled sensors, the bins the other effects of escalating urban- devices and sensors to increase the effi- can tell sanitation workers when they ization. For all of these problems, the ciency of public services, reduce pollu- are full or nearly so, so that they can Internet of Things (IoT) is a tool of great tion, and improve quality of life in other be collected. This approach is more promise and potential power. ways, while also keeping costs under energy efficient and cost effective than To be effective in such applications, control. Local governments around sticking to a set schedule, because the IoT would have to be deployed at a the world are exploring smart-city sys- trash accumulation varies widely: It’s scale larger than any seen so far. And tems as a way to address a variety of heaviest during holiday periods such for that to happen, a country like India municipal crises. These projects are a as C hristmas, and lowest when many would first need a robust infrastruc- cornerstone of efforts to create smart households are away on vacation, for ture backbone that could support such cities. There isn’t any universal agree- example during the summer. Such a a vast rollout of machine-to-machine ment about what the term smart city flexible schedule also helps to ease the communications. That’s why Tata Com- means, though it now implies that a flow of waste into landfill and recycling munications (at which I am a senior vice city is using IoT to make its municipal facilities, and to improve air quality president and head of the IoT division) services more efficient by some metric. by reducing pollution. City planners has built a superlow-power, secure, Analysts at the technology analysis firm in many developed regions are also bidirectional network specifically for IHS Markit project that there will be at exploring how systems of IoT nodes can massive IoT communications. It is now least 88 smart cities globally by 2025— enable more interactive and respon- the world’s largest IoT network, connect- up from just 21 in 2013. sive city administration, create safer ing over 400 million people in India’s This fourfold boom in smart cities is public spaces, and meet the needs of 44 largest cities as well as hundreds one of the reasons the advisory com- aging populations. of villages along 12 national highways. pany Gartner predicts that there will In developing countries, smart Our goal was to offer a simple, energy- be more than 20 billion IoT nodes, or devices have the potential to truly efficient way for any organization to pro- connected “things,” by 2020. You may transform people’s quality of life by completely changing the way in which services are delivered. For India especially, the Make in India program— the government’s recent plan to turn the country into a leading global manufacturing hub—is just the sort of ambitious project that can benefit from large-scale IoT networks. Launched in 2014, Make in India has prioritized upgrading infrastructure, supporting innovation, and developing skills in the country across 25 sectors of the economy, including railways, mining, and pharmaceuticals. The program has

FLOODED STREETS: Mumbai’s monsoon season gridlock might not be so bad with IoT

devices watching—and redirecting—traffic. ANDRADE/REUTERS SHAILESH

44 | FEB 2019 | NORTH AMERICAN | SPECTRUM.IEEE.ORG DELHI: Tata Communications held 35 INDIA’S different IoT trials in Delhi—as well as Amritsar Mumbai and Bengaluru—including remotely IOT monitored air conditioners and sensors to optimize electricity, gas, and water usage. Ludhiana Chandigarh NETWORK Dehradun Tata Communication’s Delhi Meerut IoT-focused network Ghaziabad stretches across India, Noida Gurgaon covering 44 cities— Faridabad with more on the way— Jaipur and villages along Rewari Lucknow 12 national highways (some of which Jodhpur Gwalior overlap on this map). Kanpur

Ahmedabad Kolkata Bhopal Jamshedpur Vadodara Indore Surat

Nagpur Mumbai Nashik Bhubneswar Aurangabad JAMSHEDPUR: Jamshedpur Pune Utilities and Services Solapur Hyderabad Co. deployed 300 smart streetlights that can be MUMBAI: Visakhapatnam remotely dimmed to conserve Mahanagar Gas energy if there is no one on a Limited deployed Huballi Vijayawada particular street. 5,000 smart gas Dharwad meters to better track natural gas usage per building. Bengaluru Chennai traffic-management system, gridlocked Mumbaikars could use more-accurate Kozhikode Mysore Tiruppur route-mapping apps, get real-time noti- Coimbatore fications of parking spaces available, Thrissur and even check predictive analytics Kochi Trivandrum to see which parking spots will most likely be available when they arrive at their destinations. The days of driving around the block endlessly to find a spot been criticized by some industry leading’s control system, management could or risking a ticket for double parking ers who feel it has failed to develop the adjust light and heating levels based on will be long gone. policies and skilled workers needed to how many people are in the building. achieve significant results. Neverthe- There’s even good news for individu- less, it has attracted investment into als. Remember those beleaguered driv- India from such large electronic sys- ers on the congested streets of Mumbai? HIS BRIGHT URBAN FUTURE tems manufacturers as Huawei, Lenovo, Think about the collective time that won’t happen unless we can and Samsung. millions of Mumbaikars could save if T overcome a significant barrier: If this effort succeeds, industrial facil- traffic lights could adjust to conditions Different IoT devices and applications ities could become smarter and better on the road ahead and redirect vehi- rely on different standards and technol- connected than ever, accruing substan- cles before traffic ever had the oppor- ogies. Because of this incompatibility, tial savings and even environmental tunity to explode into rush hour chaos. engineers must often coax devices to benefits. For example, with data from In addition, with data collected from work together in order to connect peo- sensors directly connected to a build- IoT sensors connected to a citywide ple, smartphones, and sensors.

ILLUSTRATION BY Erik Vrielink SPECTRUM.IEEE.ORG | NORTH AMERICAN | FEB 2019 | 45 Up-chirp Down-chirp Frequency

Sweep

Sync Symbol Preamble message

Time

LISTENING FOR CHIRPS: Long-range low-power, or LoRa, communication wireless options that would otherwise sends information through a signal’s change in frequency—that is, its chirp. The make large-scale networks challenging. eight up-chirp preamble catches the receiver’s attention in a noisy environment, LoRa keeps power consumption low and then two down-chirps sync the message. Then come the symbols—up-chirps by using a signal’s chirp—its change in starting at specific frequencies, with each frequency signifying a specific string frequency—to send information. Chirps of bits. LoRa’s chirps make the method more resistant to interference. come in two varieties: up‑chirps, in which the signal’s frequency increases, and down-chirps, in which it decreases. This was one of the key challenges we at plan to install thousands more of these LoRa modulates these up-chirps and Tata Communications wanted to address streetlights in cities across India. down-chirps to encode data. This four years ago as we started to build our In Mumbai, we partnered with technique uses less power at the IoT-focused network in India. By build- Mahanagar Gas Limited (MGL) to deploy cost of requiring a wider frequency ing our new network from the ground 5,000 smart gas meters in the city. MGL band for communication. The power up, we could be sure that our engineers can monitor these meters to analyze usage is so low that the battery in an wouldn’t get headaches trying to make each building’s gas usage in finer detail, IoT device connected to our network devices communicate with one another which will allow the company to deliver could last for more than a decade with- later on. The end result is a uniform IoT- natural gas more effectively. out replacement. focused network that cities or companies Building a single network for a coun- This chirp modulation is an example can easily tap into for their own projects. try as expansive as India without break- of a technique called spread spectrum. For example, in one use of the network, ing the bank or sucking too much power Spread-spectrum methods take advan- we deployed 300 smart streetlights in is a major challenge. To make it possi- tage of a larger bandwidth than other the city of Jamshedpur for Jamshedpur ble, we’re using long-range low-power types of signals, whether by a trans- Utilities and Services Co. ( JUSCO). We (LoRa) wireless communications in the mitter shifting the signal to different hope the project can reduce electricity 865-to-867-megahertz range. LoRa is a frequencies or deliberately filling other consumption by switching the lights on communication technology patented by frequencies in the band with interfer- and off or dimming them remotely from wireless semiconductor manufacturer ence. Regardless of the method, the a central command center based on the Cycleo (now owned by semiconductor goal of any spread-spectrum technique amount of traffic on any given street. If supplier Semtech). It doesn’t require is to prevent interference from other the system works in Jamshedpur, we the ravenous power consumption of signals or provide more secure commu-

46 | FEB 2019 | NORTH AMERICAN | SPECTRUM.IEEE.ORG ILLUSTRATION BY Erik Vrielink receiver just needs to note the point in ture, including streetlights, utilities, and the band where each chirp started and parking meters, all connected to India’s match it to the corresponding symbol. first IoT network. The command cen- The network has impressive reach ter will allow JUSCO to reduce the city’s because an antenna receiving informa- energy use, increase waste-management tion is listening only for the frequency at efficiency, make construction sites safer, which the next chirp starts. That’s much and alleviate rush-hour traffic. easier for an antenna to hear than infor- While this IoT network will connect mation that has been encoded through more people than ever before, there other methods, like frequency or ampli- are other, similar projects in the coun- tude modulation. Signals can easily travel try planned to help make cities and more than 15 kilometers and can pen- communities better connected: Kochi etrate a concrete wall—or three plaster in the state of Kerala, Coimbatore in walls. Such penetration means the sys- Tamil Nadu, and Bh ubaneswar in tem can be used to monitor trains coming Odisha have ambitions to become and going in metro stations, the number smart cities, and there is a smart- of cars in an underground parking lot, city project already under way near Data the water level in a r eservoir—all sites Ahmedabad, in the western state of that are difficult or impossible for tra- Gujarat, called Gujarat International ditional wireless systems. Compared Finance Tec-City (GIFT). with traditional wireless technologies— Its developers hope GIFT becomes such as Bluetooth, LTE, Wi-Fi, or Z igBee— not only the country’s first full-fledged nications. LoRa’s resistance to interfer- LoRa allows networks to reach farther, smart city but also a rival for Hong Kong, ence works by “sweeping” the signal with less power and cost. London, New York City, and Singapore across a 125-, 250-, or 500-kilohertz- Even with the capabilities of LoRa, as a major global financial services wide band assigned to the network. building a network that can function center. The Bombay Stock Exchange Any LoRa signal starts with eight well across India’s vast geographical dis- has already set up an international up-chirps—in which the signal sweeps parity and for its huge population was exchange in GIFT, and the data-center across the entire band each time—to still a significant challenge. The trickiest and connectivity infrastructure to give a receiver a heads-up that it needs part is that the frequencies from 865 to underpin the smart systems of the city to start listening for a message, as 867 MHz are unlicensed spectrum in is being deployed now. shown in the illustration above. This India. So, when designing the network, But the most promising development is alert is critical, as all the IoT devices in a we had to consider that others might be that India’s smart ambitions aren’t con- network are using that same frequency using the same band at the same time for fined to cities. The state of Rajasthan, for band. Without the use of a “preamble” other applications. Because LoRa signals example, has plans to develop more than to grab a receiver’s attention, the sub- sweep across an entire band, they are 3,000 smart villages with connected util- sequent message could easily be missed. somewhat impervious to interference ity systems to provide access to clean Two down-chirps follow, to sync the caused by other signals transmitting on drinking water, widespread Wi-Fi con- receiver as it receives the message. a specific frequency in that band. Even nectivity, e-libraries, smart streetlights, The message itself is composed of so, enough other traffic on the same fre- and telehealth facilities. symbols. Each symbol represents a quencies could still interfere with LoRa From Mumbai and Jamshedpur in India predetermined string of bits and cor- signals and prevent them from reaching to the rest of Asia and beyond, smart responds to the point in the band where their destination. cities are being built using large-scale the chirp begins. As the signal moves IoT networks. These always-connected to the next symbol in the message, it communities will transform the lives of jumps to a new frequency and completes hundreds of millions of people. We are a sweep across the band, ending at the N 2018, THE NETWORK’S just at the beginning of this journey, but same frequency where it began. Each first command center opened in nowhere will the positive outcomes be subsequent symbol denotes another I the city of Jamshedpur. In this cen- greater than in the cities of India. n jump in the frequency before the next ter, JUSCO is using over 100,000 sensors ↗ POST YOUR COMMENTS at https://spectrum.ieee.org/ sweep. To decode the message, the to digitize 15 elements of its infrastruc- indiaiot0219

SPECTRUM.IEEE.ORG | NORTH AMERICAN | FEB 2019 | 47 RESOURCES_TOOLS & TOYS

CONTINUED FROM PAGE 20 |

coupled telephone modem and a microcas- sette drive to store and retrieve programs and data. The main attraction for me was that you could jump into the monitor program from BASIC and work in 6301 machine code directly. I learned how to write structured assembly -language code on this machine. I was very proud of the fact that I wrote an entire spreadsheet program in BASIC with assembly-language fixed-point math. It took me a week to write and almost a month to debug.” —SAM FELTON

Sony 1R-81: “In 1966, I was in the eighth grade in junior high school in California. The Los Angeles Dodgers had made it to the World Series against Baltimore. I was working at a neighborhood store after school and had saved my money to buy the smallest transistor radio I had ever seen [the Sony 1R-81]. It was small enough to fit inside your ILLUMINATE ENGAGE pants pocket to sneak into school to listen to the possibilities a wider audience in the game during lunch. Back then the sign of of technology by appreciating the value a quality radio was the number of transistors using it to address and importance of it used—the more the better. Cheap radios global challenges engineering and technology were four to five transistors, but the Sony had eight! I think the Dodgers lost the series that year, but I still have my Sony radio!” EDUCATE ENERGIZE —DANIEL KIKUTA the next generation innovation of innovators by celebrating Sony WM-FX195: “It is amazing how an and engineers technological excellence electronic gadget can impact a little girl’s life. It goes from looking cool to getting an interest into technology. First off, I am from Bolivia, where [the latest] technology would arrive so The world’s most daunting challenges require late and in expensive prices. Thus, for a small innovations in engineering, and IEEE is committed girl to have a [radio-equipped] Walkman was to finding the solutions. such an honor. I could stand out among my friends and show it off. I also enjoyed listening The IEEE Foundation is leading a special campaign to radio. Tuning in a station was a challenge! I to raise awareness, create partnerships, and generate had to move the tuner [very carefully] in order financial resources needed to combat these to get as little noise as possible. This drove global challenges. me to wonder how radio transmission works and how I could make it easier to tune. Those Our goal is to raise $30 million by 2020. ideas aroused my curiosity and were my first steps to becoming a telecommunications engineer. All I can say to my Walkman is, ‘Thank DONATE NOW you for letting me discover nice music, be the popular child, and become a telecommunica- ieeefoundation.org tions engineer.’ ” —GABRIELA CASPA HUAYHUA

POST YOUR COMMENTS at https://spectrum.ieee.org/ gadgets0219 Applications for faculty positions in the Department of Electrical and Computer Engineering for tenure, research, and teaching-tracks.

The Department of Electrical and Computer Engineering (ECE) at Carnegie Mellon University is accepting applications from candidates in all areas of ECE for tenure, research, and teaching-track positions at its Pittsburgh, Kigali (Rwanda), and Silicon Valley campuses. We especially value individuals who can contribute to vertically integrated systems research that connects our department’s spectrum of strengths ranging from emerging nanoscale devices up to novel computing modalities exploiting heterogeneous integration.

Our department and the College of Engineering are ranked among the top programs in the United States both at the undergraduate and graduate levels. We house and have ties to several multidisciplinary institutes and centers. We collaborate with colleagues around the world through a number of formal research and educational programs. We have extensive experimental and computing infrastructure, including state-of-the-art nanofabrication facilities.

We are a department strongly committed to all members of our community: students, faculty, and sta . Our vision is to be a creative driving force, within the university and worldwide, of the highest scholarly and entrepreneurial quality. Our mission is to inspire and educate engineers capable of pursuing fundamental scientic problems and important societal challenges. We strive to accomplish this with the highest commitment to quality, integrity, and respect for others. We are particularly interested in applicants who are committed to and have passion for a culturally diverse environment in research and/or teaching, and demonstrate a willingness to nurture the uniquely inclusive Carnegie Mellon environment. We take pride and active steps in considering a diverse applicant pool in terms of gender, race, veteran status, and disability. Carnegie Mellon University seeks to meet the needs of dual-career couples and is a member of the Higher Education Recruitment Consortium (HERC) that assists with dual-career searches.

For all tracks, we are seeking individuals who hold a Ph.D. in a relevant discipline and have demonstrated commitment to our core values: scientic truth, creativity, quality, innovation, and engineering solutions, all within a diverse and tight-knit community guided by respect and joy of doing. Faculty positions are primarily at the Assistant Professor level; however, appointments may be made at the rank of Associate Professor or Professor depending on the qualications.

Tenure-track faculty carry a moderate teaching load that allows time for quality research and close involvement with students. We expect you to establish and grow a strong research program, contribute to our teaching mission, and show your passion for mentoring and advising students.

Research-track faculty are not required to teach, but do so occasionally when of clear benet to the faculty and the department; you will be compensated for both teaching and advising Ph.D. students. You will typically focus on developing leadership within your area of research, developing research collaborations, and supervising Ph.D. students.

Teaching-track faculty typically focus exclusively on teaching and service, but may conduct research as well. We will rely on you to help strengthen our teaching and mentoring mission. The emphasis of the teaching track position in Silicon Valley is on Software Engineering (software engineering foundations, functional programming, large software systems, software analytics, cloud computing, and engineering of big data and AI-based systems). The emphasis of the teaching track position in Pittsburgh is on Computer Systems (digital system design, FPGAs, C and assembly programming, computer architecture, cyber-physical and embedded systems, etc.). The emphasis of the teaching track position in Africa is on Data Science, Cybersecurity, and Software Engineering.

Please submit an online application at www.ece.cmu.edu/about/employment-opportunities.html. We will consider applications tting our needs throughout the academic year. Carnegie Mellon is an EEO/Armative Action Employer -- M/F/Disability/Veteran. Professor & Chair Department of Electrical Engineering Assistant/Associate/Full Professor in the ECE University of South Florida Department at Stevens Institute of Technology The College of Engineering at the University of South Florida invites applications The Department of Electrical and Computer at Stevens Institute of Technology and nominations for the position of Professor and Chair of the Department of invites applications for several tenure-track/tenured faculty positions at the rank of Electrical Engineering. The new Chair will have a unique opportunity to impact Assistant/Associate/Full Professors, starting on August 16, 2019 or later. Qualified the direction of a department with a vibrant research environment, a commitment candidates can also be considered for an endowed chair professor position. to excellence in undergraduate and graduate education, a strong record of Applicants should have earned a Ph.D. in Electrical or Computer Engineering or a engagement with industrial and community partners, and a trajectory for continued related discipline. The department is looking for researchers with a strong funding significant growth. The position is expected to commence August 2019. and publication record in key areas of interest: computer architecture, electronics Candidates should qualify to be appointed as a tenured full professor with and digital system design, smart and automated systems, and artificial intelligence. Successful applicants are expected to develop a strong externally funded, globally expertise in fields related to electrical engineering. Applicants are expected recognized research program. They should also possess a passion for and be to have established research enterprises that are well-funded and recognized committed to excellence in both undergraduate and graduate education. nationally and internationally. Preference will be given to candidates who possess an excellent record of scholarship in programs offering both undergraduate and Stevens Institute of Technology is a private university located in Hoboken, New graduate degrees in fields related to electrical engineering. Experience working in a Jersey. Stevens is an Equal Opportunity Employer that is building a diverse faculty, multicultural and diverse environment will also be considered a strength. Preferred staff and student body and strongly encourages applications from female and minority candidates as well as veterans and individuals with disabilities. Stevens is candidates will have a record of leadership and administrative and management an NSF ADVANCE institution committed to equitable practices and policies. responsibilities, which may include leading a large federally-funded research group or center, and holding a Chair or Associate Chair position. Applications will be accepted until the positions are filled. All applications must be submitted electronically through the HR website at https://stevens. The Department includes 32 faculty members, that include several NSF CAREER wd5.myworkdayjobs.com/en-US/External/job/Hoboken-NJ---Main-Campus/ Award winners, Fellows of the IEEE and AAAS, Distinguished University Assistant-Associate-Professor--Electrical-and-Computer-Engineering_ Professors, and a member of the National Academy of Engineering. Many serve in RQ22188. Applicants should submit their curriculum vitae, a research plan (3-5 national leadership roles in technical societies such as the IEEE and have industry pages), teaching interests and philosophy, and contact information including at experience to complement their academic backgrounds. least three references to the HR system. For any inquiries, please contact the Search Committee Chair, Prof. Hong Man ([email protected]). https://www.usf.edu/engineering/ee/employment/currentopenings.aspx

Best New Journal in STM 2015

Electrical Engineering seeks dynamic new faculty, at any rank, to begin Fall 2019. Become a For more information on research areas of particular interest, visit http://cec.sc.edu/employment published author Apply at: https://uscjobs.sc.edu/postings/44888 One of the most To ensure full consideration, applications should be in 4 to 6 weeks. received by February 28, 2019. Search will remain Published online only, IEEE Access is ideal influential reference open until the position is filled for authors who want to quickly announce resources for engineers Ph.D required. recent developments, methods, or new around the world. USC is an EO/AA employer. products to a global audience. For over 100 years, Proceedings of the IEEE • Submit multidisciplinary articles that do not fit neatly in traditional journals has been the leading journal for engineers • Reach millions of looking for in-depth tutorial, survey, and global users through review coverage of the technical the IEEE Xplore® developments that shape our world. digital library with Department Chair free access to all The Department of Electrical Engineering The University of Texas at Arlington Included in The Electrical Engineering Department at The Web of Science To learn more and start and has an University of Texas at Arlington seeks nominations your subscription today, visit Impact Factor and applications for Department Chair. Qualified ieee.org/proceedings-subscribe candidates will be considered for the Janet and Mike Greene Endowed Professorship. Further information and an online application may be found Learn more at: ieeeaccess.ieee.org

at: https://uta.peopleadmin.com/postings/7424 17-PUB-013 3/17

50 | FEB 2019 | NORTH AMERICAN | SPECTRUM.IEEE.ORG Faculty Position The Electrical and Computer Engineering Department of Baylor University seeks faculty applicants for a Clinical Assistant/Associate Professor. Any area of expertise in ECE will be considered. Applicants must possess an earned masters or doctorate degree and Tenure-Track Faculty Position extensive industry experience, and demonstrate potential for excellent teaching; applicants for Electrical Engineering Clinical Associate Professor must additionally present evidence of deeper industry experience and Western Washington University invites applications for achievement in teaching commensurate with the desired rank. The ECE department offers B.S., the full-time, tenure-track, PACCAR Professor of Electrical M.S., M.E. and Ph.D. degrees and is rapidly expanding its faculty size. Facilities include the Baylor Engineering position with a start date of September 16, Research and Innovation Collaborative (BRIC), a newly-established research park minutes from the 2019. main campus. While expecting to hire at the Assistant Professor level, candidates with exceptional quali cations may be Chartered in 1845 by the Republic of Texas, Baylor University is the oldest university in Texas. considered at the Associate Professor or Professor rank. Baylor has an enrollment of over 15,000 students and is a member of the Big XII Conference. The PACCAR Professor of Electrical Engineering will have Baylor’s mission is to educate men and women for worldwide leadership and service by integrating an important role in the growth of the EE program by academic excellence and Christian commitment within a caring community. The department seeks developing curriculum and facilitating research and to hire faculty with an active Christian faith; applicants are encouraged to read about Baylor’s vision development partnerships with PACCAR Inc. We seek an individual with teaching and research interests in elds for the integration of faith and learning at www.baylor.edu/profuturis/. applicable to automated vehicles, connected vehicles, or Applications will be considered on a rolling basis starting January 1, 2019. Applications must include: vehicle electri cation. The responsibilities include developing and maintaining 1) a letter of interest that identifies the applicant’s anticipated rank, a program of scholarship and developing and teaching 2) a complete CV, lab-based electrical engineering courses with a commitment to cultivating learning environments that 3) a concise statement of teaching interests, are equitable and inclusive of all students. Western is the highest-ranking public, master's-granting 4) the names and contact information for at least three professional references. university in the Paci c Northwest, according to the 2019 Additional information is available at www.ecs.baylor.edu. Should you have any questions on the U.S. News & World Report rankings and is committed to creating equitable and inclusive learning and working position, feel free to contact the search chair, Dr. Keith Schubert at [email protected]. environments for their students, faculty, and sta. Upload materials via Baylor’s iApply system accessible at http://apply.interfolio.com/57439. See the full announcement at: Baylor University is affiliated with the Baptist General Convention of Texas. As an Affirmative Action/Equal https://cse.wwu.edu/engineering-design/employment Employment Opportunity employer, Baylor encourages candidates of the Christian faith who are minorities, Job no: 496353 Review of applications will begin February 11, 2019 and women, veterans, and persons with disabilities to apply. will continue until the position is lled. AA/EOE

Lecturer of Assistant, Associate or Full Professor – Multidisciplinary Computer Engineering Technology Program Engineering The Department of Engineering Technology and Industrial Distribution at Texas A&M University invites applications for a tenured or tenure-track faculty position at the assistant, associate, or The Department of Electrical and Computer full professor level with expertise in one or more of the following areas: Mechatronics, Industrial Engineering invites applications for a and Mobile Robotics, Automation, Product Design, Industrial Internet of Things (IIoT), Cyber- 12-month, non-tenure track lecturer position in the area of Computer Physical Systems, and Embedded Systems. The successful applicant will be required to teach; Engineering to start September 2019. The advise and mentor undergraduate and graduate students; develop an independent, externally successful candidate will teach computer funded research program; participate in all aspects of the department’s activities; and serve the engineering undergraduate and graduate profession. Strong written and verbal communication skills are required. Preference will be given classes in computer hardware, computer to candidates with recent and relevant hands-on experience with applied research and technology software and computer systems such as data development in robotics and automation, academic leadership experience and/or experience with structures, embedded systems, computer ABET and accreditation processes Applicants should consult the department’s website to review architecture, computer networks, and operating systems. our academic and research programs (https://engineering.tamu.edu/etid). Applicants must have an earned doctorate in an appropriate engineering field and/or a closely related engineering or Applicants should apply at: science discipline. http://careers.umass.edu/amherst/en-us/ job/496606/lecturer-computer-engineering Applicants should submit a cover letter, curriculum vitae, teaching statement, research statement, and a list of four references (including postal addresses, phone numbers and email addresses) by the priority deadline of January 14, 2019 to apply for this specific position at www.tamengineeringcareers.org. Full consideration will be to ensure consideration. given to applications received by February 15, 2019. Applications received after that date may be The University of Massachusetts Amherst is considered until the position is filled. It is anticipated the appointment will begin Fall 2019. an Affirmative Action/Equal Opportunity Employer of women, minorities, protected The members of Texas A&M Engineering are all Equal Opportunity/Affirmative Action/Veterans/Disability veterans, and individuals with disabilities employers committed to diversity. It is the policy of these members to recruit, hire, train and promote without and encourages applications from these and regard to race, color, sex, religion, national origin, age, disability, genetic information, veteran status, sexual other protected group members. orientation or gender identity.

SPECTRUM.IEEE.ORG | NORTH AMERICAN | FEB 2019 | 51

Publication: IEEE Global Spec Inc Size: 2.18752.1875” x 4.75” Notes: b/w Job# 45659 IO#: 45659-45659 Screen: 85 dpi Mechanical: mbb, mbb Proofreader: PAST FORWARD_BY ALLISON MARSH

When a company monopolizes an industry, as AT&T did with telecommunications in the United States for PRETTY much of the 20th century, flashy design isn’t a priority. And yet in 1959 the company debuted a beauty queen— or rather a Princess. Smaller and lighter weight than the blocky black desk phone, the streamlined Princess burst IN PINK onto the scene in a delicious palette that included pink, aqua, and turquoise. It even had a built-in n ight-light. Advertisements targeting women and teens pitched the phone as the perfect extension for the bedroom. And so the Princess led its users unwittingly down the path to being always connected. ■ For more on the Princess phone, see https://spectrum.ieee.org/pastforward0219 AT&T ARCHIVES AND HISTORY CENTER HISTORY AND ARCHIVES AT&T

52 | FEB 2019 | NORTH AMERICAN | SPECTRUM.IEEE.ORG Some things don’t make sense

Some things do. IEEE Member Group Term Life Insurance Plan — It just makes sense.

IEEE acts as an advocate to ensure member benefits remain valuable.* As a result, the IEEE Member Group Term Life Insurance Plan has affordable group rates, portable coverage, premium and volume discounts, Wantan accelerated to join thousandsdeath benefit of and your benefit fellow amounts IEEE upmembers? to $1 million . Apply online today!

IEEEinsurance.com/TLplan.** Learn more about the IEEE Member Group Term Life Insurance Plan. Program Administered by Mercer *TheVisit Group Term Life Insurance Plan is available only for residents of the U.S. (except territories), Puerto Rico Health & Benefits Administration LLC and Canada (except Quebec). This plan is underwritten by New York Life Insurance Company, 51 Madison Ave., New York, NY 10010 on Policy Form GMR. This plan is administered by Mercer Consumer, a service of In CA d/b/a Mercer Health & Benefits Mercer Health & Benefits Administration LLC. This coverage is available to residents of Canada (except Quebec). Mercer (Canada) Limited, represented by its employees Nicole Swift and Suzanne Dominico, acts as broker Insurance Services LLC with respect to residents of Canada. AR Insurance License #100102691 **For information on features, costs, eligibility, renewability, limitations and exclusions. CA Insurance License #0G39709 85516 (2/19) Copyright 2019 Mercer LLC. All rights reserved.

85516 IEEE TL Humor Ad 2019.indd 1 11/8/18 2:06 PM 85516 (2/19) IEEE Group Term life Ad Trim Size: 7.875" x 10.5" Live Area: 7" x 10" Bleed: 8.125" x 10.75" Colors 4C - CMYK Stock: N/A MERCER Title: na-mls-machine-learn-blue-7.87X10.75 Creative ServicesProject Manager This advertisementprepared by: Client Name: The Mathworks [email protected] Size: 7.875”x10.75” Natick, MA01760 Elizabeth Putnam 1 AppleHillDrive REQ #:110718D 508-647-0589 MathWorks Page 4/c LEARNING MACHINE MATLAB mathworks.com/machinelearning With MATLAB and puttheminto production. learning to buildpredictive models regression,deep and classification, SPEAKS Cosmos Communications Cosmos Communications ® you can use clustering, 38686a2 K Y M C 11.9.18 133 ej 1 4

1Q2 Q1

@2019 The MathWorks, Inc.