Security and Privacy > Smart Homes > Autonomous Vehicles > Robotics

> Security and Privacy > Smart Homes > Autonomous Vehicles > Robotics

AUGUST 2019 www.computer.org Keep Your Career Options Open Upload Your Resume Today!

Whether your enjoy your current position or you are ready for change, the IEEE Computer Society Jobs Board is a valuable resource tool. Take advantage of these special resources for job seekers:

JOB ALERTS TEMPLATES

CAREER RESUMES VIEWED ADVICE BY TOP EMPLOYERS No matter your career WEBINARS level, the IEEE Computer Society Jobs Board keeps you connected to workplace trends and exciting new career prospects.

www.computer.org/jobs IEEE COMPUTER SOCIETY computer.org • +1 714 821 8380

STAFF

Editor Publications Portfolio Managers Cathy Martin Carrie Clark, Kimberly Sperka

Publications Operations Project Specialist Publisher Christine Anthony Robin Baldwin Publications Marketing Project Specialist Meghan O’Dell Senior Advertising Coordinator Debbie Sims Production & Design Carmen Flores-Garvey

Circulation: ComputingEdge (ISSN 2469-7087) is published monthly by the IEEE Computer Society. IEEE Headquarters, Three Park Avenue, 17th Floor, New York, NY 10016-5997; IEEE Computer Society Publications Office, 10662 Los Vaqueros Circle, Los Alamitos, CA 90720; voice +1 714 821 8380; fax +1 714 821 4010; IEEE Computer Society Headquarters, 2001 L Street NW, Suite 700, Washington, DC 20036. Postmaster: Send address changes to ComputingEdge-IEEE Membership Processing Dept., 445 Hoes Lane, Piscataway, NJ 08855. Periodicals Postage Paid at New York, New York, and at additional mailing offices. Printed in USA. Editorial: Unless otherwise stated, bylined articles, as well as product and service descriptions, reflect the author’s or firm’s opinion. Inclusion in ComputingEdge does not necessarily constitute endorsement by the IEEE or the Computer Society. All submissions are subject to editing for style, clarity, and space. Reuse Rights and Reprint Permissions: Educational or personal use of this material is permitted without fee, provided such use: 1) is not made for profit; 2) includes this notice and a full citation to the original work on the first page of the copy; and 3) does not imply IEEE endorsement of any third-party products or services. Authors and their companies are permitted to post the accepted version of IEEE-copyrighted material on their own Web servers without permission, provided that the IEEE copyright notice and a full citation to the original work appear on the first screen of the posted copy. An accepted manuscript is a version which has been revised by the author to incorporate review suggestions, but not the published version with copy-editing, proofreading, and formatting added by IEEE. For more information, please go to: http://www.ieee.org/publications_standards/publications/rights/paperversionpolicy.html. Permission to reprint/republish this material for commercial, advertising, or promotional purposes or for creating new collective works for resale or redistribution must be obtained from IEEE by writing to the IEEE Intellectual Property Rights Office, 445 Hoes Lane, Piscataway, NJ 08854-4141 or [email protected]. Copyright © 2019 IEEE. All rights reserved. Abstracting and Library Use: Abstracting is permitted with credit to the source. Libraries are permitted to photocopy for private use of patrons, provided the per- copy fee indicated in the code at the bottom of the first page is paid through the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923. Unsubscribe: If you no longer wish to receive this ComputingEdge mailing, please email IEEE Computer Society Customer Service at [email protected] and type “unsubscribe ComputingEdge” in your subject line. IEEE prohibits discrimination, harassment, and bullying. For more information, visit www.ieee.org/web/aboutus/whatis/policies/p9-26.html.

IEEE Computer Society Magazine Editors in Chief

Computer IEEE Security & Privacy Computing in Science David Alan Grier (Interim), David Nicol, University of Illinois & Engineering Djaghe LLC at Urbana-Champaign Jim X. Chen, George Mason University IEEE Micro IEEE Software IEEE Intelligent Systems Lizy Kurian John, University of Ipek Ozkaya, Software V.S. Subrahmanian, Dartmouth Texas, Austin Engineering Institute College IEEE MultiMedia IEEE Internet Computing IEEE Computer Graphics Shu-Ching Chen, Florida George Pallis, University of and Applications International University Cyprus Torsten Möller, University of Vienna IEEE Annals of the History IT Professional of Computing Irena Bojanova, NIST IEEE Pervasive Computing Marc Langheinrich, University of Gerardo Con Diaz, University of Lugano California, Davis

www.computer.org/computingedge 1 AUGUST 2019 • VOLUME 5, NUMBER 8

THEME HERE 14 18 40 Penetration IoT Safety: State Lance Gharavi: Testing in the of the Art Performance- IoT Age Inspired Science + Technology Security and Privacy 8 The Fuzzing Hype-Train: How Random Testing Triggers Thousands of Crashes MATHIAS PAYER 14 Penetration Testing in the IoT Age CHUNG-KUAN CHEN, ZHI-KAI ZHANG, SHAN-HSIN LEE, AND SHIUHPYNG SHIEH

Smart Homes 18 IoT Safety: State of the Art JANUSZ ZALEWSKI 23 Physical Computing’s Connected and Shape- Changing Future HEATHER M. PATTERSON

Autonomous Vehicles 28 Next-Generation Smart Environments: From System of Systems to Data Ecosystems EDWARD CURRY AND AMIT SHETH 37 My Mother the Car (or Why It’s a Bad Idea to Give Your Car a Personality) PHIL LAPLANTE

Robotics 40 Lance Gharavi: Performance-Inspired Science + Technology BRUCE CAMPBELL AND FRANCESCA SAMSEL 47 A Comet Revisited: Lessons Learned from Philae’s Landing ANDRÁS BALÁZS

Departments 4 Magazine Roundup 7 Editor’s Note: Finding Flaws in Security 47 58 Conference Calendar A Comet Revisited: Lessons Learned from Philae’s Landing Subscribe to ComputingEdge for free at www.computer.org/computingedge. CS FOCUS

Magazine Roundup

eases software reuse but has yet to be widely adopted in scientifi c computing. In this article from the March/April 2019 issue of Computing in Science & Engi- neering, the authors propose embedding component frame- enterprises have been pro- works inside high-performance he IEEE Computer posed, but how eff ective are languages directly to improve Society’s lineup of 12 these schemes? The authors of fl exibility. They present this T peer-reviewed tech- this article from the April 2019 approach through the example nical magazines covers cut- issue of Computer evaluate the of a high-performance Bayesian ting-edge topics ranging from eff ectiveness of Cyber Essen- inference application. software design and computer tials and fi nd that its security graphics to Internet comput- controls work well to mitigate IEEE Annals of the ing and security, from scien- threats that exploit vulnerabili- History of Computing tifi c applications and machine ties remotely with commodity- intelligence to visualization level tools. Hacking the Cis-tem and microchip design. Here are This article from the Janu- highlights from recent issues. Computing in Science & ary–March 2019 issue of IEEE Engineering Annals of the History of Comput- Computer ing looks at the case of trans- Fostering Reuse in gender Britons who tried to Basic Cyber Hygiene: Scientifi c Computing With correct the gender listed on their Does It Work? Embedded Components government-issued ID cards, A number of security certifi ca- Component-based programming but ran up against the Brit- tions for small- and medium-size is a programming paradigm that ish government’s increasingly

4 August 2019 Published by the IEEE Computer Society 2469-7087/19/$33.00 © 2019 IEEE computerized methods for track- data. The network takes the binary as a multi-objective optimization ing, identifying, and defi ning citi- image of a sketched object as input problem for optimizing cost and zens. These newly computerizing and produces a corresponding time. The performance of the algo- systems show some of the earli- segmentation map with per-pixel rithm is evaluated by mapping it est examples of transphobic algo- labelings as output. A subsequent to a dynamic model of the multi- rithmic bias; explicit attempts to post-process procedure with multi- commodity transportation problem program trans people out of the label graph cuts further refi ne the with multiple optimizing parame- system can be seen in the program- segmentation and labeling result. ters. The algorithm is found to ter- ming of the early Ministry of Pen- The authors validate the proposed minate successfully in linear time sions computer system designed method on two sketch datasets. at each destination accounting for to apportion benefi ts to all tax- Experiments show that the method uncertainties, which are the natu- paying British citizens. Transgen- outperforms the state-of-the-art ral property of the real world, thus der citizens pushed back against method in terms of segmentation optimizing the objective function these developments, attempting to and labeling accuracy and is sig- at all nodes. hack the bureaucratic avenues and nifi cantly faster, enabling further categories available to them, lay- integration in interactive draw- IEEE Internet Computing ing the groundwork for a coalesc- ing systems. The authors demon- ing political movement. This strate the method’s effi ciency in a meChat: In-Device Personal article argues that uncovering the sketch-based modeling applica- Assistant for Conversational deep prehistory of algorithmic tion that automatically transforms Photo Sharing bias and investigating instances input sketches into 3D models by It is still challenging to search in- of resistance within this history is part assembly. device photos without the sac- essential to understanding current rifi ce of privacy or poor latency, debates about algorithmic bias, IEEE Intelligent Systems despite the fact that photos have and how computerized systems long been an essential part of have long functioned to create and Multimodal Foraging communication in messaging enforce norms and hierarchies. by Honey Bees toward applications. In this article from Optimizing Profi ts at Multiple the March/April 2019 issue of IEEE IEEE Computer Graphics Colonies Internet Computing, the authors and Applications Honey bees single out from avail- propose a novel in-device per- able foraging sources by evaluat- sonal assistant for conversational Fast Sketch Segmentation and ing the amount of energy needed photo sharing, called meChat. Labeling With Deep Learning to transport an article for liveli- By understanding the user’s con- The authors of this article from the hood. In this article from the Janu- versations and in-device photos, March/April 2019 issue of IEEE ary/February 2019 issue of IEEE meChat intelligently searches in- Computer Graphics and Applica- Intelligent Systems, the authors device photos that are semanti- tions present a simple and effi cient propose a novel method based on cally relevant to the conversation method based on deep learn- the dynamic and distributed com- context. Notably, meChat works in ing to automatically decompose puting behavior of honey bees at a stand-alone, privacy-protecting sketched objects into semantically distinct colonies in gathering mul- manner without sending out any valid parts. They train a deep neu- tiple resources for meeting their in-device personal content. The ral network to transfer existing demands at respective destina- scenario-based user studies show segmentations and labelings from tions by maximizing the profi t. A that meChat effi ciently searches 3D models to freehand sketches computational model depicting the highly relevant in-device photos without requiring numerous well- multidimensional swarm behav- with low perceived latency and annotated sketches as training ior in bee colonies is developed energy consumption. www.computer.org/computingedge 5 MAGAZINE ROUNDUP

IEEE Micro method proposed in this article machine-learning (ML) systems, is applied to calculate the video the knowledge gap between the Samsung M3 Processor experience scores and achieves ML and privacy communities The M3 processor is Samsung’s 8% higher average test accuracy must be bridged. This article from third-generation custom microarchi- than other prevailing methods. the March/April 2019 issue of IEEE tecture. As performance demands Extensive experimental results Security & Privacy aims to provide continue to grow, major design show that this approach can an introduction to the intersection improvements are required. In this accurately reveal the user’s expe- of both fi elds with special empha- generation, the core is enhanced rience toward video content and sis on the techniques used to pro- with a six-wide microarchitecture, is expected to become a valid and tect the data. deeper out-of-order resources, and useful QoE model. faster instructions. The M3 deliv- IEEE Software ers a new level of performance to IEEE Pervasive Computing the Android ecosystem. Read more Strategies for Competing in in the March/April 2019 issue of Supporting the IoT the Automotive Industry’s IEEE Micro. Business Value Through the Software Ecosystem: Platformization of Pilots Standards and Bottlenecks IEEE MultiMedia Testbeds are powerful large-scale The automotive industry includes experimentation tools. Often, how- many actors engaged in soft- QoE-Oriented Multimedia ever, their limited scope and/or ware. This article from the May/ Assessment: A Facial unrealistic functionality result in June 2019 issue of IEEE Software Expression Recognition anecdotal use. Knowledge transfer focuses on the controlling position Approach becomes more eff ective through of car manufacturers in the auto- Multimedia services are predomi- pilots, but they involve techno-eco- motive software ecosystem and nant in current wireless networks nomic challenges with uncertain suggests three strategies for soft- and are becoming ubiquitous in outcomes. Platform-based design ware innovators: contesting, coop- the upcoming 5G era in which the has proven a valuable strategy to erating, and circumventing. video quality of experience (QoE) overcome similar barriers. How- is a fundamental metric. How- ever, the authors of this article from IT Professional ever, no widely accepted QoE the October–December 2018 issue model exists due to its subjective of IEEE Pervasive Computing iden- Decentralization: The Failed nature. This article from the Jan- tify that while this platformization is Promise of Cryptocurrencies uary–March 2019 issue of IEEE happening in the cloud elements of Cryptocurrencies promise to revo- MultiMedia proposes a frame- the value chain, Internet-of-Things lutionize the fi nancial market due work for quantifying the QoE (IoT) devices are still designed in to two main features: security and of multimedia content based on an application-specifi c manner, decentralization. In this article the facial expression approach, thus limiting pilots’ initiatives. from the March/April 2019 issue of which can directly refl ect users’ IT Professional, the authors analyze intrinsic attitudes toward the ser- IEEE Security & Privacy whether cryptocurrencies are fully vices. To achieve this objective, decentralized—in other words, a face database is established, Privacy-Preserving Machine whether the transaction process- which contains over 1,000 videos Learning: Threats and ing is distributed among diff erent and serves as a dataset for the Solutions entities. In addition, they present subsequent experience mining. For privacy concerns to be the consequences that a possible The semi-supervised clustering addressed adequately in today’s centralization entails.

6 ComputingEdge August 2019 EDITOR’S NOTE

Finding Flaws in Security

atching and fi xing security and privacy from IEEE Pervasive Computing, goes a step fur- vulnerabilities before they are exploited ther and argues that smart-home devices should C is a crucial part of computer hardware not only be secure and safe, but also contribute to and software engineering. A growing number of social progress and human wellbeing. tools are helping engineers assess and test for Smart devices are also key components in security weaknesses. This issue of ComputingEdge autonomous vehicles. In IEEE Intelligent Systems’ covers two eff ective testing techniques: fuzzing “Next-Generation Smart Environments: From Sys- and penetration testing. tem of Systems to Data Ecosystems,” the authors Fuzzing is the process of executing a software discuss the enormous amount of data generated program using random inputs with the goal of by self-driving cars and the opportunities and discovering bugs. IEEE Security & Privacy’s “The challenges associated with processing and shar- Fuzzing Hype-Train: How Random Testing Trig- ing that data. “My Mother the Car (or Why It’s a gers Thousands of Crashes” describes fuzzing’s Bad Idea to Give Your Car a Personality),” from many benefi ts and challenges. Penetration test- IT Professional, explores the concept of anthropo- ing, on the other hand, simulates cyberattacks to morphic autonomous vehicles through the lens of uncover fl aws. Computer’s “Penetration Testing science-fi ction stories. in the IoT Age” proposes strategies for improving This ComputingEdge issue concludes with penetration testing of IoT objects. two articles on robotics. IEEE Computer Graphics IoT devices must be not only secure, but also and Applications’ “Lance Gharavi: Performance- safe—especially in smart homes. “IoT Safety: Inspired Science + Technology” features an art- State of the Art,” from IT Professional, stresses ist who integrates robots and other cutting-edge the importance of preventing Internet-connected technologies into his projects. IEEE Software’s “A appliances and control systems from malfunction- Comet Revisited: Lessons Learned from Philae’s ing and hurting people or property. “Physical Com- Landing” provides takeaways from the robotic puting’s Connected and Shape-Changing Future,” lander’s mission.

2469-7087/19/$33.00 © 2019 IEEE Published by the IEEE Computer Society August 2019 7 SYSTEMS ATTACKS AND DEFENSES Editors: D. Balzarotti, [email protected] | W. Enck, [email protected] | T. Holz, [email protected] | A. Stavrou, [email protected]

e Fuzzing Hype-Train: How Random Testing Triggers ousands of Crashes

Mathias Payer | EPFL, Lausanne, Switzerland

o ware contains bugs, and some System so ware, such as a browser, discovered crash. As a dynamic testing S bugs are exploitable. Mitigations a runtime system, or a kernel, is writ- technique, fuzzing is incomplete for protect our systems in the presence ten in low-level languages (such as C nontrivial programs as it will neither of these vulnerabilities, o en stop- and C++) that are prone to exploit- cover all possible program paths nor ping the program once a security able, low-level defects. Undened all data-ow paths except when run violation has been detected. e alter- behavior is at the root of low-level for an innite amount of time. Fuzz- native is to discover bugs during de- vulnerabilities, e.g., invalid pointer ing strategies are inherently optimiza- velopment and x them tion problems where the in the code. e task of available resources are nding and reproducing e idea of fuzzing is simple: execute used to discover as many bugs is dicult; however, bugs as possible, covering fuzzing is an ecient way a program in a test environment with as much of the program to find security-critical random input and see if it crashes. functionality as possible bugs by triggering ex- through a probabilistic ceptions, such as crashes, exploration process. Due memory corruption, or to its nature as a dynamic assertion failures automatically (or dereferences resulting in memory testing technique, fuzzing faces several with a lile help). Furthermore, fuzz- corruption, casting to an incompat- unique challenges: ing comes with a witness (proof of the ible type leading to type confusion, vulnerability) that enables developers integer overflows, or application ■ Input generation: Fuzzers generate to reproduce the bug and x it. programming interface (API) con- inputs based on a mutation strat- So ware testing broadly focuses fusion. To cope with the complexity egy to explore a new state. Because on discovering and patching bugs of current programs and nd bugs, the fuzzer is aware of the program during development. Unfortunately, companies such as Google, Micro- structure, it can tailor input gener- a program is only secure if it is free so , and Apple integrate dynamic ation to the program. e under- of unwanted exceptions. Security, testing into their so ware develop- lying strategy determines how therefore, requires proof of the ab- ment cycle. effectively the fuzzer explores a sence of security violations. For exam- Fuzzing, the process of provid- given state space. A challenge for ple, a bug becomes a vulnerability if ing random input to a program to input generation is finding the any aacker-controlled input reaches intentionally trigger crashes, has been balance between exploring new a program location that allows a around since the early 1980s. A revival paths (control ow) and execut- security violation, such as memory of fuzzing techniques is taking place ing the same paths with dierent corruption. So ware security testing, as evidenced by papers presented at input (data ow). therefore, requires reasoning about all top-tier security conferences show- ■ Execution engine: e execution possible executions of code at once ing improvements in the techniques’ engine takes newly generated input to produce a witness that violates the eectiveness. e idea of fuzzing is and executes the program under security property. As Edsger W. Dijks- simple: execute a program in a test test with that input to detect aws. tra said in 1970: “Program testing can environment with random input and Fuzzers must distinguish between be used to show the presence of bugs, see if it crashes. e fuzzing process is benign and buggy executions. Not but never to show their absence!” inherently sound but incomplete. By every bug results in an immediate producing trial cases and observing segmentation fault, and detecting a

Digital Object Identifier 10.1109/MSEC.2018.2889892 whether the tested program crashes, state violation is a challenging task, Date of publication: 20 March 2019 fuzzing produces a witness for each especially as code generally does

8 August 2019 Published by the IEEE Computer Society 2469-7087/19/$33.00 © 2019 IEEE 78 January/February 2019 Copublished by the IEEE Computer and Reliability Societies 1540-7993/19©2019IEEE SYSTEMS ATTACKS AND DEFENSES Editors: D. Balzarotti, [email protected] | W. Enck, [email protected] | T. Holz, [email protected] | A. Stavrou, [email protected]

not come with a formal model. Knowledge of the input structure during fuzzing to provide input to e Fuzzing Hype-Train: How Random Additionally, the fuzzer must dis- given through a formal description the tness function. White-box fuzz- ambiguate crashes to identify bugs enables model-based input genera- ing infers the program specication without missing true positives. tion to produce (mostly) valid test through program analysis but oen Testing Triggers ousands of Crashes ■ Coverage wall: Fuzzing struggles cases. e model species the input results in untenable cost. For exam- with some aspects of code. It may, format and implicitly indicates the ple, the scalable automated guided for example, have diculty han- explorable state space. Based on the execution white-box fuzzer leverages Mathias Payer | EPFL, Lausanne, Switzerland dling a complex API, checksums model, the fuzzer can produce valid symbolic execution to explore dier- in le formats, or hard compari- test cases that satisfy many checks ent program paths. Black-box fuzzing sons, such as a password check. in the program, such as valid state blindly generates new input without Preparing the fuzzing environ- checks, dependencies between elds, reection. e lack of a tness func- o ware contains bugs, and some System so ware, such as a browser, discovered crash. As a dynamic testing ment is a crucial step to increase or checksums such as a CRC32. For tion limits black-box fuzzing to func- S bugs are exploitable. Mitigations a runtime system, or a kernel, is writ- technique, fuzzing is incomplete for the eciency of fuzzing. example, without an input model, tionality close to the provided test protect our systems in the presence ten in low-level languages (such as C nontrivial programs as it will neither ■ Evaluating fuzzing effectiveness: most randomly generated test cases cases. Grey-box fuzzing leverages of these vulnerabilities, o en stop- and C++) that are prone to exploit- cover all possible program paths nor Dening the metrics for evaluat- will fail the equality check for a cor- lightweight program instrumenta- ping the program once a security able, low-level defects. Undened all data-ow paths except when run ing the effectiveness tion instead of heavier violation has been detected. e alter- behavior is at the root of low-level for an innite amount of time. Fuzz- of a fuzzing campaign program analysis to infer native is to discover bugs during de- vulnerabilities, e.g., invalid pointer ing strategies are inherently optimiza- is challenging. For rough input generation, the fuzzer coverage during the fuzz- velopment and x them tion problems where the most programs, the ing campaign itself, merg- in the code. e task of available resources are state space is (close to) implicitly selects which parts of the ing analysis and testing. nding and reproducing e idea of fuzzing is simple: execute used to discover as many innite, and fuzzing is tested program are executed. Coverage-guided gray- bugs is dicult; however, bugs as possible, covering a brute-force search in box fuzzing combines fuzzing is an ecient way a program in a test environment with as much of the program this state space. Decid- mutation-based input to find security-critical random input and see if it crashes. functionality as possible ing, for example, when generation with program bugs by triggering ex- through a probabilistic to move to another target, path, or rect checksum and quickly error instrumentation to detect whenever ceptions, such as crashes, exploration process. Due input is a crucial aspect of fuzzing. out without triggering any complex a mutated input reaches new cover- memory corruption, or to its nature as a dynamic Orthogonally, comparing dier- behavior. e model allows input age. Program instrumentation tracks assertion failures automatically (or dereferences resulting in memory testing technique, fuzzing faces several ent fuzzing techniques requires an generation to balance the created test which areas of the code are executed, with a lile help). Furthermore, fuzz- corruption, casting to an incompat- unique challenges: understanding of the strengths of inputs according to the underlying and the coverage prole is tied to ing comes with a witness (proof of the ible type leading to type confusion, a fuzzer and the underlying statis- input protocol. e disadvantage of specic inputs. Whenever an input vulnerability) that enables developers integer overflows, or application ■ Input generation: Fuzzers generate tics to enable a fair comparison. model-based input generation is that mutation generates new coverage, it to reproduce the bug and x it. programming interface (API) con- inputs based on a mutation strat- it needs an actual model. Most input is added to the set of inputs for muta- So ware testing broadly focuses fusion. To cope with the complexity egy to explore a new state. Because Input Generation formats are not formally described tion. is approach is highly ecient on discovering and patching bugs of current programs and nd bugs, the fuzzer is aware of the program Input generation is essential to the and will require an analyst to dene due to the low-cost instrumentation during development. Unfortunately, companies such as Google, Micro- structure, it can tailor input gener- fuzzing process as every fuzzer must the intricate dependencies. but still results in broad program cov- a program is only secure if it is free so , and Apple integrate dynamic ation to the program. e under- automatically generate test cases to Mutation-based input genera- erage. Coverage-guided fuzzing is the of unwanted exceptions. Security, testing into their so ware develop- lying strategy determines how be run on the execution engine. e tion requires a set of seed inputs current de facto standard, with Amer- therefore, requires proof of the ab- ment cycle. effectively the fuzzer explores a cost of generating a single input must that trigger valid functionality in the ican fuzzy lop1 and honggfuzz2 as the sence of security violations. For exam- Fuzzing, the process of provid- given state space. A challenge for be low, following the underlying program and then leverages random most prominent implementations. ple, a bug becomes a vulnerability if ing random input to a program to input generation is finding the philosophy of fuzzing where itera- mutation to modify these seeds. Pro- ese fuzzers leverage execution feed- any aacker-controlled input reaches intentionally trigger crashes, has been balance between exploring new tions are cheap. rough input gen- viding a set of valid inputs is signi- back to tailor input generation with- a program location that allows a around since the early 1980s. A revival paths (control ow) and execut- eration, the fuzzer implicitly selects cantly easier than formally specifying out requiring the analyst to have deep security violation, such as memory of fuzzing techniques is taking place ing the same paths with dierent which parts of the tested program an input format. e input-mutation insight into the program structure. corruption. So ware security testing, as evidenced by papers presented at input (data ow). are executed. Input generation must process then constantly modifies A diculty for input generation is therefore, requires reasoning about all top-tier security conferences show- ■ Execution engine: e execution balance data-ow and control-ow these input seeds to trigger behavior nding the perfect balance between possible executions of code at once ing improvements in the techniques’ engine takes newly generated input exploration (discovering new code that researchers want to study. the need to discover new paths and to produce a witness that violates the eectiveness. e idea of fuzzing is and executes the program under areas compared to revisiting previ- Regardless of the input-mutation the need to evaluate existing paths security property. As Edsger W. Dijks- simple: execute a program in a test test with that input to detect aws. ously executed code areas with alter- strategy, fuzzers need a tness func- with dierent data. While the rst tra said in 1970: “Program testing can environment with random input and Fuzzers must distinguish between nate data) while considering what tion to assess the quality of the new increases coverage and explores new be used to show the presence of bugs, see if it crashes. e fuzzing process is benign and buggy executions. Not areas to focus on. ere are two fun- input and guide the generation of program areas, the laer explores but never to show their absence!” inherently sound but incomplete. By every bug results in an immediate damental forms of input generation: new input. A fuzzer may leverage the already covered code through the producing trial cases and observing segmentation fault, and detecting a model- and mutation-based input program structure and code coverage use of dierent data. Existing metrics

Digital Object Identifier 10.1109/MSEC.2018.2889892 whether the tested program crashes, state violation is a challenging task, generation. e rst is aware of the as tness functions. ere are three have a heavy control-ow focus as Date of publication: 20 March 2019 fuzzing produces a witness for each especially as code generally does input format while the laer is not. approaches to observing the program coverage measures how much of the

sanitizers cover undefined behavior, uninitialized memory, or type safety violations through illegal 4 Debug casts. Each sanitizer requires a cer- Input Generation Exe Coverage tain type of instrumentation, which increases the performance cost. e use of sanitizers for fuzzing, therefore, has to be carefully Tests evaluated as, on one hand, it makes (a) (b) (c) error detection more likely but, on Figure 1. Fuzzing consists of an execution engine and an input-generation process that runs executables, the other hand, it reduces fuzz- which are often instrumented with explicit memory safety checks. (a) e input-generation mechanism ing throughput. (the blue box marked “Input Generation”) may leverage existing test cases (“Tests”) and execution e main goal of the execution coverage to generate new test inputs. For each discovered crash, the fuzzer provides a witness (the engine is to conduct inputs as fast input that triggers the crash). (b) e execution engine. (c) A “bug” triggers the crash. e icon marked as possible. Several fuzzing optimi- “Coverage” indicates input that has passed through the execution engine. Some of that input may pass zations, such as fork servers, per- through the input-generation process again. Arrows indicate the direction of process. Exe: executable. sistent fuzzing, or special operating system (OS) primitives, reduce the time for each execution by adjust- program has already been explored. data are used. e challenge for this ing system parameters. Fuzzing Data- ow coverage is only measured component of the fuzzing process with a fork server executes the pro- implicitly with inputs that execute is to eciently enable the detec- gram up to a certain point and then the same paths but with dierent tion of security violations. For forks new processes at that location data values. A good input-generation example, without instrumentation, for each new input. is allows the mechanism balances the explicit only illegal pointer dereferences to execution engine to skip over ini- goal of extending coverage with the unmapped memory, control- ow tialization code that would be the implicit goal of rerunning the same transfers to nonexecutable memory, same for each execution. Persistent input paths with dierent data. division by zero, or similar viola- fuzzing allows the execution engine tions will trigger an exception. to reuse processes in a pool with Execution Engine To detect security violations new fuzzing input, resetting the Aer the fuzzer generates test cases, early, the tested program may be state between executions. Dierent it must execute them in a controlled instrumented with additional so- OS primitives for fuzzing reduce environment and detect when a ware guards. It is especially tricky the cost of process creation by, for bug is triggered. The example, simplifying the execution engine takes creation of page tables and the fuzz input, executes e main goal of the execution engine is optimizing scheduling for the program under test, to conduct inputs as fast as possible. short-lived processes. extracts runtime infor- Modern fuzzing is heav- mation, such as cover- ily optimized and focuses age, and detects crashes on eciency, measured (Figure 1). Ideally, a program would to nd security violations through by the number of bugs found per terminate whenever a aw is trig- undefined behavior for code writ- unit of time. Sometimes fuzzing gered. For example, an illegal pointer ten in system languages. Sanitiza- eciency is implicitly measured by dereference on an unmapped mem- tion analyzes and instruments the the number of crashes found per ory page results in a segmentation program during the compilation unit of time. However, crashes are fault, which terminates the program, process to detect security violations. not necessarily unique, and many allowing the executing engine to Address Sanitizer,3 the most com- crashes could point to the same detect the aw. Unfortunately, only monly used sanitizer, employs prob- bug. Disambiguating crashes to a small subset of security violations ability to detect spatial and temporal locate unique bugs is an important will result in program crashes. Buf- memory safety violations by placing but challenging task. Multiple bugs fer over ows into adjacent memory red zones around allocated memory may cause a program crash at the locations, for instance, may never objects, keeping track of allocated same location, whereas one input be detected at all or may only be memory, and checking mem- may trigger multiple bugs. A fuzzer detected later if the overwritten ory accesses. Other LLVM-based must triage crashes conservatively

10 ComputingEdge August 2019 80 IEEE Security & Privacy January/February 2019 SYSTEMS ATTACKS AND DEFENSES

sanitizers cover undefined behav- so that no true bugs are removed. Existing approaches, such as Lib- ■ Crash triaging: Heuristics cannot ior, uninitialized memory, or type Yet the triaging must not overload Fuzzer, require an analyst to prepare be the only way to measure per- safety violations through illegal the analyst with redundant crashes. a test program that calls the library formance. For example, collect- 4 Debug casts. Each sanitizer requires a cer- functions in a valid sequence to ing crashing inputs or even stack Input Generation Exe Coverage tain type of instrumentation, which Coverage Wall build up the necessary state to fuzz bucketing is insucient to iden- increases the performance cost. In addition to massive parallelism, a complex functions. tify unique bugs. Ground truth e use of sanitizers for fuzz- key advantage of fuzzing compared is needed to disambiguate crashing, therefore, has to be carefully to more heavyweight analysis tech- Evaluating Fuzzing ing inputs and correctly count Tests evaluated as, on one hand, it makes niques is its simplicity. However, In theory, evaluating fuzzing is straight- the number of discovered bugs. A (a) (b) (c) error detection more likely but, on due to this simplicity, fuzzing can forward: in a given domain, if tech- benchmark suite with ground truth Figure 1. Fuzzing consists of an execution engine and an input-generation process that runs executables, the other hand, it reduces fuzz- get stuck in local minima in front nique A nds more unique bugs than will help. which are often instrumented with explicit memory safety checks. (a) e input-generation mechanism ing throughput. of a coverage wall. When this hap- technique B, then technique A is ■ Seed justi cation: e choice of seed (the blue box marked “Input Generation”) may leverage existing test cases (“Tests”) and execution e main goal of the execution pens, continuous input generation superior to technique B. In practice, must be documented, as dierent coverage to generate new test inputs. For each discovered crash, the fuzzer provides a witness (the engine is to conduct inputs as fast will not result in either starting seeds provide input that triggers the crash). (b) e execution engine. (c) A “bug” triggers the crash. e icon marked as possible. Several fuzzing optimi- additional crashes or vastly different start- “Coverage” indicates input that has passed through the execution engine. Some of that input may pass zations, such as fork servers, per- new coverage. A com- Rerunning the same experiment with ing congurations, and through the input-generation process again. Arrows indicate the direction of process. Exe: executable. sistent fuzzing, or special operating mon approach to cir- a di erent random seed may result in not all techniques cope system (OS) primitives, reduce the cumvent the coverage equally well with dif- time for each execution by adjust- wall is to extract seed val- vastly di erent numbers of crashes, ferent seed characteris- program has already been explored. data are used. e challenge for this ing system parameters. Fuzzing ues used for compari- discovered bugs, and iterations. tics. Some mechanisms Data- ow coverage is only measured component of the fuzzing process with a fork server executes the pro- sons. ese seed values require a head start with implicitly with inputs that execute is to eciently enable the detec- gram up to a certain point and then are then used during the seeds to execute reason- the same paths but with dierent tion of security violations. For forks new processes at that location input-generation pro- able functionality, while data values. A good input-generation example, without instrumentation, for each new input. is allows the cess. Orthogonally, a developer evaluating fuzzing is very dicult others are perfectly ne to start with mechanism balances the explicit only illegal pointer dereferences to execution engine to skip over ini- can comment out hard checks, such due to the randomness of the pro- empty inputs. goal of extending coverage with the unmapped memory, control- ow tialization code that would be the as CRC32 comparisons, or checks cess and domain specialization (e.g., ■ Reasonable execution time: Fuzzing implicit goal of rerunning the same transfers to nonexecutable memory, same for each execution. Persistent for magic values. Removing these a fuzzer may only work for a certain campaigns are generally executed input paths with dierent data. division by zero, or similar viola- fuzzing allows the execution engine noncritical checks from the program type of bug or in a certain environ- over days or weeks. Comparing tions will trigger an exception. to reuse processes in a pool with requires a knowledgeable developer ment). Rerunning the same experi- different mechanisms based on Execution Engine To detect security violations new fuzzing input, resetting the to tailor fuzzing for each program. ment with a dierent random seed a few hours of execution time is Aer the fuzzer generates test cases, early, the tested program may be state between executions. Dierent Several recent extensions5–8 try may result in vastly dierent numbers not enough. A realistic evaluation, it must execute them in a controlled instrumented with additional so- OS primitives for fuzzing reduce to bypass the coverage wall by auto- of crashes, discovered bugs, and itera- therefore, must run fuzzing cam- environment and detect when a ware guards. It is especially tricky the cost of process creation by, for matically detecting when the fuzzer tions. A recent overview of the state paigns for at least 24 h. bug is triggered. The example, simplifying the gets stuck and, then, if the problem of the art9 evaluated the common execution engine takes creation of page tables and is detected, leveraging an auxil- practices of recently published fuzz- ese recommendations make the fuzz input, executes e main goal of the execution engine is optimizing scheduling for iary analysis to either produce new ing techniques. e study’s authors, fuzzing evaluation more com- the program under test, to conduct inputs as fast as possible. short-lived processes. inputs or modify the program. It is aer identifying common bench- plex. Evaluating each mechanism extracts runtime infor- Modern fuzzing is heav- essential that this (sometimes heavy- marking mistakes when comparing now takes considerable time with mation, such as cover- ily optimized and focuses weight) analysis is executed only dierent fuzzers, drew four observa- experiments running multiple days to age, and detects crashes on eciency, measured rarely, as alternating between analy- tions from their ndings: get enough statistical data for a fair (Figure 1). Ideally, a program would to nd security violations through by the number of bugs found per sis and fuzzing is costly and reduces and valid comparison. Unfortu- terminate whenever a aw is trig- undefined behavior for code writ- unit of time. Sometimes fuzzing fuzzing throughput. ■ Multiple executions: A single exe- nately, such a thorough evaluation gered. For example, an illegal pointer ten in system languages. Sanitiza- eciency is implicitly measured by Fuzzing libraries also face the cution is not enough due to the is required for a true comparison and dereference on an unmapped mem- tion analyzes and instruments the the number of crashes found per challenge of experiencing low cov- randomness in the fuzzing pro- analysis of factors leading to beer ory page results in a segmentation program during the compilation unit of time. However, crashes are erage during unguided fuzzing cam- cess. Input mutation relies on ran- fuzzing results. fault, which terminates the program, process to detect security violations. not necessarily unique, and many paigns. Programs oen call exported domness to decide, according to allowing the executing engine to Address Sanitizer,3 the most com- crashes could point to the same library functions in sequence, build- the mutation strategy, where to A Call for Future Work detect the aw. Unfortunately, only monly used sanitizer, employs prob- bug. Disambiguating crashes to ing up a complex state in the pro- mutate input and what to mutate. With the advent of coverage-guided a small subset of security violations ability to detect spatial and temporal locate unique bugs is an important cess. e library functions execute In a single run, one mechanism grey-box fuzzing,1,2 dynamic test- will result in program crashes. Buf- memory safety violations by placing but challenging task. Multiple bugs sanity checks and quickly detect an could discover more bugs simply ing has seen a renaissance. Many fer over ows into adjacent memory red zones around allocated memory may cause a program crash at the illegal or missing state. ese checks by chance. To evaluate different new techniques that improve secu- locations, for instance, may never objects, keeping track of allocated same location, whereas one input make library fuzzing challenging, as mechanisms and measure noise, rity testing have appeared. An be detected at all or may only be memory, and checking mem- may trigger multiple bugs. A fuzzer the fuzzer is not aware of the depen- we require multiple trials and sta- important advantage of fuzzing is detected later if the overwritten ory accesses. Other LLVM-based must triage crashes conservatively dencies between library functions. tistical tests. that each reported bug comes with

www.computer.org/computingedge 11 80 IEEE Security & Privacy January/February 2019 www.computer.org/security 81 SYSTEMS ATTACKS AND DEFENSES

a witness that enables the deter- References evolutionary fuzzing,” in Proc. ministic reproduction of the bug. 1. M. Zalewski, “American fuzzy lop ISOC Network and Security Sys- Sanitization, the process of instru- (AFL),” 2013. [Online]. Available: tem Symp., 2017. doi: 10.14722/ menting code with additional soft- http://lcamtuf.coredump.cx/afl ndss.2017.23404. ware guards, helps in discovering /technical_details.txt 7. H. Peng, Y. Shoshitaishvili, and M. bugs closer to their source. Over- 2. R. Swiecki, “Honggfuzz,” 2010. Payer, “T-Fuzz: Fuzzing by program all, security testing remains chal- [Online]. Available: hps://github transformation,” in Proc. 2018 IEEE lenging, especially for libraries or .com/google/honggfuzz Symp. Security and Privacy. doi: complex code, such as kernels or 3. K. Serebryany, D. Bruening, 10.1109/SP.2018.00056. large software systems. As fuzz- A. Potapenko, and D. Vyukov, 8. I. Yun, S. Lee, M. Xu, Y. Jang, and ers become more domain speci c, “AddressSanitizer: A fast address T. Kim, “QSYM: A practical con- an interesting challenge will be sanity checker,” presented at the colic execution engine tailored for to make comparisons across dif- 2012 USENIX Annual Techni- hybrid fuzzing,” presented at the ferent domains (e.g., comparing a cal Conference, Boston, MA. 27th USENIX Security Symp., Bal- grey-box kernel fuzzer for use-aer- [Online]. Available: hps://www timore, MD, 2018. free vulnerabilities with a black-box .usenix.org/conference/atc12 9. G. Klees, A. Ruef, B. Cooper, protocol fuzzer). Given the sig- /technical-sessions/presentation S. Wei, and M. Hicks, “Evaluat- nificant recent improvements in /serebryany ing fuzz testing,” in Proc. ACM fuzzing, exciting new results can be 4. Y. Jeon, P. Biswas, S. A. Carr, B. Lee, Conf. Computer and Communica- expected. Fuzzing will help make and M. Payer, “HexType: Ecient tions Security (CCS), 2018. doi: our systems more secure by nd- detection of type confusion errors 10.1145/3243734.3243804. ing bugs during the development for C++,” in Proc. 2017 ACM SIG- of code before they can cause harm SAC Conf. Computer and Communi- Mathias Payer is a security researcher during deployment. cations Security, pp. 2373–2387. doi: and an assistant professor at the Fuzzing is a hot research area 10.1145/3133956.3134062. EPFL School of Computer and with researchers striving to improve 5. N. Stephens et al., “Driller: Augment- Communication Sciences, leading input generation, reduce the impact ing fuzzing through selective symbolic the HexHive group. His research of each execution on performance, execution,” in Proc. ISOC Network focuses on protecting applica- beer detect security violations, and Security System Symp., 2016. doi: tions in the presence of vulner- and push fuzzing to new domains, 10.14722/ndss.2016.23368. abilities, with a focus on memory such as kernel fuzzing or hardware 6. S. Raway, V. Jain, A. Kumar, L. corruption and type violations. fuzzing. ese eorts bring excite- Cojocar, C. Giuffrida, and H. Contact him at mathias.payer@ ment to the eld. Bos, “VUzzer: Application-aware nebelwelt.net.

This article originally IEEE DataPort is an accessible online platform that enables researchers to easily share, access, and manage datasets in one trusted location. The platform accepts all types of appeared in Last Worddatasets, up tocontinued 2TB, and dataset om uploads p. 84 are currently free of charge. IEEE Security & Privacy, vol. 17, no. 1, 2019. Public-interest technology technology-driven journalism. there is a viable career path for isn’t new. Many organizations are There are even programs and ini- public-interest technologists. working in this area, from older or- tiatives related to public-interest ere are many barriers. A report ganizations, such as EFF and EPIC, technology inside for- profit titled “A Pivotal Moment” (hps:// to newer ones, such as Verified corporations. www.netgainpartnership.org/s/pivot Voting and Access Now. Many aca- is might all seem like a lot, almoment.pdf) includes this quote: demic classes and programs com- but it’s really not. There aren’t bine technology and public policy. enough people doing it, there While we cite individual My cybersecurity policy class at aren’t enough people who know instances of visionary leader-

the HarvardUPLOAD Kennedy DATASETS School AT isIEEE-DATAPORT.ORG it needs to be done, and there ship and successful deploy- just one example. Media start- aren’t enough places to do it. ment of technology skill for ups like The Markup are doing We need to build a world where the public interest, there was

12 ComputingEdge August 2019 82 IEEE Security & Privacy January/February 2019 PURPOSE: The IEEE Computer Society is the world’s largest EXECUTIVE COMMITTEE association of computing professionals and is the leading provider President: Cecilia Metra of technical information in the field. President-Elect: Leila De Floriani MEMBERSHIP: Members receive the monthly magazine Past President: Hironori Kasahara Computer, discounts, and opportunities to serve (all activities First VP: Forrest Shull; Second VP: Avi Mendelson; are led by volunteer members). Membership is open to all IEEE Secretary: David Lomet; Treasurer: Dimitrios Serpanos; members, affiliate society members, and others interested in the VP, Member & Geographic Activities: Yervant Zorian; computer field. VP, Professional & Educational Activities: Kunio Uchiyama; COMPUTER SOCIETY WEBSITE: www.computer.org VP, Publications: Fabrizio Lombardi; VP, Standards Activities: OMBUDSMAN: Direct unresolved complaints to Riccardo Mariani; VP, Technical & Conference Activities: [email protected]. William D. Gropp 2018–2019 IEEE Division V Director: John W. Walz CHAPTERS: Regular and student chapters worldwide provide the opportunity to interact with colleagues, hear technical experts, 2019 IEEE Division V Director Elect: Thomas M. Conte and serve the local professional community. 2019–2020 IEEE Division VIII Director: Elizabeth L. Burd AVAILABLE INFORMATION: To check membership status, report BOARD OF GOVERNORS an address change, or obtain more information on any of the Term Expiring 2019: Saurabh Bagchi, Gregory T. Byrd, following, email Customer Service at [email protected] or call David S. Ebert, Jill I. Gostin, William Gropp, Sumi Helal +1 714 821 8380 (international) or our toll-free number, Term Expiring 2020: Andy T. Chen, John D. Johnson, +1 800 272 6657 (US): Sy-Yen Kuo, David Lomet, Dimitrios Serpanos, Hayato Yamana • Membership applications • Publications catalog Term Expiring 2021: M. Brian Blake, Fred Douglis, • Draft standards and order forms Carlos E. Jimenez-Gomez, Ramalatha Marimuthu, • Technical committee list Erik Jan Marinissen, Kunio Uchiyama • Technical committee application • Chapter start-up procedures EXECUTIVE STAFF • Student scholarship information Executive Director: Melissa A. Russell •Volunteer leaders/staff directory Director, Governance & Associate Executive Director: • IEEE senior member grade application (requires 10 years Anne Marie Kelly practice and significant performance in five of those 10) Director, Finance & Accounting: Sunny Hwang Director, Information Technology & Services: Sumit Kacker PUBLICATIONS AND ACTIVITIES Director, Marketing & Sales: Michelle Tubb Director, Membership Development: Eric Berkowitz Computer: The flagship publication of the IEEE Computer Society, Computer publishes peer-reviewed technical content that covers COMPUTER SOCIETY OFFICES all aspects of computer science, computer engineering, Washington, D.C.: 2001 L St., Ste. 700, Washington, D.C. technology, and applications. 20036-4928; Phone: +1 202 371 0101; Fax: +1 202 728 9614; Periodicals: The society publishes 12 magazines, 15 transactions, Email: [email protected] and two letters. Refer to membership application or request Los Alamitos: 10662 Los Vaqueros Cir., Los Alamitos, CA 90720; information as noted above. Phone: +1 714 821 8380; Email: [email protected] Conference Proceedings & Books: Conference Publishing Asia/Pacific: Watanabe Building, 1-4-2 Minami-Aoyama, Services publishes more than 275 titles every year. Minato-ku, Tokyo 107-0062, Japan; Phone: +81 3 3408 3118; Standards Working Groups: More than 150 groups produce IEEE Fax: +81 3 3408 3553; Email: [email protected] standards used throughout the world. Technical Committees: TCs provide professional interaction in MEMBERSHIP & PUBLICATION ORDERS more than 30 technical areas and directly influence computer Phone: +1 800 678 4333; Fax: +1 714 821 4641; engineering conferences and publications. Email: [email protected] Conferences/Education: The society holds about 200 conferences each year and sponsors many educational activities, including IEEE BOARD OF DIRECTORS computing science accreditation. President & CEO: Jose M.D. Moura Certifications: The society offers three software developer President-Elect: Toshio Fukuda credentials. For more information, visit Past President: James A. Jefferies www.computer.org/certification. Secretary: Kathleen Kramer Treasurer: Joseph V. Lillie 2019 BOARD OF GOVERNORS MEETINGS Director & President, IEEE-USA: Thomas M. Coughlin (TBD) October: Teleconference Director & President, Standards Association: Robert S. Fish Director & VP, Educational Activities: Witold M. Kinsner Director & VP, Membership and Geographic Activities: Francis B. Grosz, Jr. Director & VP, Publication Services & Products: Hulya Kirkici Director & VP, Technical Activities: K.J. Ray Liu revised 23 July 2019 SECTIONCYBERTRUST TITLE

Penetration Testing in the IoT Age

Chung-Kuan Chen, Zhi-Kai Zhang, Shan-Hsin Lee, and Shiuhpyng Shieh, National Chiao Tung University

Internet of Things (IoT) objects offer new employs o ensive attack techniques services but also pose new security threats. to discover vulnerabilities, is often used to complement defensive se- Due to the heterogeneity, large number, and curity methods before IoT objects resource constraints of these objects, new are deployed. Because malicious attacks need only a single exploit to be penetration testing tools and techniques are successful, improving PT coverage is crucial. To enhance manual PT, needed to complement defensive mechanisms. security researchers use automated tools to carry out three types of specialized PT: interface testing, trans- nternet of Things (IoT) devices and services are now in- portation testing, and system testing. tegral to most daily activities. However, the IoT brings Interface testing targets interfaces that interact with not only added convenience but, by connecting more external users or devices. Major vulnerabilities can exist and more objects to the Internet, new security threats. in an application if its input validation mechanisms are IMany applications in IoT ecosystems, from smart homes not in e ect. In the Open Web Application Security Proj- to customized healthcare, contain sensitive personal in- ect (OWASP) tester guidelines for IoT applications (www formation that can become the targets of network attacks. .owasp.org/index.php/IoT_Testing_Guides), the categories Unfortunately, ensuring the security of IoT objects “insecure web interface” and “insecure network services,” is not straightforward for three major reasons. First, the among others, would be addressed by interface testing. IoT’s heterogeneous nature makes it vulnerable to many Transportation testing focuses on misuse issues and kinds of attacks. Second, heavyweight protection mecha- design aws in communication protocols and weak cryp- nisms are infeasible for resource-constrained IoT devices. tographic schemes. In the OWASP guidelines, “insu - Third, many IoT objects are deployed only once and there- cient authentication/authorization,” “lack of transport after are rarely maintained or updated. encryption/integrity veri cation,” and “privacy concerns” fall into this type of testing. PENETRATION TESTING System testing examines rmware, OSs, and system ser- Due to these challenges, penetration testing (PT), which vices for implementation aws, insecure system settings,

r4cyb.indd 82 4/16/18 10:43 AM EDITOR JEFFREY VOAS EDITORNIST; EDITOR [email protected] NAME SECTIONCYBERTRUST TITLE A liation;

and other known vulnerabilities. In a more systematic way—for example, environment with numerous net- the OWASP guidelines, “insu cient to implement adaptive, prioritized, or work services is difficult and time- security con gurability” and “inse- automutation test strategies. consuming. Because service entry cure software/ rmware” are relevant points can be dynamically generated, for system testing. Intelligent payload mutation the links between them can be com- To cope with the heterogeneity and Because IoT objects can lack complex, and loops might be produced large quantity of IoT objects, we pro- prehensive input validation mecha- across IoT objects. In addition, a dis- pose modularization of test modules nisms, extending the coverage of test patcher might be built into an IoT to scale up all three types of testing. payloads is desirable. A widely used application to manage entry points. At the same time, due to IoT devices’ method, fuzz testing, employs ran- As the dispatcher can be in either a resource limitations, intelligent ap- domly generated payloads, but this is centralized or distributed structure, a proaches are desirable for generating ine cient due to resources wasted on crawler should be able to discover as Penetration Testing test plans based on available test mod- meaningless inputs. An alternative is many entry points as possible in both ules to reduce wasted resources and to exhaustively or randomly generate types of structures to locate more test redundant e ort while extending test syntax-correct inputs. This method targets. A proof-of-concept vulnerabil- in the IoT Age coverage. provides better test coverage but is still ity scanner that does this, VulScan, inefficient, as the space of syntax- has been developed to complement INTERFACE TESTING correct inputs is usually large. manual PT. Chung-Kuan Chen, Zhi-Kai Zhang, Shan-Hsin Lee, and Shiuhpyng Shieh, Many user-facing IoT objects have National Chiao Tung University web-based interfaces, and these can have various vulnerabilities. Among Internet of Things (IoT) objects offer new the most common is input validation Modularizing interface testing would make employs o ensive attack techniques failure. Unlike traditional web inter- it easier to create testing tools for speciﬁ c services but also pose new security threats. to discover vulnerabilities, is often faces, which are linked to operations vulnerabilities and install them on demand. used to complement defensive se- closely coupled with data manipula- Due to the heterogeneity, large number, and curity methods before IoT objects tion, IoT object interfaces can also be resource constraints of these objects, new are deployed. Because malicious at- linked to code-oriented operations Intelligently mutating known pay- TRANSPORTATION TESTING tacks need only a single exploit to be such as controlling system programs. loads is a compromise between man- Transportation testing is performed penetration testing tools and techniques are successful, improving PT coverage Code-oriented attacks such as com- ual testing and exhaustive/random both on the network infrastructure is crucial. To enhance manual PT, mand injection and code injection testing. Combining existing evasion interconnecting IoT objects as well as needed to complement defensive mechanisms. security researchers use automated could be even more severe than data- techniques provides greater ability to the associated cryptographic schemes tools to carry out three types of spe- oriented attacks. Improving input circumvent validation mechanisms. and communication protocols used to cialized PT: interface testing, trans- validation testing is thus critical for In this case, con icting or overlapping protect messages. nternet of Things (IoT) devices and services are now in- portation testing, and system testing. the IoT. Although testing web-based techniques should be manipulated tegral to most daily activities. However, the IoT brings Interface testing targets interfaces that interact with interfaces is our focus, the same mod- carefully to prune unnecessary test New network infrastructures not only added convenience but, by connecting more external users or devices. Major vulnerabilities can exist ularization and intelligence mecha- cases. On the other hand, converting Messages between IoT objects traverse and more objects to the Internet, new security threats. in an application if its input validation mechanisms are nisms described below can be applied payloads to syntactically or seman- heterogeneous networks such as TCP/ IMany applications in IoT ecosystems, from smart homes not in e ect. In the Open Web Application Security Proj- to other types of IoT applications. tically equivalent payloads is worthy IP, Zigbee, and LoWPAN. To allow to customized healthcare, contain sensitive personal in- ect (OWASP) tester guidelines for IoT applications (www of further investigation. Syntactic more e cient object communica- formation that can become the targets of network attacks. .owasp.org/index.php/IoT_Testing_Guides), the categories Modularized design mutation generates payloads with tion, new infrastructures such as FIA Unfortunately, ensuring the security of IoT objects “insecure web interface” and “insecure network services,” Testers employ various techniques for slight changes at the syntax level. For (www.nets- a.net), HUBNGI (www is not straightforward for three major reasons. First, the among others, would be addressed by interface testing. di erent input validation vulnerabili- example, SQL code “‘or 1 = 1” can .hubngi.eu), and PNS have been pro- IoT’s heterogeneous nature makes it vulnerable to many Transportation testing focuses on misuse issues and ties. However, these methods are con- be mutated to “‘|| 1 = 1”. Semantic posed. New PT tools are needed to test kinds of attacks. Second, heavyweight protection mecha- design aws in communication protocols and weak cryp- ceptually similar in that they all crawl mutation converts the whole payload these infrastructures, the protocols, nisms are infeasible for resource-constrained IoT devices. tographic schemes. In the OWASP guidelines, “insu - to the entry points and submit the test to functional equivalent ones. For in- and the gateways or converters be- Third, many IoT objects are deployed only once and there- cient authentication/authorization,” “lack of transport payload. Modularizing interface test- stance, “id = 1 or 1” is semantically tween the infrastructures and proto- after are rarely maintained or updated. encryption/integrity veri cation,” and “privacy con- ing would make it easier to create test- equivalent to “id = id xor 0”. cols. Because network heterogeneity cerns” fall into this type of testing. ing tools for speci c vulnerabilities is a key issue in IoT communication, PENETRATION TESTING System testing examines rmware, OSs, and system ser- and install them on demand. More- Intelligent entry-point crawling transportation testing should be mod- Due to these challenges, penetration testing (PT), which vices for implementation aws, insecure system settings, over, algorithms could be developed in Entry-point discovery in an IoT ularized to provide better exibility.

r4cyb.indd 82 4/16/18 10:43 AM r4cyb.indd 83 4/16/18 10:43 AM CYBERTRUST

Cryptographic issues In conventional computing envi- infeasible. An alternative approach is In general, the cryptographic algo- ronments the x/x instruction-set virtual machine introspection (VMI), rithms that protect network communi- architecture (ISA) dominates, but which monitors VM execution in the cation are believed to be secure due to other ISAs such as ARM, MIPS, and hypervisor outside the VM., Because theoretical proofs. When vulnerabil- PPC are also used in the IoT. OSs vary VMI doesn’t modify the guest OS, IoT ities are discovered, they’re generally among IoT objects as well, with general- objects are easier to deploy. Through attributable to misuse, implementa- purpose OSs such as Linux, Windows, VMI, the emulator’s out-of-box moni- tion failures, and bad protocol design. and Android often customized. The di- toring, memory forensics, and debug- However, resource-constrained IoT versity of IoT objects makes automated ging features can be developed more objects can’t aord heavyweight cryp- reverse-engineering challenging. easily to enable both manual and auto- tographic mechanisms. Moreover, matic PT. messages between devices usually are Encapsulation well formatted and lack entropy. The To mitigate the impacts of system Intelligent grey-box testing combination of these factors could diversity, encapsulation can enable As the boundary of grey-box testing is make dierential cryptanalysis or sta- cross-platform analysis. Encapsulation more obscure than white- and black- tistical attacks possible. involves using an abstract language box testing, a systematic division of Trusted platform modules (TPMs) such as LLVM (http://llvm.org) or VEX testing phases enables the develop- enable new applications but also raise (http://valgrind.org) to create an in- ment of future testing techniques. In- new threats. For example, the ROCA termediate representation (IR) of dif- telligent grey-box PT can be divided vulnerability is caused by a weak ferent machine languages to emulate into four phases: vulnerability model construction, execution path exploration, vulnerability path searching, and vulnerability path verication. To discover vulnerabilities, the model of ab- To mitigate the impacts of system diversity, normal behaviors is rst constructed. encapsulation can enable cross-platform analysis. Next, control ows are analyzed to nd each execution path. The vulnerability risk for each path is then estimated us- prime-number generator in the RSA li- ISAs. Hardware-assisted emulators ing information from the IR and VMI brary within TPMs. This vulnerability are used to test programs running on to prioritize testing order. Once the aects many vendors including Micro- specic ISAs, but software-based emu- path with highest risk is identied, the soft, Google, and HP. Another exam- lators such as QEMU (www.qemu.org) symbolic execution resolves inputs ple is KRACK attacks, which exploit and Bochs (http://bochs.sourceforge to the path. During the nal phase, if a aw in Wi-Fi’s WPA encryption and .net) can leverage multiple ISAs and the resolved input is available, a veri- aects all major software platforms. are more suitable for IoT objects. er can monitor the program with the As cryptographic operations are rarely Another method for building an IR input to check whether the vulnera- computed in cleartext, developing PT is symbolic execution, which trans- bility model can be satised. Using methods to discover such vulnerabili- lates a program to mathematical con- this systematic approach, intelligent ties in the IoT is challenging. straints and evaluates whether certain grey-box PT can discover system-level properties can be satised. With these vulnerabilities. SYSTEM TESTING constraints, developing an intelligent In contrast to interface testing, which PT method with a more formalized focuses on commonly used technol- foundation is possible. o cope with the heterogeneity, ogies such as web interfaces, propri- large number, and resource etary programs are the main targets Virtual machine introspection constraints of IoT objects, PT of system testing. Without having While symbolic execution mostly deals Ttools and techniques should apply the knowledge of such systems, testers of- with per-process information, system- principles of modularization and in- ten resort to black-box methods, such wide runtime information is also im- telligence. Modularization provides as fuzz testing. Given the large number portant for PT. However, runtime anal- the exibility to test various targets, of IoT objects to be tested, exhausting ysis tools might not be available for IoT and intelligence enlarges test coverage all test cases is infeasible. It’s therefore objects. Due to resource constraints, ob- and improves accuracy. In interface helpful generating test cases through ject diversity, and proprietary architec- testing, input validation mechanisms automatic reverse-engineering, what tures, developing debugging and anal- should be tested using an intelli- is termed grey-box PT. ysis tools for dierent objects is usually gent mutation engine and entry-point

16 ComputingEdge August 2019 84 COMPUTER WWW.COMPUTER.ORG/COMPUTER

r4cyb.indd 84 4/16/18 10:43 AM CYBERTRUST

Cryptographic issues In conventional computing envi- infeasible. An alternative approach is discovery automated. Transportation Security: Threats, Countermeasures, In general, the cryptographic algo- ronments the x/x instruction-set virtual machine introspection (VMI), testing must address the problem of and Pitfalls,” Computer, vol. , no. , CHUNG KUAN CHEN is a PhD rithms that protect network communi- architecture (ISA) dominates, but which monitors VM execution in the messages between IoT objects travers- , pp. –. candidate in the Department of cation are believed to be secure due to other ISAs such as ARM, MIPS, and hypervisor outside the VM., Because ing heterogeneous networks. To deal . Z.-K. Zhang et al., “Identifying and Computer Science at National Chiao theoretical proofs. When vulnerabil- PPC are also used in the IoT. OSs vary VMI doesn’t modify the guest OS, IoT with emerging IoT network infra- Authenticating IoT Objects in a Natu- Tung University (NCTU). Contact him ities are discovered, they’re generally among IoT objects as well, with general- objects are easier to deploy. Through structures, PT tools should be com- ral Context,” Computer, vol. , no. , at [email protected]. attributable to misuse, implementa- purpose OSs such as Linux, Windows, VMI, the emulator’s out-of-box moni- patible with the overlay networks. , pp. –. tion failures, and bad protocol design. and Android often customized. The di- toring, memory forensics, and debug- Cryptographic misuse issues and im- . M. Nemec et al., “The Return of ZHI KAI ZHANG is a PhD candidate However, resource-constrained IoT versity of IoT objects makes automated ging features can be developed more plementation aws must also be con- Coppersmith’s Attack: Practical in the Department of Computer objects can’t aord heavyweight cryp- reverse-engineering challenging. easily to enable both manual and auto- sidered. In system testing, the chal- Factorization of Widely Used RSA Science at NCTU. Contact him at tographic mechanisms. Moreover, matic PT. lenge is IoT objects with various ISAs Moduli,” Proc. ACM SIGSAC [email protected]. messages between devices usually are Encapsulation and OSs. If encapsulation and related Conf. Computer and Communications well formatted and lack entropy. The To mitigate the impacts of system Intelligent grey-box testing translation modules are available, Security (CCS ), , pp. –. SHAN HSIN LEE is a PhD student combination of these factors could diversity, encapsulation can enable As the boundary of grey-box testing is cross-platform analysis becomes fea- . M. Vanhoef and F. Piessens, “Key in the Department of Computer make dierential cryptanalysis or sta- cross-platform analysis. Encapsulation more obscure than white- and black- sible. VMI and symbolic execution can Reinstallation Attacks: Forcing Science at NCTU. Contact him at tistical attacks possible. involves using an abstract language box testing, a systematic division of be applied on top of encapsulation. In Nonce Reuse in WPA,” Proc. [email protected]. Trusted platform modules (TPMs) such as LLVM (http://llvm.org) or VEX testing phases enables the develop- this way, intelligent analysis methods ACM SIGSAC Conf. Computer and enable new applications but also raise (http://valgrind.org) to create an in- ment of future testing techniques. In- can be used to discover vulnerabilities Communications Security (CCS ), SHIUHPYNG WINSTON SHIEH is a new threats. For example, the ROCA termediate representation (IR) of dif- telligent grey-box PT can be divided in variant platforms. , pp. –. university chair professor and past vulnerability is caused by a weak ferent machine languages to emulate into four phases: vulnerability model . K. Nance, M. Bishop, and B. Hay, chair of the Department of Computer construction, execution path explora- REFERENCES “Virtual Machine Introspection: Science at NCTU. Contact him at tion, vulnerability path searching, and . Z.-K. Zhang et al., “IoT Security: Observation or Interference?,” IEEE [email protected]. vulnerability path verication. To dis- Ongoing Challenges and Research Security & Privacy, vol. , no. , , cover vulnerabilities, the model of ab- Opportunities,” Proc. IEEE th Int’l pp. –. To mitigate the impacts of system diversity, normal behaviors is rst constructed. Conf. Service-Oriented Computing . C.-W. Wang et al., “Cloudebug: A Read your subscriptions encapsulation can enable cross-platform analysis. Next, control ows are analyzed to nd and Applications (SOCA ), , Programmable Online Malware Test- through the myCS This article originallypublications appeared portal at in each execution path. The vulnerability pp. –. bed,” Computer, vol. , no. , , Computer,http://mycs.computer.org vol. 51, no. 4, 2018. risk for each path is then estimated us- . H.-C. Huang et al., “Web Application pp. –. prime-number generator in the RSA li- ISAs. Hardware-assisted emulators ing information from the IR and VMI brary within TPMs. This vulnerability are used to test programs running on to prioritize testing order. Once the aects many vendors including Micro- specic ISAs, but software-based emu- path with highest risk is identied, the soft, Google, and HP. Another exam- lators such as QEMU (www.qemu.org) symbolic execution resolves inputs ple is KRACK attacks, which exploit and Bochs (http://bochs.sourceforge to the path. During the nal phase, if a aw in Wi-Fi’s WPA encryption and .net) can leverage multiple ISAs and the resolved input is available, a veri- aects all major software platforms. are more suitable for IoT objects. er can monitor the program with the As cryptographic operations are rarely Another method for building an IR input to check whether the vulnera- computed in cleartext, developing PT is symbolic execution, which trans- bility model can be satised. Using methods to discover such vulnerabili- lates a program to mathematical con- this systematic approach, intelligent ties in the IoT is challenging. straints and evaluates whether certain grey-box PT can discover system-level properties can be satised. With these vulnerabilities. IEEE Security & Privacy magazine provides articles with SYSTEM TESTING constraints, developing an intelligent both a practical and research bent by the top thinkers in In contrast to interface testing, which PT method with a more formalized the ﬁ eld. focuses on commonly used technol- foundation is possible. o cope with the heterogeneity, ogies such as web interfaces, propri- large number, and resource ✔ Stay current on the latest security tools and theories etary programs are the main targets Virtual machine introspection constraints of IoT objects, PT and gain invaluable practical and research knowledge, of system testing. Without having While symbolic execution mostly deals Ttools and techniques should apply the ✔ Learn more about the latest techniques and cutting- knowledge of such systems, testers of- with per-process information, system- principles of modularization and in- edge technology, and ten resort to black-box methods, such wide runtime information is also im- telligence. Modularization provides ✔ as fuzz testing. Given the large number portant for PT. However, runtime anal- the exibility to test various targets, Discover case studies, tutorials, columns, and of IoT objects to be tested, exhausting ysis tools might not be available for IoT and intelligence enlarges test coverage in-depth interviews and podcasts for the information all test cases is infeasible. It’s therefore objects. Due to resource constraints, ob- and improves accuracy. In interface security industry. helpful generating test cases through ject diversity, and proprietary architec- testing, input validation mechanisms automatic reverse-engineering, what tures, developing debugging and anal- should be tested using an intelli- is termed grey-box PT. ysis tools for dierent objects is usually gent mutation engine and entry-point

www.computer.org/computingedge 17 84 COMPUTER WWW.COMPUTER.ORG/COMPUTER APRIL 2018 85 www.computer.org/subscribe r4cyb.indd 84 4/16/18 10:43 AM r4cyb.indd 85 4/16/18 10:43 AM 21mitp01-laplante-2883858.3d (Style 5) 05-06-2019 16:50

Internet of Things

IoT Safety: State of the Art

Janusz Zalewski Florida Gulf Coast University

Editors: Phillip A. Laplante, Penn State ([email protected]), Ben Amaba, IBM ([email protected])

& AMONG MULTIPLE ISSUES or concerns with BASIC NOTIONS respect to the Internet of Things (IoT), it is defi- Technically, in computing systems, safety as a nitely security that is given the most attention. system property is strictly related to security. Users, developers, managers, and other stake- They are complementary. While safety relates to holders are concerned that the heterogeneity ensuring that the computing device does not and complexity of this technology, essentially cause harm to the environment, security relates composed of systems of systems, may open the to ensuring that the computing device is not hurt door to security breaches on an unprecedented from the environment. This is illustrated in scale. There is, however, another important Figure 1, where a smart device may impact the system property, which is not very often brought environment negatively due to malfunctioning, up as an imminent concern, but is equally impor- which is a safety concern, but may also be affected tant. This is device safety, the violation of which negatively and experience harm due to the environ- may cause severe harm to the environment in ment acting maliciously, which is a security con- which the device operates. cern. The environment is understood here very While normally, in the use of technology, broadly and means everything with which the com- concerns are placed on actions the device takes to puting system or smart device may interface but is accomplish its desired functions accurately, as normally limited to one or more of the following: specified, safety is a concern related to ensuring network, a human operator, a database, or a plant that the device causes no damage or harm. Just (technical object with which it interacts). like with a regular TV set, which must meet In practice, IoT security violations occur specific regulations that prevent it from catching as breaches due to threats (attacks) exploi- fire, there are multiple instances of IoT applica- ting vulnerabilities in hardware or software. tions, where the environment can suffer due to IoT safety violations, on the other hand, usually unintended misbehavior of the device. Examples occur as a result of computer failures due of Internet-enabled devices that may lead to such to hardware or software faults activated by consequences include wearable medical device hazards. The terminology for safety and security causing harm to a patient, a vehicle causing an has a lot of parallel concepts and mutually accident due to the malfunction of software, a ther- matching terms,1 which are illustrated in Table 1. mostat in a smart home causing overheating, etc. In addition, the two properties can hardly be viewed in isolation, because poor security can have negative impact on safety, and vice Digital Object Identifier 10.1109/MITP.2018.2883858 versa, safety violations may negatively affect Date of current version 26 February 2019. security.

1520-9202 ß 2018 IEEE Published by the IEEE Computer Society IT Professional

Internet of Things

IoT Safety: State of the Art

Janusz Zalewski Florida Gulf Coast University

Figure 1. Analogy between safety and security properties. Editors: Phillip A. Laplante, Penn State ([email protected]), Ben Amaba, IBM ([email protected])

& AMONG MULTIPLE ISSUES or concerns with BASIC NOTIONS Referring to IoT safety, one can list a number households, with smart homes, where prop- respect to the Internet of Things (IoT), it is defi- Technically, in computing systems, safety as a of industries, where IoT device safety is crucial, erly operating home appliances, air condi- nitely security that is given the most attention. system property is strictly related to security. but rarely is a different perspective brought up: tioners, door locks, animal feeders, and Users, developers, managers, and other stake- They are complementary. While safety relates to how a cross section of our society is influenced others make up crucial components of us holders are concerned that the heterogeneity ensuring that the computing device does not by safety concerns, which involves the following feeling safe; and complexity of this technology, essentially cause harm to the environment, security relates (see Figure 2): smart cities, where street vehicles, traffic nav- composed of systems of systems, may open the to ensuring that the computing device is not hurt igation systems, parking spaces, building individual level, with wearables, such as a door to security breaches on an unprecedented from the environment. This is illustrated in automation, power and energy distribution scale. There is, however, another important Figure 1, where a smart device may impact the smart watch, or medical devices, such as may all adversely impact safety, if improperly system property, which is not very often brought environment negatively due to malfunctioning, pacemakers or insulin pumps; used or controlled as a part of IoT; up as an imminent concern, but is equally impor- which is a safety concern, but may also be affected tant. This is device safety, the violation of which negatively and experience harm due to the environ- may cause severe harm to the environment in ment acting maliciously, which is a security con- Table 1. Illustration of a dualism between safety and security concepts. which the device operates. cern. The environment is understood here very Security Safety broadly and means everything with which the com- While normally, in the use of technology, Conse- Conse- Concept Definition Concept Definition concerns are placed on actions the device takes to puting system or smart device may interface but is quences quences accomplish its desired functions accurately, as normally limited to one or more of the following: Any circumstance or event with Intrinsic property or the potential to adversely impact specified, safety is a concern related to ensuring network, a human operator, a database, or a plant Exploits condition that has the an information system through Activates a that the device causes no damage or harm. Just (technical object with which it interacts). Threat vulnera- Hazard potential to cause unauthorized access, destruction, fault bilities harm or damage. like with a regular TV set, which must meet In practice, IoT security violations occur disclosure, modification of data, [SSEV] specific regulations that prevent it from catching as breaches due to threats (attacks) exploi- and/or denial of service. [CNSS] fire, there are multiple instances of IoT applica- ting vulnerabilities in hardware or software. Weakness in an information system, Manifestation of an Vulnera- system security procedures, inter- Results in a Results in tions, where the environment can suffer due to IoT safety violations, on the other hand, usually Fault error in software. bility nal controls, or implementation that breach a failure unintended misbehavior of the device. Examples occur as a result of computer failures due [SSEV] could be exploited. [CNSS] of Internet-enabled devices that may lead to such to hardware or software faults activated by Termination of the consequences include wearable medical device hazards. The terminology for safety and security An event in which a system or ability of a system to causing harm to a patient, a vehicle causing an has a lot of parallel concepts and mutually system component is compromised, perform a required Leads to Leads to matching terms,1 which are illustrated in Table 1. Breach so its required functions within Failure function or its inability harm or accident due to the malfunction of software, a ther- losses mostat in a smart home causing overheating, etc. In addition, the two properties can hardly be specified limits are impaired. to perform within damage [Author] previously specified viewed in isolation, because poor security limits. [SSEV] can have negative impact on safety, and vice SSEV – Software and Systems Engineering Vocabulary – http://computer.org/sevocab versa, safety violations may negatively affect Digital Object Identifier 10.1109/MITP.2018.2883858 CNSS – Committee on National Security Systems Glossary Date of current version 26 February 2019. security. – https://rmf.org/wp-content/uploads/2017/10/CNSSI-4009.pdf

1520-9202 ß 2018 IEEE Published by the IEEE Computer Society IT Professional January/February 2019 16 www.computer.org/computingedge 17 19 21mitp01-laplante-2883858.3d (Style 5) 05-06-2019 16:50

Internet of Things

Figure 2. Layered perspective on IoT safety concerns.

industrial world, in which factories, agricul- Overall, due to the massive amount of appli- tural fields, critical infrastructure (water and cations that are Internet-enabled, unintended waste processing plants, gas and oil pipelines) connections arise, new uses emerge, and vendors make us all vulnerable if unsafe. are losing control over how their products are utilized and may not even anticipate some of the Each upper layer is immersed into the lower purposes the devices serve. This situation, if not layers, forming a consistent view of IoT safety properly addressed, may have a dramatic impact concerns. on the entire ecosystem and endanger life safety As viewed from Figure 2, computer and and consumer safety, leading to sustainability software safety is of paramount importance at concerns. Therefore, the issue of developing all layers and has been addressed by various safety standards for various industries to include industries before, resulting in multiple guid- IoT has to be revisited. elines, both industry specific,2 and general international standards.3 With the emergence of IoT, however, the situation changes drasti- STATE OF THE ART cally, because these standards may no longer There is an advocacy for IoT safety from pro- be 100% applicable. They have been devel- fessional communities. Cerf et al., discussing4 oped mostly for a single layer, say, industrial the shared governance of the Internet and the operation, rarely taking into account cross IoT, state that “user’s safety must be the first pri- passing to other layers. Nowadays, because ority for all hardware and software providers” IoT has emerged, there are other “players” in and focus the discussion on “how to address the picture, which have to be accounted for safety issues that become much more prominent in each standard. with the spread of Internet-enabled physical

IT Professional 20 18 ComputingEdge August 2019 21mitp01-laplante-2883858.3d (Style 5) 05-06-2019 16:50 21mitp01-laplante-2883858.3d (Style 5) 05-06-2019 16:50

Internet of Things

Table 2. Sample structure of the proposed IoT Safety Case (based on IPA).9

IoT safety case #001 Name [optional] Automotive vehicles (cars) and homes The driver of a car uses the on-board voice-operated system [...] to control the lighting, thermostat, and security systems in the home, such as opening/closing the garage door, turning on/off the lighting of Description Scenario the home’s entrance, and turning on/off the home security system. The person relaxing on a couch at home uses the home’s cloud-based voice recognition service to start/stop the car’s engine, lock/unlock the car’s door, and check its fuel gauge.

Negative consequences See Reference

Potential countermeasures See Reference

References See9

environments.” During the IEEE Experts Technol- electronic systems as a whole.” Consequently, the ogy and Policy Forum,5 Robert Martin of Mitre OMG document defines the Dependability Assur- said “We need to make sure we don’t fall prey to ance Framework (DAF) as a standard specification calling this end-to-end security, when really we to assess and assure the dependability of SSCD’s. want to talk about end-to-end security and safe- The DAF approach and their document look very ty.” Further: “For the IoT, safety needs to be con- promising on its face, but the life will show how sidered along with privacy, the performance practical is this to apply. issues, reliability, resilience, and, of course, the At the government level, the Consumer Prod- security of these systems.” uct Safety Commission (CPSC) issued in March Figure 2. Layered perspective on IoT safety concerns. Baba et al. present briefly a similar view in 2018 a Notice of Public Hearing and Request for their report,6 published by the Internet Engineer- Written Comments on The IoT on Consumer ing Task Force: “Recognition of the importance of Product Hazards, to which a wide response

industrial world, in which factories, agricul- Overall, due to the massive amount of appli- information security has grown in step with the has been received. The Center for Democracy tural fields, critical infrastructure (water and cations that are Internet-enabled, unintended rising use of the Internet. Closer examination and Technology (CDT) provided an extensive waste processing plants, gas and oil pipelines) connections arise, new uses emerge, and vendors reveals that the IoT era may see a new direct comment,8 in which it outlined five Case Studies make us all vulnerable if unsafe. are losing control over how their products are physical threat to users. [...]. These kinds of on IoT Safety, including smart: thermostats, TV’s, utilized and may not even anticipate some of the scenarios may occur without identity fraud, hack- lights, locks, and speakers, arguing the following: Each upper layer is immersed into the lower purposes the devices serve. This situation, if not ing, and other means of compromising informa- “As these examples demonstrate, consumer IoT layers, forming a consistent view of IoT safety properly addressed, may have a dramatic impact tion security. Therefore, [...] this issue shall be devices have the potential to introduce physical concerns. on the entire ecosystem and endanger life safety referred to as “IoT Safety” to distinguish it from harms that are not present in other products.” As viewed from Figure 2, computer and and consumer safety, leading to sustainability Information Security.” Further, the CDT letter recommended five specific software safety is of paramount importance at concerns. Therefore, the issue of developing Partially in response to such concerns, the actions to be taken to improve IoT product safety. all layers and has been addressed by various safety standards for various industries to include Object Management Group (OMG) defined7 On the international arena, similar activi- industries before, resulting in multiple guid- IoT has to be revisited. Safety-Sensitive Consumer Device (SSCD) as a ties are observed. The Japanese Information- elines, both industry specific,2 and general category of industrial products used by consumer technology Promotion Agency (IPA) issued international standards.3 With the emergence users, including automobiles, service robots, guidelines on IoT safety/security development. of IoT, however, the situation changes drasti- STATE OF THE ART medical devices and clinical systems, and smart The most recent one9 acknowledges that “an cally, because these standards may no longer There is an advocacy for IoT safety from pro- houses. They argue the following: “Taking the increasing number of corporations are placing be 100% applicable. They have been devel- fessional communities. Cerf et al., discussing4 future of electronics systems into consideration, the creation of value through interconnectivity oped mostly for a single layer, say, industrial the shared governance of the Internet and the each electronics system is going to be one of the of IoT devices and relevant systems [...] as one operation, rarely taking into account cross IoT, state that “user’s safety must be the first pri- terminals of IoT and will be expected to play a sig- of their major business strategies.” Although the passing to other layers. Nowadays, because ority for all hardware and software providers” nificant role as a part of smart city. This consider- approach advocated in the IPA report is a little IoT has emerged, there are other “players” in and focus the discussion on “how to address ation indicates that the safety of electronics confusing, since it relies on providing high reli- the picture, which have to be accounted for safety issues that become much more prominent systems cannot be achieved alone, but have to be ability rather than safety or security, a detailed in each standard. with the spread of Internet-enabled physical achieved together with other electrical and discussion of risk analysis for five IoT use cases

IT Professional January/February 2019 18 www.computer.org/computingedge 19 21 21mitp01-laplante-2883858.3d (Style 5) 05-06-2019 16:50

Internet of Things

is very valuable, as it involves, among other 2. DO-178C, Software Considerations in Airborne things, structural analysis of IoT components, Systems and Equipment Certification. RTCA SC-205, expected threats/damage and main factors or Jan. 2012. issues, and possible countermeasures. A more 3. Functional Safety of Electrical/Electronic/Programmable extensive list of case studies like this could form Electronic Safety-Related Systems. IEC 61508:2010, Int. a database similar to Common Vulnerabilities Electrotechn. Comm., Geneva, Switzerland, 2010. and Exposures (CVE) maintained by Mitre,10 4. V. G. Cerf et al., “IoT safety and security as shared which is a directory of entries—each containing responsibility,” J. Bus. Inform., vol. 1, no. 36, pp. 7–19, an identification number, a description, and at 2016. least one public reference—for publicly known 5. IEEE Experts Technology and Policy (ETAP) Forum on cyber security vulnerabilities. Each entry in this Internet Governance, Cybersecurity and Privacy, new safety list, tentatively named IoT safety Washington, DC, USA, Feb. 5, 2016. cases, could have the data fields for safety 6. H. Baba et al., Problems in and among Industries for concerns as shown in Table 2. the Prompt Realization of IoT and Safety Considerations. Internet Draft, IETF, Nov. 15, 2018. CONCLUSION [Online]. Available: https://tools.ietf.org/id/draft-baba- There is no doubt that safety of IoT devices is a iot-problems-06.html critical issue for the society and must be taken 7. Dependability Assurance Framework for Safety- into account in the design processes to develop Sensitive Consumer Devices (DAF). Ver. 1.0. Object the safe products. It appears that the industry and Management Group, Feb. 2016. the government understand the challenges com- 8. J. L. Hall et al., Response to Docket No. CPSC- ing with the advent of IoT and are taking steps to 2018-0007—The Internet of Things and Consumer improve the design and operational principles in Product Hazards. A Letter to the CPSC. Jun. 15, respective processes. This note presented a pro- 2018. fessional perspective on IoT safety concerns and 9. Guidance for Practice Regarding IoT Safety/Security proposed establishing the IoT safety cases data- Development Guidelines, Software Reliability base. Two important issues have not been dis- Enhancement Center, Information-Technology cussed here, which is risk assessment, including Promotion Agency, Tokyo, Japan, Dec. 2017. the legal aspects, and required certification. 10. Common Vulnerabilities and Exposures Database. [Online]. Available: https://cve.mitre.org/

ACKNOWLEDGMENTS Janusz Zalewski is a professor of software engi- Work presented in this article has been done in neering and computer science at Florida Gulf part during the author’s summer fellowships at Coast University. He has extensive experience the Air Force Research Lab in Rome, New York. in the design and development of real-time safety critical systems. His recent research & REFERENCES interests include security of cyberphysical sys- 1. A. Kornecki and J. Zalewski, “Aviation software: Safety tems and IoT. He is currently the secretary of and security,” in Wiley Encyclopedia of Electrical and the IEEE P1876 WG Networked Smart Learning Electronics Engineering, J. G. Webster, Ed. New York, Objects for Online Laboratories. Contact him at NY, USA: Wiley, 2015. [email protected].

This article originally appeared in IT Professional, vol. 21, no. 1, 2019

IT Professional 22 20 ComputingEdge August 2019 Social Impact Editor: Heather M. Patterson n Intel Labs n [email protected]

Physical Computing’s Connected and Shape-Changing Future

Heather M. Patterson, Intel Labs

echnologies have a way of unex- technology. However, new oppor- in which complex interactivities will T pectedly upending established tunities also bring new challenges. be designed into even the most ordi- social practices, often at a pace far out- It behooves us to examine how these nary “stuff”—from faucets to furni- stripping the ability of a given society new objects impact humans so that we ture to walls themselves3—how will to absorb or process disruptions with can thoughtfully engineer systems that we go about determining an informa- awareness. From stirrups to gunpowder are mindful of our social practices— tion object’s utility, intended use, or to magnets, history teaches that even sidestepping potentially troubling capacity?4 How will we discover what seemingly mundane objects have the outcomes and creating technologies knowledge is embedded in a particu- capacity to change everything.1 worth having. lar object, as well as the limits of that Physical computing has become an knowledge? More importantly, how umbrella term whose practitioners NOTEWORTHY TRANSITIONS can technological designs be adapted draw theoretical and practical inspira- At this point in time, several transi- to anticipate and meet the needs of tion from research in a cluster of asso- tions associated with physical com- their human users, and not the other ciated research domains, including puting stand out as being particular way around? tangible interactions, human-material worthy of further scrutiny. I will elab- interactions, shape-changing inter- orate on two. From Contextual to Universal faces, organic user interfaces, interac- The second transition concerns an tive materiality, and material ecology. acceleration of merging social con- For many, a common goal is to bridge How will we discover what texts brought about by novel data analog and physical worlds by infus- knowledge is embedded in a ows that connect and enliven these ing objects with compute and sense- new informational objects. By selec- making capabilities, in some cases particular object, as well as tively embedding objects with comput- enabling changes in a material’s physi- the limits of that knowledge? ing technologies, we enable new forms cal characteristics. The most intriguing of data collection, analysis, and dis- implementations also strip away lay- tribution, eroding natural data dams ers of abstraction and allow people to From Material to Informational that have, up until now, shaped social interact directly with objects with fewer The rst concerns the shift from the conceptions of privacy. What might intermediate interfaces, potentially material to the informational and be the consequences of unexpected reducing interaction friction, improv- presents questions about how interac- data tides, eddies, and oods? If the ing legibility, and fulfilling unmet tions between humans, objects, and future is one in which our devices are human needs (see the sidebar for a list physical spaces will change over time. watching us, listening to us, and even of potential applications). From shaping the way that we con g- physically recon guring themselves The transformation of material ure space to guiding the daily ows to enhance our experiences, are we objects into intelligent, information- of activities, our built environments still able to truly do, feel, be, share, rich ones holds great promise for fruit- (and the objects within them) orga- and withhold portions of ourselves ful partnerships between humans and nize our lives.2 However, in a world at will?

PHYSICAL COMPUTING APPLICATION AREAS different users, uses, and contexts,”6 hat does the future hold? Virtual, in-air touchpad interfaces that enable real- are we moving toward a reality that W time sensing of arm, hand, and nger positions could support surgical training has the appearance of being concrete 1 by tracking ne hand movements. Deformable interfaces that mimic mechanical and unchanging, but that is actually far properties of anatomical materials, water, and clay could help medical practitio- 7 ners distinguish between tumor types, or assist geologists in modeling tsunami or less perceptible or discoverable? When earthquake outcomes.2 Screens that curve inward, remain at, or skitter away in presented with new materialities, will response to the presence of authorized (or unauthorized) persons could provide bet- we be more able to super cially engage 3 ter privacy, security, and personalization. Water faucets that narrow their aperture with objects but less able to understand or bend away from users could provide unobtrusive nudges to conserve in periods of higher-than-normal water usage.3 Shape-changing tablets with co-located 3D what they really do, what they really graphics could simulate wave frequency and wind strength.4 Tactile representations know, and who they really talk to? And of navigable spaces (and more),5 such as insoles that buzz to instruct a wearer to if so, how should we, as developers, change direction, could improve accessibility and correct some setbacks unwittingly designers, and users, set about manag- imposed upon communities for whom at screens might be as functionally meaningless as sheets of glass.6 ing this complexity? Although answers to these ques- REFERENCES tions are far from settled, one early

1. T. Okonski, “ZeroTouch: A New Multi nger Sensing Technology,” Texas A&M Computer intriguing approach was taken by the Science and Engineering Magazine, 2011, pp. 3–6; http://ecologylab.net/research MIT Media Lab’s theoretical explora- /publications/Kerne-Moeller-ZeroTouch-tamuEngNews.pdf. tions of “radical atoms”—materials

2. K. Nakagaki et al., “Materiable: Rendering Dynamic Material Properties in Response to Direct that can dynamically change their Physical Touch with Shape Changing Interfaces,” Proc. 2016 CHI Conf. Human Factors in form and appearance to make infor- Computing Systems (CHI), 2016, pp. 2764–2772. mation directly manipulable. In this line of research, objects change shape 3. A. Roudaut, A. Karnik, and S. Subramanian, “Morphees: Toward High ‘Shape Resolution’ in Self-Actuated Flexible Mobile Devices,” Proc. SIGCHI Conf. Human Factors in Computing to reect changes in their underlying Systems (CHI), 2013, pp. 593–602. computational states, such that affordances change concomitantly and 4. D. Lindlbauer et al., “Combining Shape-Changing Interfaces and Spatial Augmented Reality Enables Extended Object Appearance,” Proc. 2016 CHI Conf. Human Factors in Computing dynamically in order to inform users Systems (CHI), 2016, pp. 791–802. of these alterations.8 [See this issue’s Interview department for a related dis- 5. E. Geissler, K. Harnack, and A. Mühlenberend, “Sensole: An Insole-Based Tickle Tactile Interface,” Proc. 10th Int’l Conf. Tangible, Embedded, and Embodied Interaction (TEI), 2016, cussion with Hiroshi Ishii.] pp. 717–722. But dynamic affordances raise additional intriguing questions. As each 6. L. Gannes, “The Disappearing Interface,” All Things D, 4 Mar. 2013; http://allthingsd .com/20130304/the-disappearing-interface. generation has discovered, the introduction of new technologies transforms the things we think about, the things we think with, and the arenas in which we think.1 How might early learners in INTERACTING WITH such as cup handles that invite grasp- future generations come to conceptu- INFORMATIONAL OBJECTS ing and doorknobs that invite turn- alize object constancy, time, and even A central goal of design is to make ing, to one in which the execution of theory of mind when their daily inter- objects and interfaces disappear—to even simple tasks requires explicit actions are dominated by objects with get out of the way and let users achieve signi ers, or indicators of use.5 Flex- readily changeable areas, curvatures, their goals. Designer Don Norman, for ible and shape-changing interfaces and densities, and in which each form example, explicitly challenges devel- appear to be introducing yet another factor comes with an attendant set of opers to create tools that “ t the task phase of interactivity, in which direct, computational capabilities customized so well that [they] become part of the intuitive relations between objects to particular users,9 perhaps appearing task, feeling like a natural extension and human users will reemerge as to merge with the ambient environment of the work, a natural extension of the a priority, but without explicitly altogether?10 person.”5 signaling the existence or manage- Intelligence and connectivity also re- Over the past decades, an inux of ment of the full set of interaction con gure connections between objects, at-screen technologies have shifted possibilities. our environments, and ourselves, lead- the daily human experience from one This presents an interesting chal- ing to an erosion of boundaries and an in which the vast majority of interac- lenge. As our environments become accompanying expansion of connected tion affordances were readily apparent rich with “physical interfaces [that] systems. Some predict that we will through objects’ physical properties, can physically change to accommodate continue to become immersed in the

ComputingEdge August 2019 8 PERVASIVE24 computing www.computer.org/pervasive SOCIAL IMPACT SOCIAL IMPACT SOCIAL IMPACT

PHYSICAL COMPUTING APPLICATION AREAS different users, uses, and contexts,”6 so-called “infosphere” as technologies of the delivered services? What are the implementation of physical computing hat does the future hold? Virtual, in-air touchpad interfaces that enable real- are we moving toward a reality that move from being mere enhancements of implications of these new information has created a new ecosystem most accu- W time sensing of arm, hand, and nger positions could support surgical training has the appearance of being concrete our bodies (like hammers) to augment- flows for me, my family, and soci- rately conceptualized as, among other 1 by tracking ne hand movements. Deformable interfaces that mimic mechanical and unchanging, but that is actually far ing interfaces between different envi- ety? Should I participate in this brave things, a hybrid of home life and clini- properties of anatomical materials, water, and clay could help medical practitio- 7 ners distinguish between tumor types, or assist geologists in modeling tsunami or less perceptible or discoverable? When ronments (like washing machines) to new world?” cal medical care, each context of which earthquake outcomes.2 Screens that curve inward, remain at, or skitter away in presented with new materialities, will re-engineered realities, in which large, Analytic philosopher scholar Helen has radically different norms of appro- response to the presence of authorized (or unauthorized) persons could provide bet- we be more able to super cially engage distributed, and connected systems en- Nissenbaum has argued that people do priate information sharing.13 In one, 3 ter privacy, security, and personalization. Water faucets that narrow their aperture with objects but less able to understand velope us, altering how we use space not care about having complete control information might facilitate a sense of or bend away from users could provide unobtrusive nudges to conserve in periods of higher-than-normal water usage.3 Shape-changing tablets with co-located 3D what they really do, what they really and how we con gure our bodies in over information about themselves. camaraderie with family and friends. In graphics could simulate wave frequency and wind strength.4 Tactile representations know, and who they really talk to? And relation to one another.11 Rather, what they care about is that another, it might present thorny issues of navigable spaces (and more),5 such as insoles that buzz to instruct a wearer to if so, how should we, as developers, Philosopher Luciano Floridi argues, information is shared appropriately.12 regarding compliance, adherence, or change direction, could improve accessibility and correct some setbacks unwittingly designers, and users, set about manag- for example, that conating the mate- Nissenbaum’s privacy framework of physical safety, threatening the cost imposed upon communities for whom at screens might be as functionally meaningless as sheets of glass.6 ing this complexity? rial/physical with the informational contextual integrity provides a process or availability of insurance. Similarly, Although answers to these ques- reshapes our relationship with our for determining the appropriateness of smart home thermostats, lighting, and REFERENCES tions are far from settled, one early physical environments and even our new information ows by reference to water-monitoring services are caught in

1. T. Okonski, “ZeroTouch: A New Multi nger Sensing Technology,” Texas A&M Computer intriguing approach was taken by the own informational identities, leading the ends and values embedded within a a liminal space between norms of data Science and Engineering Magazine, 2011, pp. 3–6; http://ecologylab.net/research MIT Media Lab’s theoretical explora- to new notions of ourselves as informa- particular social context, such as home sharing in the service of energy-saving, /publications/Kerne-Moeller-ZeroTouch-tamuEngNews.pdf. tions of “radical atoms”—materials tional objects that collect experiences, life, employment, or medical care. An and norms that keep home life free

2. K. Nakagaki et al., “Materiable: Rendering Dynamic Material Properties in Response to Direct that can dynamically change their keep memories, and transmit curated, indiscretion confessed to a trusted fam- from potentially meddlesome outside 14 Physical Touch with Shape Changing Interfaces,” Proc. 2016 CHI Conf. Human Factors in form and appearance to make infor- reshaped historical narratives of our ily member might be considered “too parties. Computing Systems (CHI), 2016, pp. 2764–2772. mation directly manipulable. In this own lives. In Floridi’s view, future gen- much information” for one’s work col- This transition toward making pri- line of research, objects change shape erations will be doubly cursed: Forced leagues; health data willingly shared vate activities more easily seen, tracked, 3. A. Roudaut, A. Karnik, and S. Subramanian, “Morphees: Toward High ‘Shape Resolution’ in Self-Actuated Flexible Mobile Devices,” Proc. SIGCHI Conf. Human Factors in Computing to reect changes in their underlying to acquire unwanted characteristics and with a medical professional bound by and potentially controlled by others Systems (CHI), 2013, pp. 593–602. computational states, such that affor- disallowed the possibility of forgetting the Hippocratic Oath might be con- matters for several reasons, not least dances change concomitantly and or reinvention, they will nonetheless sidered “off limits” to a commercial because in technology, as with many 4. D. Lindlbauer et al., “Combining Shape-Changing Interfaces and Spatial Augmented Reality Enables Extended Object Appearance,” Proc. 2016 CHI Conf. Human Factors in Computing dynamically in order to inform users feel “deprived, excluded, handicapped, wellness app; location data given to a domains, “the bene ts and de cits are Systems (CHI), 2016, pp. 791–802. of these alterations.8 [See this issue’s or poor to the point of paralysis and navigation service in exchange for route not distributed equally,”1 and it is not Interview department for a related dis- psychological trauma…like sh out of planning might take on a new meaning always clear who will “win” and who 5. E. Geissler, K. Harnack, and A. Mühlenberend, “Sensole: An Insole-Based Tickle Tactile 11 1 Interface,” Proc. 10th Int’l Conf. Tangible, Embedded, and Embodied Interaction (TEI), 2016, cussion with Hiroshi Ishii.] water” if ever disconnected from the when it is later shared with law enforce- will lose. Clear and visible account- pp. 717–722. But dynamic affordances raise addi- infosphere. ment. As Nissenbaum explains, “when ability, as well as signals of loyalty and tional intriguing questions. As each actions or practices violate entrenched discretion, will become particularly 6. L. Gannes, “The Disappearing Interface,” All Things D, 4 Mar. 2013; http://allthingsd .com/20130304/the-disappearing-interface. generation has discovered, the intro- UNDERSTANDING ERODING informational norms, they provoke pro- important as computation becomes fur- duction of new technologies transforms BOUNDARIES test, indignation, or resistance. When ther embedded in structural elements in the things we think about, the things A second important consequence of actions or practices are in compliance, our environments, including furniture, we think with, and the arenas in which fusing physical and computational they respect contextual integrity.”12 walls, and other infrastructural compo- we think.1 How might early learners in environments is the new prospect of From this perspective, determining nents that are able to observe, react, and INTERACTING WITH such as cup handles that invite grasp- future generations come to conceptu- collecting, analyzing, and distributing whether and how new information always remember. INFORMATIONAL OBJECTS ing and doorknobs that invite turn- alize object constancy, time, and even vast amounts of personal data that sup- ows violate privacy requires an assess- A central goal of design is to make ing, to one in which the execution of theory of mind when their daily inter- port inferences about people’s habits, ment of whether and how they violate RECOMMENDATIONS objects and interfaces disappear—to even simple tasks requires explicit actions are dominated by objects with preferences, lifestyles, and social af li- or enhance current contextual social We are still in the earliest days of get out of the way and let users achieve signi ers, or indicators of use.5 Flex- readily changeable areas, curvatures, ations. As the engineering accomplish- norms.13 determining how new systems should their goals. Designer Don Norman, for ible and shape-changing interfaces and densities, and in which each form ments of physical computing advance, However, a central challenge to pri- behave when facing uncertain cir- example, explicitly challenges devel- appear to be introducing yet another factor comes with an attendant set of so does the importance of understand- vacy today is that physical computing cumstances. In practice, which posi- opers to create tools that “ t the task phase of interactivity, in which direct, computational capabilities customized ing how products, data flows, and is hastening the blurring of previously tive steps must be taken to maximize so well that [they] become part of the intuitive relations between objects to particular users,9 perhaps appearing policies can be architected to respect well-established contextual boundaries. human value and prevent or minimize task, feeling like a natural extension and human users will reemerge as to merge with the ambient environment user privacy and maximize fairness. To take but one example, consider so- harm? There are no easy answers, of the work, a natural extension of the a priority, but without explicitly altogether?10 Although individuals might be eager called “aging-in-place” devices and ser- but for now, I suggest that we think person.”5 signaling the existence or manage- Intelligence and connectivity also re- to sample new technologies, appearing vices. Embedding sensors in common aspirationally. What would a “good” Over the past decades, an inux of ment of the full set of interaction con gure connections between objects, on the surface to forsake privacy for household objects such as refrigerators, future world look like? How can physi- at-screen technologies have shifted possibilities. our environments, and ourselves, lead- convenience or utility, research sug- door locks, and even mattresses has cal computing systems be designed to the daily human experience from one This presents an interesting chal- ing to an erosion of boundaries and an gests that a host of doubts lurk beneath: the laudable aim of helping the elderly bring that future closer? in which the vast majority of interac- lenge. As our environments become accompanying expansion of connected “Why is this object here with me, and maintain independence and forestall To start, we would be wise to opti- tion affordances were readily apparent rich with “physical interfaces [that] systems. Some predict that we will whose interests is it serving—mine, the moves to assisted living facilities. But mize for coordination, discoverability, through objects’ physical properties, can physically change to accommodate continue to become immersed in the company who made it, or the providers recent analyses make clear that this and understanding.3 Coordinating with

www.computer.org/computingedge 8 PERVASIVE computing www.computer.org/pervasive OCTOBER–DECEMBER 2017 PERVASIVE computing25 9 SOCIAL IMPACT SOCIAL IMPACT

humans means complementing, rather carries different social implications in maintenance needs, adjust performance than duplicating, our strengths and cre- the cases of say, information collection in real time to meet uctuating targets, ating objects that understand and work vs. organ donation vs donating grocery and even recongure themselves, tak- with human mental models, rather than bag fees to charity), a pro-ethical sys- ing into consideration the state of the imposing their own. Discoverability tem would leave choices open and ask system as a whole. Digitization of man- and understanding challenge us to cre- a user to make a decision before he or ufacturing technologies will enable ate the conditions under which humans she can proceed with a transaction. In greater individualization, bringing new can easily grasp core system function- this approach, the challenge is to rst opportunities in elds such as automo- alities and comprehend its basic opera- create the reflection infrastructure tive design.18 tions and limitations. This entails also (infraethics) and then approach the This transition will also implicate thinking carefully about what types of contents (ethics) itself, considering how system security, reliability, and, no less feedback are informative and action- to present the implications of various pressing, employment prospects for the able without being too irritating or alternative choices while being mindful humans whose professional expertise intrusive, too often. of the nudges that design imposes upon in operating complex machinery may From a privacy and information human users. be subjected to new demands made ow perspective, best practices suggest for, and perhaps by, replacements of designing privacy features into systems FUTURE APPLICATIONS: the objects they once controlled. As at the outset, rather than attempting to BUILDING WITH machines begin to break free from tack them on at the end of the develop- SELF-AWARENESS deterministic instructions and train ment process.15 In practice, this means Looking ahead, much interesting work themselves, we would be wise to ensure carefully exploring the aims, values, will be happening in a variety of pro- that they receive human guidance along and ends of the social context in which duction environments, where a shift the way. Whether for purposes of plan- a particular technology or set of tech- from instruction-based to behavior- ning, operations, or maintenance and nologies will be implemented and cre- based fabrication offers intriguing repair, human input is critical for devel- ating an integrated set of information possibilities for industry. Sometimes oping procedures for assigning control, ow settings that enhance these values. referred to as Industry 4.0 or the Fourth accountability, and liability when sys- If contextual boundaries are blurry, the Industrial Revolution, a merger of tems fail. challenge becomes determining how to material synthesis and connected com- Let us remember that even the most design for the optimum degree of trans- puting is ushering in an ecosystem in promising new technologies may rein- parency and granular control: when, which machines embedded with sensors force social inequalities and create new how, and how frequently should users and actuators are predicted to substi- forms of disadvantage for which we be offered information about what tute real-time physical sensing for pre- are ill-equipped to deal. Developing a information is being collected about dictive modeling. No longer limited to stronger sense of what really matters to them, how it is being processed, stored, executing predetermined tasks, robotic people—politically, economically, and and distributed? What types of simple entities will be able to sense, learn, and culturally—will set us on a path toward tools would allow them to selectively create adaptively, reconguring them- creating new technologies that are not dismiss or reject unwanted informa- selves to new environments.16 merely transformative but also respon- tion ows without losing access to core In the architectural and design sive to human needs.19 services? domain, scholars such as Achim Menges In this vein, we might look beyond have asked, “what happens if the pro- task objectives and toward larger social duction machine no longer remains REFERENCES goals. Luciano Floridi introduces the just the obedient executor of predeter- 1. N. Postman, Technopoly: The Surren- term “infraethics,” which he denes mined instructions, but begins to have der of Culture to Technology, Knopf, as “the design of environments that the capacity to sense, react and act; in 1992. 17 can facilitate ethical choices, actions, other words, to become self-aware?” 2. M. McCullough, Digital Ground, MIT or processes.”11 Different from Eth- A large-scale adoption of machines that Press, 2004. ics by Design, which privileges a set of “self-predict, self-congure, and self- 3. K. Nakagaki et al., “Materiable: Ren- 17 behaviors pre-determined by a designer organize” could have enormous eco- dering Dynamic Material Properties to meet particular ethical standards, a nomic upsides for manufacturers that in Response to Direct Physica Touch “pro-ethical” design privileges user benet from dynamically recongured with Shape Changing Interfaces,” Proc. 2016 CHI Conf. Human Factors re ection. and streamlined work ows. Data from in Computing Systems (CHI), 2016, For example, rather than designing supply chains and production lines will pp. 2764–2772. a system with defaults and allowing enable factories to keep track of their 4. D.A. Norman, The Design of Everyday a person to opt in or opt out (which own (and each other’s) production and Things, Basic Books, 2013.

ComputingEdge August 2019 10 PERVASIVE26 computing www.computer.org/pervasive SOCIAL IMPACT SOCIAL IMPACT

SOCIAL IMPACT This article originally appeared in IEEE Pervasive Computing, vol. 16, no. 4, 2017.

humans means complementing, rather carries different social implications in maintenance needs, adjust performance 5. D.A. Norman, The Invisible Computer: Object Appearance,” Proc. 2016 CHI Computational, vol. 85, no. 5, 2015, than duplicating, our strengths and cre- the cases of say, information collection in real time to meet uctuating targets, Why Good Products Can Fail, the Per- Conf. Human Factors in Computing pp. 93–99. sonal Computer Is So Complex, and Systems (CHI), 2016, pp. 791–802. ating objects that understand and work vs. organ donation vs donating grocery and even recongure themselves, tak- Information Appliances Are the Solu- 17. A. Menges, “The New Cyber-Physical with human mental models, rather than bag fees to charity), a pro-ethical sys- ing into consideration the state of the tion, MIT Press, 1998. 11. L. Floridi, The Fourth Revolution: Making in Architecture: Computational imposing their own. Discoverability tem would leave choices open and ask system as a whole. Digitization of man- How the Infosphere Is Reshaping Construction,” Material Synthesis: 6. M. Coelho and J. Zigelbaum, “Shape- Human Reality, Oxford University Fusing the Physical and the Computa- and understanding challenge us to cre- a user to make a decision before he or ufacturing technologies will enable Changing Interfaces,” J. Personal and Press, 2014. tional, vol. 85, no. 5, 2015, pp. 28–33. ate the conditions under which humans she can proceed with a transaction. In greater individualization, bringing new Ubiquitous Computing, vol. 15, no. 2, can easily grasp core system function- this approach, the challenge is to rst opportunities in elds such as automo- 2011, pp. 161–173. 12. H. Nissenbaum, Privacy in Context: 18. B. Baudy et al., “Computational Design 18 Technology, Policy, and the Integrity of and Automotive Material Gestalt,” alities and comprehend its basic opera- create the reflection infrastructure tive design. 7. J.J. Gibson, “The Theory of Affor- Social Life, Stanford Univ. Press, 2009. Material Synthesis: Fusing the Physical tions and limitations. This entails also (infraethics) and then approach the This transition will also implicate dances,” Perceiving, Acting, and Know- and the Computational, vol. 85, no. 5, thinking carefully about what types of contents (ethics) itself, considering how system security, reliability, and, no less ing: Toward an Ecological Psychology, 13. P. Bruening and H. Patterson, “A Con- 2015, pp. 114–121. Wiley, 1977. text-Driven Rethink of the Fair Infor- feedback are informative and action- to present the implications of various pressing, employment prospects for the mation Practice Principles,” J. Business 19. C. Madsbjerg, Sensemaking: What able without being too irritating or alternative choices while being mindful humans whose professional expertise 8. H. Ishii et al., “Radical Atoms: Beyond & Technology Law, vol. 13, 2017. Makes Human Intelligence Essential in intrusive, too often. of the nudges that design imposes upon in operating complex machinery may Tangible Bits, Towards Transformable the Age of the Algorithm, Brown, 2017. Materials,” Interactions, Jan./Feb. 14. H. Nissenbaum and H. Patterson, “Bio- From a privacy and information human users. be subjected to new demands made 2012, pp. 38–51. sensing in Context: Health Privacy in ow perspective, best practices suggest for, and perhaps by, replacements of a Connected World,” Quanti ed: Bio- designing privacy features into systems FUTURE APPLICATIONS: the objects they once controlled. As 9. A. Roudaut, A. Karnik, and S. Sub- sensing Technologies in Everyday Life, ramanian, “Morphees: Toward High 2016, pp. 79–100. at the outset, rather than attempting to BUILDING WITH machines begin to break free from ‘Shape Resolution’ in Self-Actuated Heather M. Patterson is tack them on at the end of the develop- SELF-AWARENESS deterministic instructions and train Flexible Mobile Devices,” Proc. SIG- 15. A. Cavoukian, “Privacy by Design,” a senior research scientist ment process.15 In practice, this means Looking ahead, much interesting work themselves, we would be wise to ensure CHI Conf. Human Factors in Comput- IEEE Technology and Society Maga- at Intel Labs. Contact her ing Systems (CHI), 2013, pp. 593–602. zine, Winter 2012, pp. 18–19. carefully exploring the aims, values, will be happening in a variety of pro- that they receive human guidance along at heather.m.patterson@ and ends of the social context in which duction environments, where a shift the way. Whether for purposes of plan- 10. D. Lindlbauer et al., “Combining 16. T. Schwinn and A. Menges, “Fab- intel.com. a particular technology or set of tech- from instruction-based to behavior- ning, operations, or maintenance and Shape-Changing Interfaces and Spatial rication Agency,” Material Syn- Augmented Reality Enables Extended thesis: Fusing the Physical and the nologies will be implemented and cre- based fabrication offers intriguing repair, human input is critical for devel- ating an integrated set of information possibilities for industry. Sometimes oping procedures for assigning control, ow settings that enhance these values. referred to as Industry 4.0 or the Fourth accountability, and liability when sys- If contextual boundaries are blurry, the Industrial Revolution, a merger of tems fail. challenge becomes determining how to material synthesis and connected com- Let us remember that even the most design for the optimum degree of trans- puting is ushering in an ecosystem in promising new technologies may rein- ADVERTISER INFORMATION parency and granular control: when, which machines embedded with sensors force social inequalities and create new how, and how frequently should users and actuators are predicted to substi- forms of disadvantage for which we Now there’s be offered information about what tute real-time physical sensing for pre- are ill-equipped to deal. Developing a Read all your IEEE magazines even more to information is being collected about dictive modeling. No longer limited to stronger sense of what really matters to and journals your WAY on them, how it is being processed, stored, executing predetermined tasks, robotic people—politically, economically, and and distributed? What types of simple entities will be able to sense, learn, and culturally—will set us on a path toward Advertising Personnel Southwest, loveCalifornia: about your tools would allow them to selectively create adaptively, reconguring them- creating new technologies that are not Mike Hughes dismiss or reject unwanted informa- selves to new environments.16 merely transformative but also respon- Debbie Sims: Advertising Coordinator Email: [email protected]... Email: [email protected] Phone: +1 805 529 6790 tion ows without losing access to core In the architectural and design sive to human needs.19 Phone: +1 714 816 2138 | Fax: +1 714 821 4010 services? domain, scholars such as Achim Menges In this vein, we might look beyond have asked, “what happens if the pro- Advertising Sales Representative (Classi eds & Jobs Board) task objectives and toward larger social duction machine no longer remains REFERENCES Advertising Sales Representatives (display) goals. Luciano Floridi introduces the just the obedient executor of predeter- Introducing myCS, the digital magazine 1. N. Postman, Technopoly: The Surren- Heather Buonadies term “infraethics,” which he denes mined instructions, but begins to have der of Culture to Technology, Knopf, portalCentral, from Northwest, IEEE Computer Southeast, Society. Far East: Email: [email protected] as “the design of environments that the capacity to sense, react and act; in 1992. Go beyondEric Kincaid static, hard-to-read PDFs Phone: +1 201 887 1703 17 with an easily accessible, customizable, can facilitate ethical choices, actions, other words, to become self-aware?” 2. M. McCullough, Digital Ground, MIT Email: [email protected] 11 and adaptive experience. or processes.” Different from Eth- A large-scale adoption of machines that Press, 2004. Phone: +1 214 673 3742 ics by Design, which privileges a set of “self-predict, self-congure, and self- Fax: +1 888 886 8599 Advertising Sales Representative (Jobs Board) 3. K. Nakagaki et al., “Materiable: Ren- 17 There’s No Additional Cost! behaviors pre-determined by a designer organize” could have enormous eco- dering Dynamic Material Properties to meet particular ethical standards, a nomic upsides for manufacturers that in Response to Direct Physica Touch Northeast, Midwest, Europe, Middle East: Marie Thompson “pro-ethical” design privileges user benet from dynamically recongured with Shape Changing Interfaces,” David Schissler Email: [email protected] Proc. 2016 CHI Conf. Human Factors Email: [email protected] re ection. and streamlined work ows. Data from Phone: 714-813-5094 in Computing Systems (CHI), 2016, Phone: +1 508 394 4026 For example, rather than designing supply chains and production lines will pp. 2764–2772. Fax: +1 508 394► 1707LEARN MORE AT: mycs.computer.org a system with defaults and allowing enable factories to keep track of their 4. D.A. Norman, The Design of Everyday a person to opt in or opt out (which own (and each other’s) production and Things, Basic Books, 2013.

IEEE myCS half Page Space Ad 2016_4-26-16.indd 1 4/28/16 3:03 PM www.computer.org/computingedge 10 PERVASIVE computing www.computer.org/pervasive OCTOBER–DECEMBER 2017 PERVASIVE computing27 11

DEPARTMENT: INTERNET OF THINGS

Next-Generation Smart Environments: From System of Systems to Data Ecosystems

Edward Curry Digital transformation is driving a new wave of large- Lero, NUI Galway scale data-rich smart environments with data on Amit Sheth every aspect of our world. The resulting data Kno.e.sis, Wright State University ecosystems present new challenges and opportunities in the design of intelligent systems and system of systems.

Smart Environments are generating significant quantities of data due to a convergence of digital infrastructure from the Internet of Things (IoT), Edge, Fog, and Cloud Computing that is driving a new wave of data-driven intelligent systems. Through the generation and analysis of data from the smart environment, data-driven systems are transforming our everyday world, from the digitization of traditional infrastructure (smart grid, water, and mobility), the revolution of industrial sectors (smart autonomous cyber-physical systems, autonomous vehicles, and industry 4.0), to changes in how our society operates (smart government and cities). At the other end of the scale, we see more human-centric thinking in our systems1 where users have growing expectations for highly personalized digital services for the “market-of-one.” The digital transformation is creating a data ecosystem with data on every aspect of our world spread across a range of intelligent systems. Data ecosystems present new challenges to the design of intelligent systems and system of systems requiring a rethink in how we deal with the needs of large-scale data-rich smart environments. How can intelligent systems leverage their data ecosystem to be “smarter?” How can we support data sharing data between smart systems in an ecosystem? How can systems adapt to take advantage of the data within the ecosystem? What are practical approaches to the governance of data within an ecosystem? How can we make trusted decisions using data and humans within the ecosystem? Solving these problems is critical if we are to progress towards next-generation, data-intensive intelligent systems.

IEEE Intelligent Systems Published by the IEEE Computer Society May/June 2018 69

FROM DETERMINISTIC TO PROBABILISTIC DECISIONS IN SMART ENVIRONMENTS Within a smart environment a range of reliability is required. Consider the example of the autonomous connected car/vehicle. We have the strict requirements of safety-critical autonomous driving system, and a failure may lead to loss of life or serious personal injury Compare that to the “good enough” infotainment systems, where a failure is acceptable and merely an inconven- ience to the user. When it comes to making decisions in smart systems, there are two general approaches: deterministic (model-driven) and probabilistic (data-driven). A critical difference between the approaches can be explored by considering the costs and level of reliability and adaptability each provides. There is a tension between reliability, predictability, and cost:1 usually the more dependable and reliable the system needs to be, the more cost is associated with its development. Typically, we can see deterministic systems as reliable but with high costs to develop and adapt (i.e., autopilot), and probabilistic as low-cost to build and adapt, but less reliable (i.e., infotainment). Where high-levels of reliability are needed, deterministic approaches are an obvious choice for the design of smart systems. This is because the environment is optimized based on a formal deterministic model, and a set of rules and/or equations details the decision logic for the system that is used to control the activity in the environment in an efficient and predictable manner. Adapting the system to meet changes in the environment is a costly process, as the model and its rules need to be updated by expert system engineers. In the probabilistic approach, the core of the decision process is a statistical model that has been learned from an analysis of training data to learn the structure of a decision model automatically from the observed data (i.e., driver behavior). Thus, a fundamental requirement of data-driven approaches is the need for data to train the algorithms. A lack of data, and training data, within a smart environment limits the use of data-driven approaches. As the IoT is enabling the deployment of lower-cost sensors, we are seeing broader adoption of intelligent systems and gaining more visibility (and data) into smart environments. Not only are smart environments generating more data, but they are also producing different types of data with an increase in the number of multimedia devices deployed such as vehicle and traffic cam- eras. The emergence of the Internet of Multimedia Things (IoMT) is resulting in large quantities of high-volume and high-velocity multimedia event streams that need to be processed. The result is a data-rich ecosystem of structured and unstructured data (i.e., images, video, audio, and even text) detailing the smart environment that can be exploited by data-driven techniques. The increased availability of data has opened the door to the use of the data-driven probabilistic models, and their use within smart environments is becoming increasingly commonplace for “good enough” scenarios. It is estimated that a single connected car will upload about twenty- five gigabytes of data per hour (http://www.cisco.com/web/about/ac79/docs/mfg/Connected-Ve- hicles_Exec_Summary.pdf), while a vehicle fitted with an autonomous vehicle imaging and scanning system generates and processes about 4 TB of data for every hour of autonomous driving (https://www.datamakespossible.com/evolution-autonomous-vehicle-ecosystem/). As a result, the conventional rule-based approach is now being augmented with data-driven approaches that support optimizations driven by techniques including machine learning, cognitive, and AI techniques that are opening up new possibilities in the design of smart systems. For example, pedestrian detection is difficult to implement in a rule-based approach. However, deep learning models for object detection and semantic segmentation using a dash-mounted camera are very effective at detecting pedestrians. Systems can now adapt to changes in the environment by leveraging the data generated in the environment within their learning process to improve performance. If systems share data on their operational experiences, then the pooled data can be used to improve the overall learning processes of all the systems, giving us a form of collective artificial intelligence through the “wisdom of the systems.” Because the process is data-driven, it can be run and re-run at low cost. This critical role of data in enabling adaptability and collective machine intelligence makes it a precious resource.

May/Junewww.computer.org/computingedge 2018 70 www.computer.org/inteligent29 INTERNET OF THINGS

SYSTEM OF SYSTEMS The need to bring together multiple systems within a smart environment to work together is becoming a standard requirement. Initiatives such as Smart Cities are showing how different systems within the city (i.e., energy and transport) can collaborate to maximize the potential to optimize overall city operations. Autonomous connected vehicles can support smart city mobility by providing a vital feedback loop for cities on the state of traffic volumes, flows, roadway design and maintenance, and the mobility requirements (trip information) of its occupants. This requires a System of Systems (SoS) approach to connect systems that cross organizational boundaries (i.e. city, automotive, personal data), come from different domains (i.e., entertain- ment, manufacturing, logistics, etc.), and operate at different levels (i.e., city, district, neighbor- hood, fleet, vehicle, or individual passenger). The joint ISO/IEC/IEEE definition of a SoS brings together a set of systems for a task that none of the systems can accomplish on its own. Each constituent system keeps its management, goals, and resources while coordinating within the SoS and adapting to meet SoS goals.”2 Maier3 identified a set of characteristics to describe a SoS:

• Operational independence: constituent systems can operate independently from the SoS and other systems. • Managerial independence: constituent systems are managed by different entities. • Geographic distribution is the degree to which a system is widely spread or localized. • Evolutionary development: the evolution of a SoS and its behavior, which requires changes to system interfaces to be maintained and kept consistent. • Emergent behavior: new emergent behavior can be observed when the SoS changes.

DATA ECOSYSTEMS Within a data ecosystem, participants (individuals or organizations) can create new value that no single participant could achieve by itself.4 A data ecosystem can form in different ways—around an organization, an activity (mobility), a community of interest (music), a geographical location (city), or within or across industrial sectors (automotive, manufacturing, pharmaceutical). In the context of a smart environment, the data ecosystem metaphor is useful to understand the challenges in maximizing the value of data within the environment. The cross-fertilization and sharing of vital resources and datasets from different participants is a key benefit of data ecosystems, leading to new business opportunities and easier access to knowledge and data.

May/June30 2018 ComputingEdge 71 www.computer.org/inteligentAugust 2019 INTERNET OF THINGS IEEE INTELLIGENT SYSTEMS

There are many challenges in bringing together the constituent systems into a SoS at the data, service, process, and organizational levels that require advanced systems engineering. At the data-level, data-driven approaches can benefit from leveraging data from multiple systems within the smart environment. This requires support for the sharing of data at new scales between multiple complex interconnected system of systems within a smart environment. Figure 1. Connected and Autonomous Vehicle Data Ecosystem DATA ECOSYSTEMS Within a data ecosystem, participants (individuals or organizations) can create new value that no Figure 1 details the data ecosystem for connected and autonomous vehicles where a community single participant could achieve by itself.4 A data ecosystem can form in different ways—around of interacting data-intensive systems share and combine their data to provide a holistic functional an organization, an activity (mobility), a community of interest (music), a geographical location view of the car, passenger, city mobility, and service & infrastructure providers. Systems within (city), or within or across industrial sectors (automotive, manufacturing, pharmaceutical). In the the ecosystem can also come together to form a SoS. The ecosystem supports the flow of data context of a smart environment, the data ecosystem metaphor is useful to understand the chal- between systems, enabling the creation of data value chains to understand, optimize, and rein- lenges in maximizing the value of data within the environment. The cross-fertilization and shar- vent processes that deliver insight to optimize the overall ecosystem. Data may be shared about ing of vital resources and datasets from different participants is a key benefit of data ecosystems, the current operating conditions of the vehicle, traffic flows, or context of the passengers; a fam- leading to new business opportunities and easier access to knowledge and data. ily on holiday, or a business executive moving between meetings. The pooled data can be used to support personalized digital services (i.e. delivering the latest episode of the family’s favorite sitcom) and real-time decision-making (i.e. delivering relevant information for the business executive’s next meeting). Data on past operating conditions can be shared to improve the learning processes of all systems in the ecosystem. The nature of the ecosystem, the systems themselves, and the system dynamics will affect the design and operation of the ecosystem. Enabling data ecosystems for smart environments will require a rethink in the design of intelligent systems to consider ecosystem concerns including governance, economics, and technical challenges. Data infrastructure is needed to support data sharing within the ecosystem—from data provided by a single dominant actor on their proprietary infrastructure, to a community pooling their data in a managed open source data platform. To understand the dynamics of a smart environment data ecosystem we can look to the literature on SoS3 and business ecosystem5 to help us understand the different types of data ecosystem that can exist. In Figure 2 we bring together these two areas in the design of a data ecosystem for a smart environment: Koening5 identified two key criteria regarding the design of a business ecosystem that will also influence a data ecosystem, namely, resource control, and interdependence:

May/June 2018 71 www.computer.org/inteligent May/Junewww.computer.org/computingedge 2018 72 www.computer.org/inteligent31 INTERNET OF THINGS

• Control of key data resources: Who controls the essential data resources in the ecosystem? Does a single “keystone”6 actor control the key data resources that all others de- pend on, or is control of the key data resources spread across multiple actors in the ecosystem? • Participant interdependence: Interdependence is based on the degree to which different participants in the ecosystem must interact and exchange data for performing their activities. Reciprocal interdependence requires high levels of coordination between the participants, while pooled interdependence enables loose coupling between participants.

Figure 2. Topology of Data Ecosystems (adapted from Koening5 and Maier3)

Drawing inspiration from the SoS classification by Maier3 (which defines Virtual, Collaborative, Acknowledged, and Directed categories) and the ecosystem topology by Koening, we can consider the different types of data ecosystems that may exist within a smart environment (Figure 2).

• Directed data ecosystems are centrally controlled to fulfill a specific purpose. Typically found within an organization setting or following a keystone model, participants within a directed ecosystem maintain an ability to operate independently, but their standard operational mode is subordinated to the centrally managed purpose of the ecosystem. • Acknowledged data ecosystems have defined objectives and pooled dedicated resources. The constituent systems retain their independent ownership and objectives. Changes in the ecosystem are based on collaboration between the distributed participants. • Collaborative data ecosystems have participants interact voluntarily to fulfill an agreed- upon central purpose. The primary players collectively decide the means of enforcing and maintaining standards between the federations of participants. • Virtual data ecosystems have no central management authority and no centrally agreed upon purpose. Bottom-up coalitions of participants emerge from a virtual data ecosystem to pool decentralized resources to achieve specific goals.

FUTURE DIRECTIONS Enabling a smart environment data ecosystem will require many challenges to be overcome regarding infrastructure, governance, systems engineering, and human-centricity.

May/June32 2018 ComputingEdge 73 www.computer.org/inteligentAugust 2019 INTERNET OF THINGS IEEE INTELLIGENT SYSTEMS

• Control of key data resources: Who controls the essential data resources in the ecosystem? Does a single “keystone”6 actor control the key data resources that all others de- Trusted Data Platforms pend on, or is control of the key data resources spread across multiple actors in the ecosystem? To support the ecosystem and the interconnection of systems, there is a need to enable the shar- • Participant interdependence: Interdependence is based on the degree to which different ing of data between systems. Platform approaches have proved successful in many areas of tech- participants in the ecosystem must interact and exchange data for performing their activ- nology, and the idea of large-scale "data" platforms have been touted as a possible next step. A ities. Reciprocal interdependence requires high levels of coordination between the par- data platform focuses on the secure and trusted data sharing among a group of participants (i.e., ticipants, while pooled interdependence enables loose coupling between participants. industrial consortiums sharing private or commercially sensitive data) within a clear legal framework. An ecosystem data platform would have to be infrastructure agnostic and have to support continuous, coordinated data flows, seamlessly moving data between systems. Data exchange could be based on models for monetization or reciprocity. Data platforms can create possibilities for smaller organizations and even individual developers to get access to large volumes of data, enabling them to explore their potential. Data platforms open up many research areas including data discovery, curation, linking, synchronization, standardization, and decentralization. How- ever, the challenges go beyond the technical to issues of data ownership, privacy, business models, and licensing and authorized reuse by third parties.

Ecosystem Data Governance For mass collaboration to take place within data ecosystems, we need to overcome the challenges of dealing with large-scale agreements between potentially decoupled interacting parties. Re- search is needed on decentralized data governance models for data ecosystems that support collaboration and fully consider ethical, legal, and privacy concerns. Data governance within an ecosystem must recognize data ownership, sovereignty, and regulation while supporting economic models for the sustainability of the data ecosystem. A range of decentralized governance approaches may guide a data ecosystem from authoritarian to democratic alternatives, including majority voting, reputation models (i.e., eBay), proxy-voting, and dynamic governance (i.e., sociocracy: circles and double linking).7 Finally, economic concerns may be considered as an in- centivization factor within governance models with "data-vote exchange" models where participants pay for votes with data.

Figure 2. Topology of Data Ecosystems (adapted from Koening5 and Maier3) Incrementally Evolving Systems Engineering: Cognitive Drawing inspiration from the SoS classification by Maier3 (which defines Virtual, Collaborative, Adaptability Acknowledged, and Directed categories) and the ecosystem topology by Koening, we can consider the different types of data ecosystems that may exist within a smart environment (Figure 2). The design of adaptive systems will need to consider the implication of operating within an ecosystem. The boundaries of systems will be fluid and will change and evolve at runtime to adapt to the context of the current situation. However, we must also consider the cost of system partici- • Directed data ecosystems are centrally controlled to fulfill a specific purpose. Typically pation, and support "pay-as-you-go" approaches at both the system and data-levels. For data found within an organization setting or following a keystone model, participants within 8 a directed ecosystem maintain an ability to operate independently, but their standard op- management, dataspaces represent one avenue where a pay-as-you-go approach has been ap- erational mode is subordinated to the centrally managed purpose of the ecosystem. plied to integrate data on an "as-needed" basis with the labor-intensive aspects of data integration 9 • Acknowledged data ecosystems have defined objectives and pooled dedicated re- postponed until they are required. How can the pay-as-you-go approach be extended to the de- sources. The constituent systems retain their independent ownership and objectives. sign of incremental and evolving systems? Changes in the ecosystem are based on collaboration between the distributed partici- Work on evolving systems engineering10 will need to consider the inclusion of data-driven prob- pants. abilistic techniques that can provide “cognitive adaptability” that will help systems adapt to • Collaborative data ecosystems have participants interact voluntarily to fulfill an agreed- changes in the environment that were unknown at design-time. Adaptive systems require new upon central purpose. The primary players collectively decide the means of enforcing iterative development processes that require training and deploying machine learning models and maintaining standards between the federations of participants. over massive volumes of training data with close collaboration between data scientists, software • Virtual data ecosystems have no central management authority and no centrally agreed upon purpose. Bottom-up coalitions of participants emerge from a virtual data ecosys- developers, data engineers, and governance professionals. System design will need to consider tem to pool decentralized resources to achieve specific goals. the varying levels of accuracy offered by data-driven approaches, providing best-effort or ap- proximate results using the data accessible at the time.8 How can we mix deterministic and statistical approaches? How can we test and verify these systems? What are the challenges in making FUTURE DIRECTIONS decisions using multiple sources from the ecosystem? Enabling a smart environment data ecosystem will require many challenges to be overcome regarding infrastructure, governance, systems engineering, and human-centricity.

May/June 2018 73 www.computer.org/inteligent May/Junewww.computer.org/computingedge 2018 74 www.computer.org/inteligent33 INTERNET OF THINGS

Towards Human-Centric Systems Currently, intelligent systems make critical decisions in highly-engineered systems (i.e., autopi- lots) where users receive specialized training to interact with them (i.e., pilots). As we move forward, intelligent systems will be making both critical and lifestyle decisions—from the course of treatment for a critical illness and safely driving a car, to choosing what takeout to order and the temperature of our shower. Data-driven decision approaches (including cognitive and AI-based techniques) will need to provide explanations and evidence to support their decisions and guar- antees for the decisions they recommend. The role of users in data ecosystems will not be a pas- sive one. Users are a critical part of socio-technical systems, and we need to consider more ways of including the “human in the loop” within future systems. Active participation of users can improve their engagement and sense of ownership of the system. Indeed, active involvement of the user could be a condition for them granting access to their private data. Research is needed to build trust in algorithms and data—in the trusted co-evolution between humans and AI-based systems, and in the legal, ethical, and privacy issues associated with making data-driven critical decisions.

ACKNOWLEDGEMENTS This work was supported, in part, by Science Foundation Ireland grant 13/RC/2094 and co- funded under the European Regional Development Fund through the Southern & Eastern Regional Operational Programme to Lero - the Irish Software Research Centre (www.lero.ie).

REFERENCES 1. A. Sheth, “Computing for Human Experience: Semantics-Empowered Sensors, Services, and Social Computing on the Ubiquitous Web,” IEEE Internet Computing, vol. 14, no. 1, 2010, pp. 88–91. 2. ISO/IEC/IEEE 15288: 2015 Systems and Software Engineering - System Life Cycle Processes, standard ISO/IEC/IEEE 15288, ISO/IEC/IEEE, 2015. 3. M. W. Maier, “Architecting Principles for Systems-of-Systems,” Systems Engineering, Wiley, 1998. 4. New Horizons for a Data-Driven Economy: A Roadmap for Usage and Exploitation of Big Data in Europe, J. M. Cavanillas, E. Curry, and W. Wahlster, Springer International Publishing, 2016. 5. G. Koenig, “Business Ecosystems Revisited,” Management, vol. 15, 2012, pp. 208– 224. 6. H. Kim, J.-N. Lee, and J. Han, “The Role of IT in Business Ecosystems,” Communications of the ACM, vol. 53, 2010, p. 151. 7. J. A. Buck and S. Villines, We the Peopleࣟ: Consenting to a Deeper Democracyࣟ: a Guide to Sociocratic Principles and Methods, Sociocracy.info, 2007. 8. M. Franklin, A. Halevy, and D. Maier, “From Databases to Dataspaces: A New Abstraction for Information Management,” ACM SIGMOD Record, vol. 34, no. 4, 2005, pp. 27–33. 9. E. Curry et al., “Internet of Things Enhanced User Experience for Smart Water and Energy Management,” IEEE Internet Computing, vol. 22, no. 1, 2018. 10. M. Hinchey and L. Coyle, “Evolving Critical Systems: A Research Agenda for Computer-Based Systems,” 217th IEEE International Conference and Workshops on Engineering of Computer Based Systems, 2010, pp. 430–435.

May/June34 2018 ComputingEdge 75 www.computer.org/inteligentAugust 2019 INTERNET OF THINGS IEEE INTELLIGENT SYSTEMS

Towards Human-Centric Systems Currently, intelligent systems make critical decisions in highly-engineered systems (i.e., autopi- ABOUT THE AUTHORS lots) where users receive specialized training to interact with them (i.e., pilots). As we move for- Edward Curry is a funded investigator at Lero: The Irish Software Research Centre and a ward, intelligent systems will be making both critical and lifestyle decisions—from the course of lecturer in informatics at the National University of Ireland Galway. http://edwardcurry.org treatment for a critical illness and safely driving a car, to choosing what takeout to order and the Amit Sheth is the LexisNexis Ohio Eminent Scholar and the executive director of Kno.e.sis temperature of our shower. Data-driven decision approaches (including cognitive and AI-based - Ohio Center of Excellence in Knowledge-enabled Computing and BioHealth Innovations. techniques) will need to provide explanations and evidence to support their decisions and guar- He is a fellow of the IEEE and the AAAI. http://knoesis.org/amit antees for the decisions they recommend. The role of users in data ecosystems will not be a pas- sive one. Users are a critical part of socio-technical systems, and we need to consider more ways of including the “human in the loop” within future systems. Active participation of users can improve their engagement and sense of ownership of the system. Indeed, active involvement of the user could be a condition for them granting access to their private data. Research is needed to build trust in algorithms and data—in the trusted co-evolution between humans and AI-based This article originally appeared in systems, and in the legal, ethical, and privacy issues associated with making data-driven critical decisions. IEEE Intelligent Systems, vol. 33, no. 3, 2018.

Rejuvenating Binary Executables ■ Visual Privacy Protection ■ Communications Jamming Policing Privacy ■ Dynamic Cloud Certiﬁcation ■ Security for High-Risk Users Smart TVs ■ Code Obfuscation ■ e Future of Trust

January/February 2016 March/April 2016 May/June 2016 6. H. Kim, J.-N. Lee, and J. Han, “The Role of IT in Business Ecosystems,” Vol. 14, No. 1 Vol. 14, No. 2 Vol. 14, No. 3 Communications of the ACM, vol. 53, 2010, p. 151. 7. J. A. Buck and S. Villines, We the Peopleࣟ: Consenting to a Deeper Democracyࣟ: a Guide to Sociocratic Principles and Methods, Sociocracy.info, 2007. 8. M. Franklin, A. Halevy, and D. Maier, “From Databases to Dataspaces: A New Abstraction for Information Management,” ACM SIGMOD Record, vol. 34, no. 4, 2005, pp. 27–33. 9. E. Curry et al., “Internet of Things Enhanced User Experience for Smart Water and IEEE Security & Privacy magazine provides articles Energy Management,” IEEE Internet Computing, vol. 22, no. 1, 2018. with both a practical and research bent by the top 10. M. Hinchey and L. Coyle, “Evolving Critical Systems: A Research Agenda for thinkers in the ﬁ eld. Computer-Based Systems,” 217th IEEE International Conference and Workshops on • stay current on the latest security tools and theories and gain invaluable practical and Engineering of Computer Based Systems, 2010, pp. 430–435. research knowledge, • learn more about the latest techniques and cutting-edge technology, and computer.org/security • discover case studies, tutorials, columns, and in-depth interviews and podcasts for the information security industry.

May/June 2018 75 www.computer.org/inteligent May/Junewww.computer.org/computingedge 2018 76 www.computer.org/inteligent35 IEEE Computer Architecture Letters is a forum for fast publication of new, high-quality ideas in the form of short, critically refereed technical papers. Submissions are accepted on a continuing basis and letters will be published shortly after acceptance in IEEE Xplore and in the Computer Society Digital Library. Submissions are welcomed on any topic in computer architecture, especially:

• Microprocessor and multiprocessor systems • Microarchitecture and ILP processors • Workload characterization • Performance evaluation and simulation techniques • Interactions with compilers and operating systems • Interconnection network architectures • Memory and cache systems • Power and thermal issues at the architectural level • I/O architectures and techniques • Independent validation of previously published results • Analysis of unsuccessful techniques • Domain-speciﬁ c processor architecture (embedded, graphics, network) • High-availability architectures • Reconﬁ gurable computer architectures www.computer.org/cal

Join the IEEE Computer Society for subscription discounts today! www.computer.org/product/journals/cal 21mitp02-laplante-2896775.3d (Style 5) 05-06-2019 17:2

Internet of Things

My Mother the Car (or Why It’s a Bad Idea to Give Your Car a Personality)

Phil Laplante Penn State

Abstract—What happens when we endow an autonomous vehicle with a personality? It sounds like a good idea—witty banter, friendly advice, and encouragement from your car. But science fiction writers predicted this technology and both the good and bad scenarios that could arise. A brief sample of some of these situations from American television and film is enlightening but suggests that granting a personality to an auto is ill-advised. This theme issue explores the state and potential benefits and ethical dilemmas of connected and autonomous vehicles. While fully autonomous vehicles are already experimentally deployed, they will become ubiquitous within a few years. Soon after we should see an experimental deployment of vehicles that have a level of artificial intelligence such that they could be considered “sentient” (self-aware) or even “anthropomorphic” (human-like). These vehicles could do much more than self-drive or give us driving advice, travel information, and various forms of infotainment. They could also remind us of important stops to be made, listen to our complaints about traffic and other drivers, and even provide us with soothing advice. And this advice could be given with the benefit of an artificial personality—one that could be formal, friendly, or even replicates that of a famous person, celebrity, or a loved one.

& BUT ADDING A personality to the car could ANTHROPOMORPHIC VEHICLES make things worse, as predicted by writers of IN AMERICAN FILM AND TELEVISION science fiction novels, short stories, television In the early silent short, Nothing Matters plays, and movie scripts. Let us review some (1926), the hero dreams of “anthropomorphic examples from American film and television. cars cowering and fleeing in fear of the new car in town”1 an interesting twist on the idea Digital Object Identifier 10.1109/MITP.2019.2896775 of communicating (connected) cars. The Date of current version 27 March 2019. vignette raises several interesting questions.

March/April 2019 Published by the IEEE Computer Society 1520-9202 ß 2019 IEEE 2469-7087/19/$33.00 © 2019 IEEE Published by the IEEE Computer Society August 2019 11 37 21mitp02-laplante-2896775.3d (Style 5) 05-06-2019 17:2

Internet of Things

For example, will anthropomorphic vehicles participated in the human protagonists’ mys- form cliques, show prejudice or discriminate? tery-solving adventures. And will connected vehicles organize to give The popular TV series Knight Rider (1982– preferential treatment to certain types of 1986) starred K.I.T.T., a sentient Pontiac Firebird vehicles, for example at merge points? powered by a “Knight 2000 microprocessor.” K.I. In a 1953 public service announcement (PSA) T.T. was self-aware and capable of learning. In film, The Talking Car, a fatherly vehicle advises addition to providing navigational aid and enter- children about safe street crossing. In a later ver- tainment, the car helped his crusading driver sion of the PSA (1969) a trio of vehicles (each, Michael solve crimes and resolve many danger- with a very distinct personality) sat in judgment ous situations. The vehicle also exhibited a very of a young boy who disregards basic street human-like personality that was egotistical, crossing protocol. These PSAs raise questions stodgy and funny. about the extent to which smart vehicles will Cars such as K.I.T.T. could not only save have authority over humans, how they may the day when called, they could also act auto- cooperate, and even if their “testimony” will be nomously and proactively. For example, the admitted in courts. “muggle” car owned by the Weasley family in In the short-lived and forgettable TV series, Harry Potter and Chamber of Secrets (1998 novel, My Mother the Car (1965–66), a man discovers 2002 movie) anticipates Harry and Ron’s need that his car hosts the reincarnated spirit of his for rescue from the giant spider den, arriving in deceased mother. Talking through the radio of a the nick of time. More recently, the Cars (2006 1928 Porter Touring car (this was an anachro- et al.) and Transformers (2007 et al.) movies and nism since the dashboard radio wasn’t invented sequels have featured vehicles with human-like until 1930), the protagonist, Dave (played by personalities, ambitions, and even expressing the actor, Jerry Van Dyke, who turned down the love. Of course, the main premise of the Trans- role of Gilligan in the much more successful, formers movies and television program featured Gilligan’s Island, for this clunker of a show), is the epic battle between the heroic autobots and pestered, advised and loved by the car. But villainous decepticon vehicles. mother also exhibits human emotions such as anger, jealousy, and greed. DANGERS OF SENTIENT AND In addition to these emotions, cars could also PSEUDO SENTIENT VEHICLES be heroic. For example, the namesake car in the What happens when a “self-aware” vehicle film Chitty Chitty Bang Bang (1968) (this is a errs, gets confused, or turns bad? Movies and British film, but it enthralled me as a child, so I Television have covered this aspect quite well. am including it in this brief survey) is self-aware. In an episode from My Mother the Car, “What This 1900s era modified roadster was autono- Makes Auntie Freeze,” Mother gets “drunk” on mous and could anticipate the need to sprout antifreeze leading to all kinds of erratic behavior. pontoons and a propeller for water travel, or In the episode, “I Remember Mama, Why Can’t wings for the flight to rescue its human family You Remember Me?” Dave’s mother gets amne- from an evil baron. sia following a fender bender. Even K.I.T.T. could An internationally successful film featuring be obstinate with Michael and Herbie could be another heroic car, Herbie the Love Bug (1968), prideful and stubborn. spawned four sequels over 40 years. Herbie The movie, Total Recall (1990), which was was self-driving, sentient Volkswagen Beetle adapted from a 1966 book, predicted autonomous with a mischievous, loyal, and loving personal- taxis and some of the problems they can present ity. Time and again, even while causing trouble in the case of misunderstanding. In one case, the with his antics, the precocious Herbie man- helpful “Johnny Cab” confuses the destination aged to rescue his owners from various predic- when the protagonist shouts expletives. But aments. Similarly, the vehicle in the animated Johnny Cab exhibits even worse behavior when “Speed Buggy” television program (1973) the fare is not paid—the heretofore obliging cab featured a goofy talking car who heroically turns homicidal and tries to run over the hero.

IT Professional 38 12 ComputingEdge August 2019 21mitp02-laplante-2896775.3d (Style 5) 05-06-2019 17:2 21mitp02-laplante-2896775.3d (Style 5) 05-06-2019 17:2

Internet of Things

For example, will anthropomorphic vehicles participated in the human protagonists’ mys- Self-awareness can also lead to paranoia. The and acting like an overbearing mother? Or do we form cliques, show prejudice or discriminate? tery-solving adventures. evil car from in Steven King’s Christine (novel need the car interrupting the radio to tell us we And will connected vehicles organize to give The popular TV series Knight Rider (1982– and movie 1983) involves a possessed 1958 Ply- are going too fast (or reporting us to the police). preferential treatment to certain types of 1986) starred K.I.T.T., a sentient Pontiac Firebird mouth Fury that exacts murderous revenge on But, more significantly, what happens if the car vehicles, for example at merge points? powered by a “Knight 2000 microprocessor.” K.I. the enemies of its owner. Both K.I.T.T. and becomes capricious, supercilious, paranoid or In a 1953 public service announcement (PSA) T.T. was self-aware and capable of learning. In Mother exhibited some signs of paranoia at goes berserk? film, The Talking Car, a fatherly vehicle advises addition to providing navigational aid and enter- times—what if they had completed turned evil? Science fiction writers have shown us the children about safe street crossing. In a later ver- tainment, the car helped his crusading driver And what happens when an anthropomorphic potential benefits and real dangers of anthropo- sion of the PSA (1969) a trio of vehicles (each, Michael solve crimes and resolve many danger- vehicle impersonates a human in some sort of morphic vehicles. Most of these dangers are with a very distinct personality) sat in judgment ous situations. The vehicle also exhibited a very Turing inspired nightmare? My Mother the Car related to the car violating our trust and untrust- of a young boy who disregards basic street human-like personality that was egotistical, foreshadowed this possibility too. In the episode, worthiness is a personality flaw. While much crossing protocol. These PSAs raise questions stodgy and funny. “TV or Not TV” Dave puts a TV in the garage for more research work is needed in this area (e.g., about the extent to which smart vehicles will Cars such as K.I.T.T. could not only save Mother, who creates a dilemma when she calls Future of Life Institute https://futureoflife.org/) have authority over humans, how they may the day when called, they could also act auto- into a game show and wins a chance to appear on I think we just should not give cars personalities. cooperate, and even if their “testimony” will be nomously and proactively. For example, the live television. Most people trust their mothers, if we cannot admitted in courts. “muggle” car owned by the Weasley family in While the premises of these situations seem trust our mother the car, which car can we trust? In the short-lived and forgettable TV series, Harry Potter and Chamber of Secrets (1998 novel, ridiculous, it is easy to imagine a computer virus 2002 movie) anticipates Harry and Ron’s need or malware negatively altering the “personality” My Mother the Car (1965–66), a man discovers & REFERENCE that his car hosts the reincarnated spirit of his for rescue from the giant spider den, arriving in of an anthropomorphic to manifest analogously. deceased mother. Talking through the radio of a the nick of time. More recently, the Cars (2006 1. J. Roots, 100 Greatest Silent Film Comedians. 1928 Porter Touring car (this was an anachro- et al.) and Transformers (2007 et al.) movies and Lanham, MD, USA: Rowman & Littlefield, 2014. nism since the dashboard radio wasn’t invented sequels have featured vehicles with human-like ROAD AHEAD until 1930), the protagonist, Dave (played by personalities, ambitions, and even expressing Artificially intelligent vehicles should provide Phillip A. Laplante (M’86–SM’90–F’08) is a love. Of course, the main premise of the Trans- the actor, Jerry Van Dyke, who turned down the great benefit, but new risks could emerge as the Professor of Software and Systems Engineering with role of Gilligan in the much more successful, formers movies and television program featured capabilities increase. Adding a personality to a Pennsylvania State University, Malvern, PA. Lately, Gilligan’s Island, for this clunker of a show), is the epic battle between the heroic autobots and vehicle will increase these risks. Some of the his research interests include the Internet of Things, pestered, advised and loved by the car. But villainous decepticon vehicles. risks are simply annoying—do we need our cars blockchain, and artificial intelligence. Contact him at mother also exhibits human emotions such as hectoring us about our weight or driving habits [email protected]. anger, jealousy, and greed. DANGERS OF SENTIENT AND In addition to these emotions, cars could also PSEUDO SENTIENT VEHICLES be heroic. For example, the namesake car in the What happens when a “self-aware” vehicle film Chitty Chitty Bang Bang (1968) (this is a errs, gets confused, or turns bad? Movies and British film, but it enthralled me as a child, so I Television have covered this aspect quite well. This article originally appeared in am including it in this brief survey) is self-aware. In an episode from My Mother the Car, “What IT Professional, vol. 21, no. 2, 2019. This 1900s era modified roadster was autono- Makes Auntie Freeze,” Mother gets “drunk” on mous and could anticipate the need to sprout antifreeze leading to all kinds of erratic behavior. pontoons and a propeller for water travel, or In the episode, “I Remember Mama, Why Can’t wings for the flight to rescue its human family You Remember Me?” Dave’s mother gets amne- from an evil baron. sia following a fender bender. Even K.I.T.T. could An internationally successful film featuring be obstinate with Michael and Herbie could be another heroic car, Herbie the Love Bug (1968), prideful and stubborn. spawned four sequels over 40 years. Herbie The movie, Total Recall (1990), which was was self-driving, sentient Volkswagen Beetle adapted from a 1966 book, predicted autonomous with a mischievous, loyal, and loving personal- taxis and some of the problems they can present ity. Time and again, even while causing trouble in the case of misunderstanding. In one case, the with his antics, the precocious Herbie man- helpful “Johnny Cab” confuses the destination aged to rescue his owners from various predic- when the protagonist shouts expletives. But aments. Similarly, the vehicle in the animated Johnny Cab exhibits even worse behavior when “Speed Buggy” television program (1973) the fare is not paid—the heretofore obliging cab featured a goofy talking car who heroically turns homicidal and tries to run over the hero.

IT Professional March/April 2019 12 www.computer.org/computingedge 13 39 39mcg02-campbell-2892863.3d (Style 5) 05-06-2019 17:44

Art on Graphics

Lance Gharavi: Performance Inspired Science Technology þ Bruce Campbell Francesca Samsel Rhode Island School of Design University of Texas

Editors: Bruce Campbell, [email protected], and Francesca Samsel, ﬁ[email protected]

Abstract—We caught up with Lance Gharavi after we heard of and investigated Beneath: A journey within, a live performance motivated by the intent to get a wider audience interested in the lithosphere and mantle beneath our feet. Beneath is an archetype of art-science-tech projects and Lance came to coordinating that show from a long history of art-science projects. We asked him about insights and lessons learned.

& DR.GHARAVI HAS been working at the inter- A Brief Anniversary of Time, a prior perfor- section of art, science, and engineering through mance project, represents well the role he was collaborating with large teams typically of artists, hired at ASU to perform. “We worked with a sys- designers, scientists, engineers, and others to tem I call a universe in a box,” he said. “The team make media rich works of performance. designed the performance for ASU’s flat screen Like many others who had a stick-to-it-ness planetarium—the Marston Exploration Theater during the first heyday of virtual reality (VR), that uses planetarium software called Sky-Skan. Lance brought the skills he acquired and fine- The Marston Exploration Theater software runs tuned to various projects when hired by Arizona on a stack of ten servers and every known object State University (ASU) to be an artist working in the universe is plotted in four dimensions. with digital technologies. VR was not officially You can fly around in this universe anywhere on the list of his job responsibilities but as Lance you want to go—even through time. It is a way suggested, “I realized it’s a matter of what you to experience the mind-bogglingly immense call VR. Many of those themes I explored trans- scope of our universe. I like to think of it as an ferred well into other digital technologies.” existential-crisis-machine.” The show was a celebration of the 25th anniversary of Stephen Hawking’s A Brief Digital Object Identifier 10.1109/MCG.2019.2892863 History of Time (seefigure1)withamixofpre- Date of current version 22 March 2019. recorded video and live performance played

0272-1716 ß 2019 IEEE Published by the IEEE Computer Society IEEE Computer Graphics and Applications

Art on Graphics

Lance Gharavi: Performance Inspired Science Technology þ Bruce Campbell Francesca Samsel Figure 1. A Brief Anniversary of Time. Media design by Daniel Fine. Photo credit: Matthew Ragan. (Used with permission.) Rhode Island School of Design University of Texas out in that planetarium software. As Lance animators, media designers, and artists outside Editors: Bruce Campbell, [email protected], and Francesca Samsel, fi[email protected] observed, “Much of what I do involves media of ASU, including Cloud Eye Control, Obscura design for live performance, with some interac- Digital, and Ohio State University. These internal Abstract—We caught up with Lance Gharavi after we heard of and investigated Beneath: tive data visualization.” and external partnerships across the ASU campus A journey within, a live performance motivated by the intent to get a wider audience Lance discussed the motivation of Beneath: and beyond have served to redefine the ways in “Scientists know a remarkable amount about which performance can function as both an arts- interested in the lithosphere and mantle beneath our feet. Beneath is an archetype of what exists far above us. We know the weight of led research practice and forum for engaged art-science-tech projects and Lance came to coordinating that show from a long history the moon. We know the composition of stars in learning. In both its goals and methods, Beneath of art-science projects. We asked him about insights and lessons learned. galaxies millions of light years away. But we know provides a model for transdisciplinary collabora- comparatively little about what lies just a few tion and public outreach in science.” dozen miles below our feet. That which is beneath “The project has three central goals: to make & DR.GHARAVI HAS been working at the inter- A Brief Anniversary of Time, a prior perfor- is our mystery and science is working to cast light current scientific research artful, accessible, and section of art, science, and engineering through mance project, represents well the role he was on the subject. Beneath takes audiences on a mul- compelling for the public; to create new visuali- collaborating with large teams typically of artists, hired at ASU to perform. “We worked with a sys- tisensory journey to the Earth’s deep interior.” zation tools that aid scientists in research, com- designers, scientists, engineers, and others to tem I call a universe in a box,” he said. “The team “Beneath’s fusion of science and live perfor- munication, and education; and to engage and make media rich works of performance. designed the performance for ASU’s flat screen mance features Christy Till, a geologist ballerina explore new models of collaboration between Like many others who had a stick-to-it-ness planetarium—the Marston Exploration Theater dancing catastrophic planetary cycles (see figure 2); artists and scientists.” during the first heyday of virtual reality (VR), that uses planetarium software called Sky-Skan. Ed Garnero, a bass-playing geophysicist interacting “The creative team behind Beneath fuses the- Lance brought the skills he acquired and fine- The Marston Exploration Theater software runs with his data through trip-hop bass-lines; and Pat- atre and science to tell a compelling story and tuned to various projects when hired by Arizona on a stack of ten servers and every known object rick Young, a belly-dancing theoretical astrophysi- communicate scientific research in an engaging State University (ASU) to be an artist working in the universe is plotted in four dimensions. cist embodying seismic waves. Audiences virtually and accessible way. It illustrates the dynamic with digital technologies. VR was not officially You can fly around in this universe anywhere visit the lab of Dan Shim, a mineral physicist who systems of the Earth while showing the ways in on the list of his job responsibilities but as Lance you want to go—even through time. It is a way uses diamonds in startling experiments, and talk which humans are connected to the immense suggested, “I realized it’s a matter of what you to experience the mind-bogglingly immense with Lindy Elkins-Tanton, the first woman to lead and ancient processes of our planet.” call VR. Many of those themes I explored trans- scope of our universe. I like to think of it as an a NASA mission beyond the Earth’s orbit.” “The best part of the collaboration was defi- ferred well into other digital technologies.” existential-crisis-machine.” As Lance suggests: nitely working with the people (see figure 3). My The show was a celebration of the 25th “Beneath is the product of a multiyear colla- main scientist collaborator, Ed Garnero, is a geo- anniversary of Stephen Hawking’s A Brief boration among a team of planetary scientists, physicist and a seismologist. He tries to under- Digital Object Identifier 10.1109/MCG.2019.2892863 History of Time (seefigure1)withamixofpre- theatre makers, performance artists, and media stand vibrations and the interior dynamics and Date of current version 22 March 2019. recorded video and live performance played designers based at ASU in collaboration with structures of the Earth. There’s a huge scientific

0272-1716 ß 2019 IEEE Published by the IEEE Computer Society IEEE Computer Graphics and Applications March/April 2019 8 www.computer.org/computingedge 9 41 39mcg02-campbell-2892863.3d (Style 5) 05-06-2019 17:44

Art on Graphics

“Christy Till studies rocks and magma; she is also a former professional ballerina and so dances expressively in the performance. Patrick Young is a semiprofessional belly dancer, who used his body in the show to demonstrate how seismic waves work. For the scientists who were live in the performance, the process was deeply moving. It was a way of bringing together different aspects of themselves.” When we asked about his opinions on sonification, Lance said: “I love sonification of data but you often don’t know what you are hearing. We took the sounds from an earthquake and increased the pitch so you could hear it. We turned out all the lights in the theater so people could experience this earthquake wave in the dark for a minute. It’s intense and intimidating, rather like someone banging on a trashcan. You feel the vibrations in your body.” “When we can ask questions of scientists that they weren’t expecting, or hadn’t thought of it Figure 2. Petrologist Christy Till dances in Beneath that way, we can help. Science is not just a bunch at the Marston Exploration Theater. Audiences view of data or a lot of numbers. Science is stories, and the stereoscopic media through 3-D glasses. thus, it depends on metaphor. That’s what we do Choreography by Liz Lerman. Stage direction by as artists, many of us, as storytellers—we deal in Erika Hughes, University of Portsmouth. Systems metaphor. Finding new metaphors with stories to design by Matthew Reagan, Obscura Digital; and Ian tell is a contribution to science.” Shelanskey, BRDG Studios. Media design by Jake “Some artists just use the outputs of science Pinholster, Arizona State University; Dallas Nichols; as fodder for art work. Some translate science Daniel Fine, University of Iowa; Alex Oliszewski, The for the public (as in A Brief Anniversary of Ohio State University; Miwa Matreyek; Boyd Branch, Time). Some get involved explicitly trying to University of Kent; and Elora Mastison, Arizona State show data in new ways. On very rare occasions, University. Sound design by Stephen Christensen, working with scientists, artist collaborations Arizona State University. Photo credit: Tim Trumble. can lead to advances in the science. That is (Used with permission.) always a useful goal. I have various projects where that is really a big focus, having gained project called EarthScope, one of the largest proj- the trust of scientists through a longer term ect ever funded by NSF, and they are rolling collaboration.” these seismometers all across North America Lance’s current work with Ars Robotica along with permanent ones mounted all over the speaks well to the potential of shifting toward a place—to try and get a 3-D picture of the Earth’s partnership of artists and scientists pursuing sci- interior. We took that data and created a 3-D ence together. As Lance explains: model of the Earth’s interior based on an enor- “The focus of the Ars Robotica initiative is mous spreadsheet of several terabytes of num- advancing robotics through art and design. I ini- bers. Ed is also a semiprofessional bass player. tially worked with a robot called Baxter created Through our software, he uses his bass to ‘play’ by Rethink Robotics, a company that makes his data, lighting up the different parts of anthropomorphic robots. I partnered with ASU’s the Earth’s interior depending on pitch (see Autonomous Systems Technologies Research figure 4).” and Integration Laboratory (ASTRIL), and it was

IEEE Computer Graphics and Applications 42 10 ComputingEdge August 2019 39mcg02-campbell-2892863.3d (Style 5) 05-06-2019 17:44 39mcg02-campbell-2892863.3d (Style 5) 05-06-2019 17:44

Art on Graphics

“Christy Till studies rocks and magma; she is very much about trying to get a also a former professional ballerina and so dan- robot to fill the needs of the proces expressively in the performance. Patrick duction we were creating (see Young is a semiprofessional belly dancer, who figure 5). We built some software used his body in the show to demonstrate how and an interface to control the seismic waves work. For the scientists who were robot and it was like an anthropo- live in the performance, the process was deeply morphic remote controlled car. At moving. It was a way of bringing together differ- the end of it, Srikanth Saripalli, the ent aspects of themselves.” lab’s Director, asked, ‘Can we keep When we asked about his opinions on sonifi- working together? Because work- cation, Lance said: ing with you has helped advance “I love sonification of data but you often don’t our research.’ With that Ars Robot- know what you are hearing. We took the sounds ica was born.” from an earthquake and increased the pitch so “I saw that many artists will you could hear it. We turned out all the lights in take the science and bend it to the theater so people could experience this the purposes of the art they are earthquake wave in the dark for a minute. It’s making. I thought, ‘What if we intense and intimidating, rather like someone put the interests of the research Figure 3. Beneath team at work. From left: Lance Gharavi, Ian Shelanskey, and Ed banging on a trashcan. You feel the vibrations in first and then make art to support Garnero discuss ways of visualizing and sonifying vast amounts of seismic data to your body.” the research goals?’ That’s what reveal the formations of the Earth’s deep interior in 3-D. Such visualizations will “When we can ask questions of scientists that Ars Robotica does. For instance, have a life beyond the performance as tools for scientists. Photo credit: Tim they weren’t expecting, or hadn’t thought of it now we are working to design a Trumble. (Used with permission.) Figure 2. Petrologist Christy Till dances in Beneath that way, we can help. Science is not just a bunch testbed for swarms of driverless at the Marston Exploration Theater. Audiences view of data or a lot of numbers. Science is stories, and cars and remotely operated the stereoscopic media through 3-D glasses. thus, it depends on metaphor. That’s what we do vehicles.” Choreography by Liz Lerman. Stage direction by as artists, many of us, as storytellers—we deal in “Initially I said to them, ‘you Erika Hughes, University of Portsmouth. Systems metaphor. Finding new metaphors with stories to can get your undergraduates and design by Matthew Reagan, Obscura Digital; and Ian tell is a contribution to science.” graduate students to build mod- Shelanskey, BRDG Studios. Media design by Jake “Some artists just use the outputs of science els for this work. You can go to Pinholster, Arizona State University; Dallas Nichols; as fodder for art work. Some translate science the nearest train store and pull Daniel Fine, University of Iowa; Alex Oliszewski, The for the public (as in A Brief Anniversary of out buildings and stuff. Why Ohio State University; Miwa Matreyek; Boyd Branch, Time). Some get involved explicitly trying to come to an artist?’ They were University of Kent; and Elora Mastison, Arizona State show data in new ways. On very rare occasions, like, ‘we don’t know.’ I said, University. Sound design by Stephen Christensen, working with scientists, artist collaborations ‘that’s a totally acceptable Arizona State University. Photo credit: Tim Trumble. can lead to advances in the science. That is answer. We can figure that out (Used with permission.) always a useful goal. I have various projects together.’ And so the project will where that is really a big focus, having gained be simultaneously a test bed, a project called EarthScope, one of the largest proj- the trust of scientists through a longer term laboratory, but also a perfor- ect ever funded by NSF, and they are rolling collaboration.” mance and art installation. It these seismometers all across North America Lance’s current work with Ars Robotica took a period of trust and work- along with permanent ones mounted all over the speaks well to the potential of shifting toward a ing together before we found a Figure 4. In Beneath, Heather Lee Harper, Lance Gharavi, and Ed Garnero place—to try and get a 3-D picture of the Earth’s partnership of artists and scientists pursuing sci- common process for advancing perform a trip-hop spoken word number about the vibrations in the Earth’s interior. interior. We took that data and created a 3-D ence together. As Lance explains: explicit goals to advance the Photo credit: Tim Trumble. (Used with permission.) model of the Earth’s interior based on an enor- “The focus of the Ars Robotica initiative is technology and science.” mous spreadsheet of several terabytes of num- advancing robotics through art and design. I ini- Lance’s work with robotics The hard part as a liaison is figuring out what bers. Ed is also a semiprofessional bass player. tially worked with a robot called Baxter created now focuses on creating a context or a site is the best way to inject the practice and training Through our software, he uses his bass to ‘play’ by Rethink Robotics, a company that makes for science. “But also a performance event, in of art into serving the science. Lance adds: his data, lighting up the different parts of anthropomorphic robots. I partnered with ASU’s the case with robots, that wants art to do “With Robotopolis, an upcoming project, one the Earth’s interior depending on pitch (see Autonomous Systems Technologies Research more. It has to serve a technical and utilitarian of the practical challenges was figuring out how figure 4).” and Integration Laboratory (ASTRIL), and it was function.” to fulfill the needs of the robotics lab while doing

IEEE Computer Graphics and Applications March/April 2019 10 www.computer.org/computingedge 11 43 39mcg02-campbell-2892863.3d (Style 5) 05-06-2019 17:44

Art on Graphics

the things that art needs to do. How am I going to create something that I feel pulls all my triggers as an artist on the way to constructing something useful for the lab?” “Robotopolis is a city for robots. We plan on taping out the ground plan the first of the year and once we confirm the ground plan is going to work for our needs, we’ll move forward with finalizing architecture and other aspects and start the build and complete the process of media design. There is a whole aspect of media design in this because we are going to hit Figure 5. Baxter performing in The Mirror. The ASTRIL lab that used Baxter has the city from several directions moved with its Director to another institution. Ars Robotica has now teamed with the with projections and map those Center for Human, Artificial Intelligence, and Robot Teaming, specifically to work with onto all the buildings—quite a swarm robotics. Instead of a single robot, they coordinate swarms of robots, like complex projection mapping bees, in the ground, air, or water. The robots can be large or tiny, but they work task.” together to perform tasks. Photo credit: Tim Trumble. (Used with permission.) Another project in the works worth tracking as it advances is Port of Mars. Lance suggests: “Some of the biggest challenges of human space exploration aren’t technological. They are social. How can we best sus- tain healthy human communities in space? What social systems and structures do we need? We have created a project to find solutions. It is a game- based social science experiment called Port of Mars. In the game, all players are members of the first Martian community and they have to support the goals of the community in surviving while also pursuing their individual goals. All the behaviors of the players are monitored and Figure 6. Brian Foley in Immerge, a large-scale transdisciplinary work of digital tracked. We will analyze their performance, a dramatic myth on the free movement of data and creativity. Featuring behavior to see what kinds of architectural projection, virtual life forms, and other media. Lead artist: Lance things lead to success and what Gharavi; media design: Jake Pinholster and David Tinapple; production design: leads to failure. What systems Anastasia Schneider, Brunella Provvidente, and Adam Vachon. At the Emerge and processes will people invent Festival 2012. Photo credit: Tim Trumble. (Used with permission.) to get along?”

IEEE Computer Graphics and Applications 44 12 ComputingEdge August 2019 39mcg02-campbell-2892863.3d (Style 5) 05-06-2019 17:44 39mcg02-campbell-2892863.3d (Style 5) 05-06-2019 17:44

Art on Graphics

the things that art needs to do. “We go into experiments in February and disciplines?’ It’s true there are different disci- How am I going to create some- March. The experiments were designed by a cou- plinary cultures, different languages, goals, and thing that I feel pulls all my trig- ple of applied mathematicians, Marco Janssen tools. Part of what’s exciting about such work is gers as an artist on the way to and Marty Anderies, doing work in the social sci- that difference. It’s a source of allure as opposed constructing something useful ences, especially commons issues. Also onboard to a problem. When you do interdisciplinary for the lab?” are game designers like Michael Yichao of Riot work, gathering people with such different skills “Robotopolis is a city for Games, planetary scientists like Mars expert and knowledge, everyone is a wizard to everyone robots. We plan on taping out Tanya Harrison, visual artists like Titus Lunter, else. But really, it’s not so difficult. We are all the ground plan the first of the and many other specialists. These relationships passionately curious. We are all skilled in asking year and once we confirm the will lead to useful contributions to the science, questions, seeking answers, and solving prob- ground plan is going to work for and also something fun and artful.” lems. We are all trained to work with a team our needs, we’ll move forward “The term ‘commons’ refers to any resource toward a goal based on limited input, with lim- with finalizing architecture and shared by a group. Earth’s ecosystem is a giant ited time and resources. I find the infectious other aspects and start the commons, for instance. The great thing about sense of wonder and excitement about the ques- build and complete the process Earth is we have such abundant resources. But tions and challenges bonds us and makes our dif- of media design. There is a now we are bumping up against some serious ferences feel negligible.” whole aspect of media design in ceilings—climate change, mass extinctions, etc. If “Mostly, I am interested in big stories and big this because we are going to hit we send people to Mars, they will be much more ideas. Science has them. Art gives them breath.” Figure 5. Baxter performing in The Mirror. The ASTRIL lab that used Baxter has the city from several directions dependent on shared resources and those resour- moved with its Director to another institution. Ars Robotica has now teamed with the with projections and map those ces will be much scarcer; there’s a very narrow Bruce Campbell is a faculty member of Web Design Interactivity at the Rhode Island School of Center for Human, Artificial Intelligence, and Robot Teaming, specifically to work with onto all the buildings—quite a margin for error. Before we spend the billions of þ Design. His research interests include ocean data visu- swarm robotics. Instead of a single robot, they coordinate swarms of robots, like complex projection mapping dollars and put people’s lives at risk, we had bet- alization and procedural design. He received the Ph.D. bees, in the ground, air, or water. The robots can be large or tiny, but they work task.” ter learn how to navigate commons dilemmas in degree in systems engineering from the University of together to perform tasks. Photo credit: Tim Trumble. (Used with permission.) Another project in the works such a hostile environment. Port of Mars is about Washington. Contact him at [email protected]. worth tracking as it advances is finding solutions to such dilemmas.” Port of Mars. Lance suggests: Collaborative projects involving the artistic Francesca Samsel is a research associate with the “Some of the biggest chal- community and the sciences, pursuing common Center for Agile Technology, University of Texas–Austin lenges of human space explora- ground and bridging professional norms, have and an artist-in-residence at the Los Alamos National tion aren’t technological. They required a lot of patience historically, as well as Laboratory. Contact her at fi[email protected]. are social. How can we best sus- the ability to bridge languages (see figure 6). In Contact department editor Bruce Campbell at tain healthy human communi- Lance’s case: [email protected] or department editor Fran- ties in space? What social “People always ask me, ‘How do you manage cesca Samsel at fi[email protected]. systems and structures do we collaborations like these across such diverse need? We have created a project to find solutions. It is a game- based social science experiment called Port of Mars. In the game, all players are members of the first Martian community and they have to support the goals of This article originally appeared in the community in surviving IEEE Computer Graphics and Applications, vol. 39, no. 2, 2019. while also pursuing their individual goals. All the behaviors of the players are monitored and Figure 6. Brian Foley in Immerge, a large-scale transdisciplinary work of digital tracked. We will analyze their performance, a dramatic myth on the free movement of data and creativity. Featuring behavior to see what kinds of architectural projection, virtual life forms, and other media. Lead artist: Lance things lead to success and what Gharavi; media design: Jake Pinholster and David Tinapple; production design: leads to failure. What systems Anastasia Schneider, Brunella Provvidente, and Adam Vachon. At the Emerge and processes will people invent Festival 2012. Photo credit: Tim Trumble. (Used with permission.) to get along?”

IEEE Computer Graphics and Applications March/April 2019 12 www.computer.org/computingedge 13 45 IEEE Security & Privacy is a bimonthly magazine communicating advances in security, privacy, and dependability in a way that is useful to a broad section of the professional community. The magazine provides articles with both a practical and research bent by the top thinkers in the ﬁ eld of security and privacy, along with case studies, surveys, tutorials, columns, and in-depth interviews. Topics include:

• Internet, software, hardware, and systems security • Legal and ethical issues and privacy concerns • Privacy-enhancing technologies • Data analytics for security and privacy • Usable security • Integrated security design methods • Security of critical infrastructures • Pedagogical and curricular issues in security education • Security issues in wireless and mobile networks • Real-world cryptography • Emerging technologies, operational resilience, and edge computing • Cybercrime and forensics, and much more www.computer.org/security

E-Currency and Fairness I Ransomware Defense I A National Cybersecurity Policy & PRIVACY SECURITY IEEE

Software and Cybersecurity ■ Big Data: Privacy Versus Accessibility ■ Resiliency in Cloud Computing ETHICS AI

Blockchain Technologies ■ e Fuzzing Revival ■ Cybersecurity for the Public Interest & PRIVACY SECURITY IEEE & PRIVACY SECURITY IEEE DIGITAL FORENSICS, PART 2 FORENSICS, DIGITAL SCREENING AUTOMATED AIRPORT AND PRIVACY VOLUME 17 VOLUME

VOLUME 16 VOLUME NUMBER 1

JANUARY/FEBRUARY 2019

NUMBER 3

January/February 2019 CYBERSECURITY AND WWW.COMPUTER.ORG/SECURITY Vol. 17, No. 1

VOLUME 17 VOLUME PRIVACY ISSUES IN BRAZIL

November/December 2018 Vol. 16, No. 6 NUMBER 2 MAY/JUNE 2018 MAY/JUNE

MARCH/APRIL 2019MARCH/APRIL

WWW.COMPUTER.ORG/SECURITY May/June 2018 Vol. 16, No. 3 March/April 2019 Join the IEEE ComputerWWW.COMPUTER.ORG/SECURITY SocietyVol. 17, No. 2 for subscription discounts today! www.computer.org/product/magazines/security-and-privacy Editor: Michiel van Genuchten VitalHealth Software IMPACT [email protected]

Editor: Les Hatton Oakwood Computing Associates [email protected]

A Comet Revisited

Lessons Learned from Philae’s Landing

András Balázs

From the Editors Two years ago, we published a column on the software involved in landing on a comet. It’s clear by now that although the landing itself was an impressive accom- plishment, not everything went as planned. We thank András Balázs for his thorough, honest analysis of what went wrong, what was done, and, importantly, what more could have been done. The software community could bene t from more such evaluations of the problems that so frequently occur in projects. —Michiel van Genuchten and Les Hatton

AFTER A 10-YEAR journey across the to the comet’s nucleus (see Figure 2a). its anomalous landing. It also man- Solar System and many maneuvers, The launch occurred approximately aged to maintain radio contact with the Rosetta spacecraft—carrying 500 million km from Earth, approx- Rosetta, which served as a relay sta- the Philae lander, a scienti c mini- imately 3 astronomical units (AUs) tion between Philae and the ight laboratory (see Figure 1)—smoothly from the sun, and 22.5 km from the operations control center on Earth. approached comet 67P/Churyumov- comet. Philae’s batteries supplied energy Gerasimenko. Rosetta then ew a Upon rst touching down after a for doing science on the comet’s sur- multitude of low- and high-altitude descent phase of 7 hours, the lander face for roughly 60 hours on the rst orbits around the comet, perform- couldn’t attach itself to the comet, run. Thereafter, the lander went into ing scienti c experiments and map- owing to unexpected, probably sys- hibernation owing to the disadvanta- ping the comet’s shape and surface in tematic failures in both parts of the geous thermal and solar-illumination detail never seen before. dual-redundant anchoring subsystem conditions at its parking site. After The Philae mission comprised and a malfunction of the nonredun- about 6 months of hibernation, these phases: dant hold-down thruster.1 How- Philae woke up at 1.8 AU from the ever, thanks mostly to the comet’s sun. Its central onboard computer 1. cruise (onboard Rosetta); gravitational attraction, Philae still (CDMS) then autonomously entered 2. separation, descent, and landing; completed its touchdown. After sev- the long-term-science phase. 3. rst comet science; eral hours of bouncing and uncon- 4. hibernation; and trolled tumbling, Philae reached its Philae’s Operation Control 5. long-term science. nal parking position, tilted to one A previous Impact department ar- side (see Figure 2b) roughly 1.2 km ticle2 and a more detailed reference On 12 November 2014, Rosetta from the planned landing site. Philae paper3 elaborate on the technical, initiated the ballistic delivery of Philae remained functionally intact, despite hardware, software, and operational

usually turns out to be impossible to get at and repair.5

Fault tolerance and the grace- ful degradation of vital subsystems in Philae, and particularly in the CDMS, were primary design goals. To help achieve these goals, we implemented a multilevel hardware and software scheme in the CDMS. The rst level involved constructing a “thing” (system) that “cannot possibly go wrong.” Such a system— for example, the CDMS, has these elements: FIGURE 1. The Philae lander explored comet 67P/Churyumov-Gerasimenko. (Source: DLR/Cologne; used with permission.) • redundant hardware and the proper architecture of redundant requirements and their implementa- driven solely by solar power and elements; tion in the CDMS. the rechargeable battery, required • self-repairing capability, sup- Once Philae landed, the starting comprehensive thermal control and ported by the hardware architec- times and durations of the radio vis- power ow management. ture and specic software; ibility windows between Philae and • robust fallback software with Rosetta depended on such factors When Good Intentions limited functionality (that is, as Rosetta’s ight track, the com- Face Reality communication capability); and et’s 12.6-hour rotation period, and As the International Academy of As- • reprogrammability of the act- Philae’s location and orientation on tronautics noted, the Philae mission ing software equipped with full the comet. Although these time win- was “the rst-ever trajectory devel- functionality. dows were nominally calculable, opment for a ballistic comet landing, Philae—in particular, its CDMS— the rst on-comet operations, and Such a scheme is in principle fault had to be prepared for deviations the rst cometary in-situ science col- tolerant. However, in practice, it from the predictions. We, as the lection.”4 However, not everything isn’t. What if, as an extreme example, CDMS hardware and software devel- went as planned. Here, I explore a buggy software version is acci- oper group, anticipated that the ight some lessons learned from our and dentally uplinked to such a system operators might have sporadic, time- the Philae team’s experiences with and falls into a deadlock without restricted opportunities for interven- problems that occurred in the hard- the system being able to communi- tion. The primary requirements were ware and software and in mission cate and thus receive the corrected exibility and onboard autonomy for operations control. software? Such a system, thought serial and parallel sequencing, and to be fault tolerant, would then fail. control and harmonization of the Lesson 1: No One Can Conquer Fate To enable intervention (“get at and operation of the ve subsystems and Originally, this statement from repair”) from the Earth, we extended nine scientic instruments. Douglas Adams inspired us to “con- the CDMS with triple-redundant The energy available for the sci- quer fate”: emergency telecommand decoders, ence programs from the primary along with other basic functions to and secondary batteries was lim- The major difference between a get out of deadlocks. ited. The CDMS also had to cope thing that might go wrong and a The irony of fate was that ex- with extreme environmental and op- thing that cannot possibly go wrong actly at the nal step of design, when erational conditions throughout the is that when a thing that cannot we thought we had achieved per- long-term-science phase. That phase, possibly go wrong goes wrong, it fect fault tolerance, we introduced

4890 IEEE SOFTWAREComputingEdge | WWW.COMPUTER.ORG/SOFTWARE | @IEEESOFTWARE August 2019 IMPACT

(a) (b)

FIGURE 2. Images from the Philae mission. (a) Comet 67P/Churyumov-Gerasimenko. (b) Philae on the comet. (Images by the Rosetta spacecraft’s navigation camera and Osiris camera.)

a source of error that remained un- passing all the environmental quali- • The touchdown sensor status noticed for a long time. A cyclic cation tests, the problem remained was cleared after each evaluation redundancy check code was sup- hidden and only came to light acci- attempt, eliminating the chances posed to protect the received tele- dentally during ight. Fortunately, for repeated evaluation to ex- commands from misinterpretation. the other processor unit automati- clude any transient errors. Nevertheless, the telecommand de- cally took over the missing one’s • Doubts arose as to whether the coders sometimes (but not often) functionality, demonstrating redun- sequence and timing of actions misinterpreted bit-serially transmit- dancy’s vital role in critical systems. in the anchoring algorithm ted cross-coupled telemetry packets Moreover, the unpowered proces- would be ef cient enough to (which were feedback noise caused sor unit could be brought to life— shoot the lander’s harpoons. by improper harnessing) as true ”thanks to Douglas Adams” (see emergency telecommands. The soft- Lesson 1)—by a hardware-decoded These issues gave the two teams ware workaround turned out to be emergency telecommand. Even strong incentives to revise the an- surprisingly simple (but energy con- more fortunately, the CDMS always choring strategy and control. We suming): both receiver units were worked outside the critical tempera- didn’t have access to Philae’s ight kept powered simultaneously, thus ture range during comet operations. hardware; all we had was the eliminating the receiver buffers’ vul- changeable software of the CDMS. nerability to cross-coupled noise. Lesson 3: When Even Redundancy With the telescope sensors in Philae’s Is Useless dumping mechanism as the basis, Lesson 2: Being Prepared Because the success of anchoring an alternative touchdown detection for the “Unbelievable” was a mission-critical requirement, algorithm was designed as a re- During the cruise phase, we faced an the anchoring-subsystem developer placement for the unusable detector. astonishing incident. In a very nar- team and we had devoted much By implementing an “Or-Majority” row temperature range around the effort to its design, quality assur- (Touchdown ϭ A or MajorityOf(B, CDMS’s thermal equilibrium (–27 ance, and implementation. In spite of C, D)) voting scheme for four (A, degrees C), one of the dual-redundant all this preparation, a string of sur- B, C, D) touchdown event sources processor units remained unpowered prising engineering issues occurred or paths, we made the detection of after being turned on. during the cruise phase: the touchdown event more robust As I mentioned before, the CDMS against transmission errors and false was required to be fully operable • One of the two touchdown event alerts. We together with the anchor under extreme environmental con- detectors—an accelerometer— team also completely reworked the ditions. Careful design and screen- was unusable because its sensi- anchoring-control software algo- ing turned this requirement into tivity range coincided with the rithm, tested it on the ground, and practice. But in spite of the CDMS vibration of Philae’s ywheel. uplinked it to the CDMS.

JULY/AUGUST 2018 | IEEE SOFTWARE 91 www.computer.org/computingedge 49 IMPACT

Even so, and even though Philae discarding—the originally planned Lesson 6: The Revenge of Missed properly detected the touchdown timelines of the prestored science Opportunties event, the anchoring failed, which sequences. By assessing risks and re- Throughout the rst-comet-science severely affected the rest of the prioritizing scienti c objectives, the phase, the entire Philae team was fully mission. Philae team was obliged to make a occupied with mastering Philae’s series of ad hoc decisions under time unexpected situation. So, it missed Lesson 4: Con icts Can Exist between pressure before each radio link ses- the opportunity to uplink science Safety and Science sion, in a continuous day–night work sequences and telecommands for For a long time, the Philae team regime of three to four days. subsystems control and autonomous felt that the delivery strategy should Before the on-comet phases of scienti c research tailored for the meet the requirements of both maxi- Philae, the team extensively ana- long-term-science phase. The rst mal scienti c throughput and bat- lyzed potential failure sources and telemetry data after six months of tery redundancy (parallel use of the prepared lengthy documents with hibernation reported that Philae was primary and secondary batteries). recovery procedures. After the mis- in good health, and no one reckoned The latter requirement would have sion’s active phases, several papers on a stepwise degradation of its vital necessitated a brief descent. were published on the impact of the hardware subsystems, particularly To not violate Rosetta’s safety failed anchoring, modeling Philae the redundant radio communication margins, the spacecraft ejected Philae as a mechanical system. You might units. relatively far from the comet, which ask, why didn’t the team do that be- In addition, as the risk grew of resulted in a prolonged descent. The forehand? During the cruise phase, completely losing Philae, the team candidate scienti c experiments for so many problems had emerged in arguably set the wrong priorities the descent required more energy than conjunction with Philae’s anchor- and adopted in part inadequate the secondary battery could provide. ing (see Lesson 3) that it was doubt- methods to achieve reliable con- So, science won out over safety in ful whether it would succeed. More tact with it. For example, because terms of a redundant battery supply. important, the possibility that Philae Philae was close to the sun at that For landing-site-targeting rea- would come to rest tilted to one side, time, the thermal and solar-power sons, regardless of whether Philae’s instead of being xed rmly on its conditions were excellent, and the main or backup ejection mechanism feet, appeared nonnegligible. control software for quick bat- was deployed, the team decided to And yet no one appreciated that tery recharging and even for an ex- set the changeable push-off velocity such a scenario was realistic but tended day–night work regime was of the main ejection mechanism to didn’t necessarily have immediate fa- obviously operable. Yet, the team be equal to that of the nonchange- tal consequences. The entire Philae had concerns that any premature able backup mechanism (0.19 m/s). team suffered from a sort of group- battery discharging might lead to The dominant factors in Philae’s think and didn’t take measures in an irreparable de cit. delivery were Rosetta’s attitude and advance to prepare both the opera- After not uplinking in advance orbital velocity vector. Compared tional ground segment and (in par- the set of telecommands, the team to these, Philae’s push-off velocity ticular) Philae’s onboard system. showed again that it hadn’t realized was nearly negligible. This suggests On the whole, the team could the urgent need to establish favorable that both ejection mechanisms could have exploited the available battery conditions—including own orbits have had a simpler but still robust energy more ef ciently by shortening of Rosetta—for commanding Philae design that produced an even lower the standby periods by relying more to collect and downlink scienti c nonchangeable push-off velocity. on onboard autonomy. That would data, instead of focusing on inves- have allowed additional science ex- tigating how and why the telecom- Lesson 5: Being Unprepared for the periments, even involving risk-taking munication units were degrading. It Conceivable actions if necessary. For example, we was of the utmost importance to get The unexpected situation of an un- could have adjusted Philae’s attitude a second set of science data for draw- anchored lander at an unknown for better solar illumination and to ing conclusions on comet evolution. location and attitude necessitated take images from additional surface The awareness that the lander did reshufing—and to a large extent elements of the comet. nothing useful in standby mode for

5092 IEEE SOFTWAREComputingEdge | WWW.COMPUTER.ORG/SOFTWARE | @IEEESOFTWARE August 2019 IMPACT

at least 2.5 months from wake-up ABOUT THE AUTHOR until its last contact with Earth was distressing. In the end, sadly, all the telecommunication units must have ANDRÁS BALÁZS is an embedded-hardware-and-software system broken down before the team could engineer at the Wigner Research Centre for Physics. Contact him at balazs deploy successful countermeasures. [email protected].

he Philae mission was a jump into the unknown. T Besides the standard systematic procedures and work ow, innovative, heuristic ideas had a comprehensive on-ground testing Onboard Computer,” IEEE Soft- specic role during software de- and validation. This shows why ware, vol. 33, no. 2, pp. 13–16. velopment. In the beginning, the software reprogrammability is so 3. A. Balázs et al., “Command and requirements to achieve all the sci- important. Data Management System (CDMS) entic objectives and the technical The Philae team exploited the of the Philae Lander,” Acta As- constraints in such a complex system onboard autonomy and exibil- tronautica, Aug.–Sept. 2016, pp. weren’t fully clear. This led to our ity in many respects, but not to the 105–117; doi:10.1007/s11214 inability to complete the ight soft- extent that would have been—in -006-9138-2. ware in the less than three years of retrospect—purposeful. In the end, 4. “The Laurels for Team Achievement hardware and software design and we might have been able to compen- Award 2015 to Philae Lander Mis- implementation before the space- sate for the relatively slow hardware sion,” Int’l Academy of Astronautics; craft launch. Afterward, during the degradation, at least to partly save https://iaaweb.org/content/view cruise phase, we had to prepare the the long-term-science phase. In hind- /143/243. system for many nominal opera- sight, this proved a bridge too far for 5. D. Adams, Mostly Harmless, Har- tional scenarios and emergency situ- all of us. mony Books, 1992. ations. All in all, we devoted more than 50 percent of development time References to making the system—not only 1. J.-P. Bibring et al., “The Rosetta the CDMS but also Philae—as fault Lander (‘Philae’) Investigations,” tolerant as possible. Space Science Reviews, vol. 128, Some of the lessons I described il- nos. 1–4, 2007, pp. 205–220; Read your subscriptions through the myCS lustrate how design and implemen- doi:10.1007/s11214-006-9138-2. This article originally appeared in publications portal at tation errors remained unrevealed 2. A. Baksa et al., “Software on a IEEEhttp://mycs.computer.org Software, vol. 35, no. 4, 2018. before the mission launch, despite Comet: The Philae Lander’s Central

Subscribe today for the latest in computational science and engineering research, news and analysis, CSE in education, and emerging technologies in the hard sciences. www.computer.org/cise

JULY/AUGUST 2018 | IEEE SOFTWARE 93 www.computer.org/computingedge 51 CAREER OPPORTUNITIES

IDX, LLC seeks a Computer Programmer The University of Alabama in Huntsville to work in Eugene, Oregon to address ar- The Department of Computer Science at The University of Alabama in Huntsville (UAH) chitectural issues of applications; analyze invites applicants for a tenure-track faculty position at the Assistant Professor level beginning application designs; modify software; de- January 2020. All applicants with a background in traditional areas of computer science will be sign, develop, modify proprietary appli- considered; however, special emphasis will be given to applicants with expertise in cybersecurity, gaming, software engineering, cloud computing, and systems related areas. cation software systems. Mail resume to: IDX, LLC, 100 East Broadway, Eugene, OR A Ph.D. in computer science or a closely related area is required. The successful candidate will have 97401, ATTN: Celeste Marshall. a strong academic background and be able to secure and perform funded research in areas typical for publication in well-regarded academic conference and journal venues. In addition, the candidate should embrace the opportunity to provide undergraduate education. The department has a strong commitment to excellence in teaching, research, and service; the candidate should have good communication skills, strong teaching potential, and research accomplishments. UAH is located in an expanding, high technology area, in close proximity to Cummings Research Park, the second largest research park in the nation and the fourth largest in the world. Nearby are the NASA Marshall Space Flight Center, the Army’s Redstone Arsenal, numerous Fortune 500 and high tech companies. UAH also has an array of research centers, including information technology and cybersecurity. In short, collaborative research opportunities are abundant, and many well-ed- ucated and highly technically skilled people are in the area. There is also access to excellent public schools and inexpensive housing. UAH has an enrollment of approximately 9,500 students. The Computer Science department offers BS, MS, and PhD degrees in Computer Science and contributes to interdisciplinary degrees. Faculty research interests are varied and include cybersecurity, mobile computing, data science, software engineering, visualization, graphics and game computing, multimedia, AI, image processing, pattern recognition, and distributed systems. Recent NSF gures indicate the university ranks 30th in the nation in overall federal research funding in computer science. Interested parties must submit a detailed resume with references to [email protected] or Chair, Search Committee, Dept. of Computer Science The University of Alabama in Huntsville, Huntsville, AL 35899. Quali ed female and minority candidates are encouraged to apply. Initial review of appli- WWW.COMPUTER.ORG cants will begin as they are received and continue until a suitable candidate is found. /COMPUTINGEDGE The University of Alabama in Huntsville is an af rmative action/equal opportunity employer/ minorities/ females/ veterans/ disabled. Please refer to log number: 19/20-545

IEEE TRANSACTIONS ON SUBMIT TODAY BIG DATA

SUBSCRIBE AND SUBMIT

For more information on paper submission, featured articles, calls for papers, and subscription links visit: www.computer.org/tbd

TBD is financially cosponsored by IEEE Computer Society, IEEE Communications Society, IEEE Computational Intelligence Society, IEEE Sensors Council, IEEE Consumer Electronics Society, IEEE Signal Processing Society, IEEE Systems, Man & Cybernetics Society, IEEE Systems Council, and IEEE Vehicular Technology Society

TBD is technically cosponsored by IEEE Control Systems Society, IEEE Photonics Society, IEEE Engineering in Medicine & Biology Society, IEEE Power & Energy Society, and IEEE Biometrics Council

www.computer.org/computingedge 52 Call for 2019 Major Awards Nominations

Deadline: 1 October 2019

Help Recognize Computing’s Most Prestigious Individuals IEEE Computer Society awards recognize outstanding achievements and highlight signiﬁ cant contributors in the teaching and R&D computing communities. All members of the profession are invited to nominate individuals they consider most eligible to receive international recognition through an appropriate society award.

Charles Babbage Award Computer Pioneer Award Certifi cate/$1,000 Silver Medal In recognition of signifi cant contributions in the fi eld Pioneering concepts and development of the of parallel computation. computing fi eld. Computer Entrepreneur Award W. Wallace McDowell Award Sterling Silver Goblet Certifi cate/$2,000 Vision and leadership resulting in the growth of some Recent theoretical, design, educational, practical, segment of the computer industry. or other tangible innovative contributions. Edward J. McCluskey Taylor L. Booth Award Technical Achievement Award Bronze Medal/$5,000 Certifi cate/$2,000 Contributions to computer science and Contributions to computer science or computer engineering education. technology. Computer Science & Engineering Harry H. Goode Memorial Award Undergraduate Teaching Award Bronze Medal/$2,000 Plaque/$2,000 Information sciences, including seminal ideas, Recognizes outstanding contributions to algorithms, computing directions, and concepts. undergraduate education. Hans Karlsson Award Harlan D. Mills Award Plaque/$2,000 Plaque/$3,000 Team leadership and achievement through Contributions to the practice of software engineering collaboration in computing standards. through the application of sound theory. Richard E. Merwin Award for Distinguished Service Bronze Medal/$5,000 Nomination Deadline Outstanding volunteer service to the profession at large, including service Submit your nomination by to the IEEE Computer Society. 1 October 2019 to www.computer.org/awards Contact us at [email protected] Education Awards Nominations Call for Award Nominations Deadline: 1 October 2019

Taylor L. Booth Education Award Computer Science and Engineering A bronze medal and $5,000 honorarium Undergraduate Teaching Award are awarded for an outstanding record A plaque, certifi cate, and a honorarium of in computer science and engineering $2,000 is awarded to recognize outstanding education. The individual must meet two contributions to undergraduate education or more of the following criteria in the through both teaching and service and computer science and engineering fi eld: for helping to maintain interest, increasing ■ Achieving recognition as a teacher the visibility of the society, and making of renown. a statement about the importance of undergraduate education. ■ Writing an infl uential text. ■ Leading, inspiring, or providing The award nomination requires a minimum signifi cant education content during the of three endorsements. creation of a curriculum in the fi eld. Read more award details at bit.ly/cs-eu ■ Inspiring others to a career in computer science and engineering education. Robert R. Kessler Two endorsements are required for an The University of Utah award nomination. 2019 Award Recipient Read more information at bit.ly/taylor-booth

Susan H. Rodger Duke University Nomination 2019 Award Recipient Deadline Submit your nomination by 1 October 2019 Contact us at [email protected] Harry H. Goode Memorial Award Call for Award Nominations Deadline: 1 October 2019

2019 Harry H. Goode Named for a pioneer and leader in the field of systems Memorial Award engineering, the Harry H. Goode Memorial Award Recipient was established to encourage further developments in and honor outstanding contributions to the field of information processing sciences. A bronze medal and $2,000 are awarded by the IEEE Computer Society on the basis of achievements in the information processing field.

About Harry H. Goode Marilyn C. Wolf One of the ﬁrst scientists to fully comprehend the Georgia Institute of Technology powers and abilities of computers, Harry H. Goode formulated many principles of systems engineering For contributions to and developed techniques for the design, analysis, and embedded, hardware- software codesign, and evaluation of large-scale systems. He was instrumental real-time computer in initiating early systems projects, including the vision systems. Typhoon computer and Whirlwind computer at MIT. He participated in the study that led to the creation of the Bomarc missile and conceived and developed the Air Defense Integrated System Project.

Nomination Requirements This award requires a minimum of three endorsements. Nominations are being accepted electronically by 1 October 2019 to bit.ly/harry-goode.

Questions? Visit bit.ly/harry-goode or contact [email protected] Richard E. Merwin Award for Distinguished Service

Call for Award Nominations Deadline: 1 October 2019

The Richard E. Merwin Award is the highest-level volunteer service award of the IEEE Computer Society for outstanding service to the profession at large, including signiﬁcant service to the IEEE Computer Society or its predecessor organizations.

About Richard Merwin 2018 Richard E. Richard Merwin was a pioneer in digital computer engineering Merwin Award who participated in the development of the ENIAC, MANIAC, Recipient for and STRETCH computers. Despite a busy and productive Distinguished technical career, Merwin found time to be active in professional Service societies, including the IEEE Computer Society, ACM, and AFIPS. His generosity of spirit and genuine helpfulness was an important element in the progress of the computer profession.

Award A bronze medal and $5,000 honorarium are awarded.

Sorel Reisman Presentation California State University, The Richard E. Merwin Award is presented at the Fullerton IEEE Computer Society’s Annual Awards Ceremony.

For sustained Nomination Requirements contributions, This award requires 3 endorsements. leadership, and service to the Nominations are being accepted electronically by IEEE Computer 1 October 2019 to bit.ly/richard-merwin Society, IEEE, and the computing Questions? profession at large. Email [email protected] IEEE Internet Computing delivers novel content from academic and industry experts on the latest developments and key trends in Internet technologies and applications. Written by and for both users and developers, the bimonthly magazine covers a wide range of topics, including:

• Applications • Architectures • Big data analytics • Cloud and edge computing • Information management • Middleware • Security and privacy • Standards • And much more In addition to peer-reviewed articles, IEEE Internet Computing features industry reports, surveys, tutorials, columns, and news. www.computer.org/internet

VOLUME 22, NUMBER 2 MARCH/APRIL 2018 IEEE INTERNET COMPUTING

July/August 2018 VOLUME 22, NUMBER 4 JULY/AUGUST 2018 IEEE INTERNET COMPUTING IEEE INTERNET COMPUTING Evolution of Rack-Scale Systems

January/February 2018 VOLUME 22, NUMBER 1 JANUARY/FEBRUARY 2018

May/June 2018 VOLUME 22, NUMBER 3 MAY/JUNE 2018 IoT-Enhanced Human Experience Connected and Autonomous Vehicles

IoT-Enhanced Human Experience Evolution of Rack-Scale Systems Volume 22 Number 1 Volume 22 Number 4

www.computer.org/internet

Healthcare Informatics and Privacy www.computer.org/internet

Connected and Autonomous Vehicles

Join the IEEE ComputerVolume 22 Number 3 Society

www.computer.org/internet for subscription discounts today!www.computer.org/internet www.computer.org/product/magazines/internet-computing Conference Calendar Questions? Contact [email protected]

EEE Computer Society conferences are valuable forums for learning on broad and dynamically I shifting topics from within the computing profession. With over 200 conferences featuring leading experts and thought leaders, we have an event that is right for you.

Find a region: Africa ■ Australia ◆ North America ◗ Asia ▲ Europe ● South America ★

SEPTEMBER OCTOBER 13 September 1 October • EWDTS (IEEE East-West Design & Test Sym- • MCSoC (IEEE 13th Int’l Symposium on Embed- posium) ● ded Multicore/Many-core Systems-on-Chip) ▲ 15 September 2 October • MODELS (ACM/IEEE 22nd Int’l Conf. on • DFT (IEEE Int’l Symposium on Defect and Model Driven Eng. Languages and Systems) Fault Tolerance in VLSI and Nanotechnology ● Systems) ● 19 September 12 October • AVSS (16th IEEE Int’l Conf. on Advanced • MICRO (52nd Annual IEEE/ACM Int’l Sympo- Video and Signal Based Surveillance) ▲ sium on Microarchitecture) ◗ • ESEM (ACM/IEEE Int’l Symposium on 14 October Empirical Software Eng. and Measurement) • ISMAR (IEEE Int’l Symposium on Mixed and ★ Augmented Reality) ▲ 23 September • LCN (IEEE 44th Conf. on Local Computer Net- • CLUSTER (IEEE Int’l Conf. on Cluster Com- works) ● puting) ◗ • VL/HCC (IEEE Symposium on Visual Lan- • PACT (28th Int’l Conf. on Parallel Architectures guages and Human-Centric Computing) ◗ and Compilation Techniques) ◗ 15 October • RE (IEEE 27th Int’l Requirements Eng. Conf.) • AIPR (IEEE Applied Imagery Pattern Recogni- ▲ tion Workshop) ◗ • SecDev (IEEE Secure Development) ◗ 16 October 25 September • FIE (IEEE Frontiers in Education Conf.) ◗ • HCC (IEEE Int’l Conf. on Humanized Comput- 20 October ing and Communication) ◗ • VIS (IEEE Visualization Conf.) ◗ 29 September 27 October • ICSME (IEEE Int’l Conf. on Software Mainte- • ICCV (IEEE/CVF Int’l Conf. on Computer nance and Evolution) ◗ Vision) ▲

58 August 2019 Published by the IEEE Computer Society 2469-7087/19/$33.00 © 2019 IEEE 28 October 16 December • EDOC (IEEE 23rd Int’l Enterprise Distrib- • CDKE (IEEE Int’l Conf. on Conversational Data uted Object Computing Conf.) ● & Knowledge Eng.) ◗ • ISSRE (IEEE 30th Int’l Symposium on Software Reliability Eng.) ● 2020

NOVEMBER January 4 November 13 January • ICTAI (IEEE 31st Int’l Conf. on Tools with Arti- • ICCPS (Int’l Conf. on Cyber-Physical Systems) ● ficial Intelligence) ◗ 7 November February • SEC (IEEE/ACM Symposium on Edge Com- 3 February puting) ◗ • ICSC (IEEE 14th Int’l Conf. on Semantic Com- 8 November puting) ◗ • ICDM (IEEE Int’l Conf. on Data Mining) ▲ 18 February 9 November • SANER (IEEE 27th Int’l Conf. on Software Anal- • FOCS (IEEE 60th Annual Symposium on Foun- ysis, Evolution and Reengineering) ◗ dations of Computer Science) ◗ 19 February 11 November • BigComp (IEEE Int’l Conf. on Big Data and • ASE (34th IEEE/ACM Int’l Conf. on Automated Smart Computing) ▲ Software Eng.) ◗ 22 February 17 November • CGO (IEEE/ACM Int’l Symposium on Code • ICCD (IEEE 37th Int’l Conf. on Computer Generation and Optimization) ◗ Design) ▲ • SC19 (SC19: Int’l Conf. for High Performance March Computing, Networking, Storage and Analy- 2 March sis) ◗ • WACV (IEEE Winter Conf. on Applications of 18 November Computer Vision) ◗ • BIBM (IEEE Int’l Conf. on Bioinformatics and 9 March Biomedicine) ◗ • DATE (Design, Automation & Test in Europe Conf. & Exhibition) ● DECEMBER • IRC (4th IEEE Int’l Conf. on Robotic Comput- 3 December ing) ▲ • RTSS (IEEE Real-Time Systems Symposium) ▲ 4 December • IREHI (IEEE Int’l Rural and Elderly Health Infor- matics Conf.) ■ 9 December • AIVR (IEEE Int’l Conf. on Artificial Intelligence Learn more about and Virtual Reality) ◗ • Big Data (IEEE Int’l Conf. on Big Data) ◗ IEEE Computer • ISM (IEEE Int’l Symposium on Multimedia) ◗ Society Conferences 10 December • ISSPIT (IEEE Int’l Symposium on Signal Pro- www.computer.org/conferences cessing and Information Technology) ▲

ce8con(all).indd 73 7/17/19 2:06 PM ipdps.org

IPDPS 2020 CALL FOR PAPERS

GENERAL CO-CHAIRS The five-day IPDPS program includes three days of contributed papers, Anu Bourgeois (Georgia State University, USA) invited speakers, industry participation, and student programs, framed by Ramachandran Vaidyanathan (Louisiana State University, two days of workshops with peer reviewed papers that complement and USA) broaden the main program. For full details, see www.ipdps.org. PROGRAM CHAIR Authors for the main conference are invited to submit manuscripts Yuanyuan Yang (NSF and Stony Brook University, USA) that present original unpublished research in all areas of parallel and distributed processing, including the development of experimental or PROGRAM AREA CHAIRS AND VICE CHAIRS commercial systems. Work focusing on emerging technologies and • Algorithms: interdisciplinary work covering multiple IPDPS areas are especially Xiaotie Deng (Peking University, China) and welcome. Topics of interest include: Songtao Guo (Chongqing University, China) • Parallel and distributed computing theory and algorithms • Architecture: (Algorithms) Ahmed Louri (George Washington University, USA) and Avinash Karanth • Experiments and practice in parallel and distributed (Ohio University, USA) • Experiments: computing (Experiments) Xin Yuan (Florida State University, USA) and • Programming models, compilers and runtimes for parallel Scott Pakin (Los Alamos National Laboratory, USA) applications and systems (Programming Models) • System Software: • System software and middleware for parallel and Alan Sussman (NSF and University of Maryland, College and distributed systems (System Software) Park, USA) Zhiling Lan (Illinois Institute of Technology, USA) • Architecture • Programming Models: • Multidisciplinary Rudolf Eigenmann (University of Delaware, USA) Zhiyuan Li (Purdue University, USA) • Abstracts due October 7, 2019 • Multidisciplinary: Manish Parashar • Submissions due October 14, 2019 (NSF and Rutgers University, USA) Ivona Brandic´ (Vienna University of Technology, Austria) • Preliminary decisions December 9, 2019 • Final submissions due January 6, 2020 New Orleans is one of the most eccentric and lively cities in the • Final notification January 20, 2020 world. Whatever your interests are, New Orleans has you covered. From its diverse culture, distinctive cuisine, rich history, colorful SPONSORED BY celebrations, live music, vibrant nightlife, and world-class restaurants, there is something for everyone. It is home to a number of engaging museums, including the World War II Museum, the New Orleans Museum of Art, the Historic Voodoo Museum, Mardi Gras World, and the Pharmacy Museum. A visit wouldn’t be complete without a swamp tour, Mississippi river cruise and a stop at Café du Monde for beignets. Join IPDPS at the Hilton New Orleans Riverside in 2020 to find out what makes New Orleans so unique and special.

IPDPS2020Ad.indd 1 7/22/19 8:57 PM

Security and Privacy &gt; Smart Homes &gt; Autonomous Vehicles &gt; Robotics

Security and Privacy > Smart Homes > Autonomous Vehicles > Robotics