How Abundance Changes Software Engineering > Multimedia Research

CLOUD COMPUTING

Also in this issue: > How Abundance Changes Software Engineering > Multimedia Research: What Is the Right Approach?

AUGUST 2017 www.computer.org Move Your Career Forward IEEE Computer Society Membership

Explore These Cloud Computing Resources

Cloud Computing Professional Development Courses The Computer Society offers three professional development courses on cloud computing:

• Cloud Computing in the Business Environment • Cloud Governance and Security • Cloud Economics, Migration, and Metrics

All three can help you in migrating your IT infrastructure to the Cloud.

IEEE Cloud Computing Initiative Helping accelerate cloud computing technology development and use, this IEEE initiative offers six interdependent resources: standards development; a Web portal (http://cloudcomputing.ieee.org); conferences; continuing education courses; publications; standards development; and a testbed.

IEEE Cloud Computing IEEE Cloud Computing is your best source for peer-reviewed articles showcasing innovative research, applications results, and case studies in all areas of cloud computing.

IEEE Transactions on Cloud Computing IEEE Transactions on Cloud Computing (TCC) publishes peer reviewed articles that provide innovative research ideas and applications results in all areas relating to cloud computing. Topics relating to novel theory, algorithms, performance analyses and applications of techniques relating to all areas of cloud computing are considered for the transactions.

FOR DIRECT LINKS TO THESE RESOURCES, VISIT The Community for Technology Leaders www.computer.org/edge-resources IEEE COMPUTER SOCIETY http://computer.org • +1 714 821 8380

STAFF

Editor Manager, Editorial Content Lee Garber Carrie Clark

Contributing Staff Senior Manager, Editorial Services Christine Anthony, Brian Brannon, Lori Cameron, Cathy Martin, Robin Baldwin Chris Nelson, Meghan O’Dell, Dennis Taylor, Rebecca Torres, Bonnie Wylie Director, Products and Services Evan Butterfield Production & Design Carmen Flores-Garvey, Monette Velasco, Jennie Zhu-Mai, Senior Advertising Coordinator Mark Bartosik Debbie Sims

Circulation: ComputingEdge (ISSN 2469-7087) is published monthly by the IEEE Computer Society. IEEE Headquarters, Three Park Avenue, 17th Floor, New York, NY 10016-5997; IEEE Computer Society Publications Office, 10662 Los Vaqueros Circle, Los Alamitos, CA 90720; voice +1 714 821 8380; fax +1 714 821 4010; IEEE Computer Society Headquarters, 2001 L Street NW, Suite 700, Washington, DC 20036. Postmaster: Send address changes to ComputingEdge-IEEE Membership Processing Dept., 445 Hoes Lane, Piscataway, NJ 08855. Periodicals Postage Paid at New York, New York, and at additional mailing offices. Printed in USA. Editorial: Unless otherwise stated, bylined articles, as well as product and service descriptions, reflect the author’s or firm’s opinion. Inclusion in ComputingEdge does not necessarily constitute endorsement by the IEEE or the Computer Society. All submissions are subject to editing for style, clarity, and space. Reuse Rights and Reprint Permissions: Educational or personal use of this material is permitted without fee, provided such use: 1) is not made for profit; 2) includes this notice and a full citation to the original work on the first page of the copy; and 3) does not imply IEEE endorsement of any third-party products or services. Authors and their companies are permitted to post the accepted version of IEEE-copyrighted material on their own Web servers without permission, provided that the IEEE copyright notice and a full citation to the original work appear on the first scree n of the posted copy. An accepted manuscript is a version which has been revised by the author to incorporate review suggestions, but not the published version with copy-editing, proofreading, and formatting added by IEEE. For more information, please go to: http://www.ieee .org/publications_standards/publications/rights/paperversionpolicy.html. Permission to reprint/republish this material for commercial, advertising, or promotional purposes or for creating new collective works for resale or redistribution must be obtained from IEEE by writing to the IEEE Intellectual Property Rights Office, 445 Hoes Lane, Piscataway, NJ 08854-4141 or pubs-permissions@ieee .org. Copyright © 2017 IEEE. All rights reserved. Abstracting and Library Use: Abstracting is permitted with credit to the source. Libraries are permitted to photocopy for private use of patrons, provided the per- copy fee indicated in the code at the bottom of the first page is paid through the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923. Unsubscribe: If you no longer wish to receive this ComputingEdge mailing, please email IEEE Computer Society Customer Service at [email protected] and type “unsubscribe ComputingEdge” in your subject line. IEEE prohibits discrimination, harassment, and bullying. For more information, visit www.ieee.org/web/aboutus/whatis/policies/p9-26.html.

IEEE Computer Society Magazine Editors in Chief

Computer IEEE Micro IEEE Intelligent Systems Sumi Helal, Lancaster University Lieven Eeckhout, Ghent V.S. Subrahmanian, University University of Maryland IEEE Software IEEE MultiMedia Diomidis Spinellis, Athens IEEE Computer Graphics Yong Rui, Lenovo Research University of Economics and and Applications and Technology Business L. Miguel Encarnação, ACT, Inc. IEEE Annals of the History IEEE Internet Computing IEEE Pervasive Computing of Computing M. Brian Blake, University of Maria Ebling, IBM T.J. Watson Nathan Ensmenger, Indiana Miami Research Center University Bloomington

IT Professional Computing in Science IEEE Cloud Computing San Murugesan, BRITE & Engineering Mazin Yousif, T-Systems Professional Services Jim X. Chen, George Mason International University IEEE Security & Privacy Ahmad-Reza Sadeghi, Technical University of Darmstadt www.computer.org/computingedge 1 AUGUST 2017 • VOLUME 3, NUMBER 8

THEME HERE 8 18 23 Open Source Quality of Cloud Extending the Solutions for Services: Expect Cloud to the Cloud Computing the Unexpected Network Edge 6 Spotlight on Transactions: Computer Engineers’ Challenges for the Next Decade: The Triangle of Power Density, Circuit Degradation, and Reliability JÖRG HENKEL AND PAOLO MONTUSCHI

7 Editor’s Note: Computing in the Cloud 8 Open Source Solutions for Cloud Computing G.R. GANGADHARAN

13 Standards at the Edge of the Cloud ALAN SILL

18 Quality of Cloud Services: Expect the Unexpected DAVID BERMBACH

23 Extending the Cloud to the Network Edge RUBEN S. MONTERO, ELISA ROJAS, ALFONSO A. CARRILLO, AND IGNACIO M. LLORENTE

28 Evidence and Forensics in the Cloud: Challenges and Future Research Directions KIM-KWANG RAYMOND CHOO, CHRISTIAN ESPOSITO, AND ANIELLO CASTIGLIONE

34 FocusStack: Orchestrating Edge Clouds Using Focus of Attention BRIAN AMENTO, ROBERT J. HALL, KAUSTUBH JOSHI, AND K. HAL PURDY

42 Connecting Fog and Cloud Computing DAVID S. LINTHICUM

46 How Abundance Changes Software Engineering DIOMIDIS SPINELLIS

50 Multimedia Research: What Is the Right Approach? ALAN HANJALIC

Departments 34 4 Magazine Roundup FocusStack: 53 Computing Careers: Cloud-Computing Careers Orchestrating Edge Clouds Using Focus of Attention Subscribe to ComputingEdge for free at www.computer.org/computingedge. CS FOCUS

Magazine Roundup

digit or character input, which has shortcomings. In “May the Force Be with You: The Future of Force-Sensitive Authenti- cation,” from Internet Comput- ing’s May/June 2017 issue, the authors examine the potential of force-PINs, which augment digit-PIN security by assigning a pressure value to each digit or character. he IEEE Computer issue on new computer-design Society’s lineup of 13 developments explores several of Computing in Science & Tpeer-reviewed technical these architectural approaches. Engineering magazines covers cutting-edge topics ranging from software IEEE Software Previous research suggests that design and computer graphics access and exposure to comput- to Internet computing and secu- Over time, software developers ing, social support, a sense of rity, from scientifi c applications have defi ned and used various belonging in computing, and a and machine intelligence to reliability-engineering models. computing identity all contrib- cloud migration and microchip Now, reliability engineering will ute to women pursuing comput- design. Here are highlights from have to adapt to today’s more con- ing as a fi eld of study or career. recent issues. nected world. The three articles in “Multiple Factors Converge to IEEE Software’s July/August 2017 Infl uence Women’s Persis- Computer special issue illustrate several tence in Computing: A Quali- ways this is already happening. tative Analysis,” from CiSE’s As we approach the limits of May/June 2017 issue, discusses Moore’s law, we increasingly rely IEEE Internet Computing a recent study that explores on computer architecture inno- what helps young women per- vations to scale performance. Most smartphone-authentication sist in computing despite the Computer’s August 2017 special schemes still rely on simple obstacles they encounter.

4 August 2017 Published by the IEEE Computer Society 2469-7087/17/$33.00 © 2017 IEEE IEEE Security & Privacy position-based dynamics for mesh- there has ever actually been such free cutting simulation. a utility. Our society is undergoing pervasive computerization and digita- IEEE Intelligent Systems IEEE Pervasive Computing lization, which is aff ecting many aspects of our personal and pro- Over the last two decades, man- The authors of “On-Device Mobile fessional lives. Such sweeping ufacturing has become more Phone Security Exploits Machine changes raise ethical issues that intelligent and data driven. The Learning,” from IEEE Pervasive computing professionals must manufacturing industry has used Computing’s April–June 2017 deal with. This is addressed in these capabilities to start focus- issue, off er a novel approach for “Ethics in Information Security,” ing on analyzing huge data sets protecting mobile devices from from IEEE S&P’s May/June 2017 from entire production lines over malware and keeping them from issue. long periods to identify perfor- connecting to malicious access mance, maintenance, and defect- points. The approach uses learn- IEEE Cloud Computing related issues. “Manufacturing ing techniques to analyze apps Analytics and Industrial Inter- and their behavior at runtime, and Data integration is still an after- net of Things,” from IEEE Intel- monitors the way devices associ- thought when it comes to cloud ligent Systems’ May/June 2017 ate with Wi-Fi access points. deployments, according to “Cloud issue, presents a related case Computing Changes Data Integra- study and looks at matters such IT Professional tion Forever: What’s Needed Right as data extraction, modeling, and Now,” from IEEE Cloud Comput- visualization. “Big Data and Big Money: The ing’s May/June 2017 issue. Enter- Role of Data in the Financial Sec- prises moving to the cloud tend IEEE MultiMedia tor,” from IT Pro’s May/June 2017 to focus on the move itself, not on issue, looks at big data’s relevance what they need to do once they get “JPEG at 25: Still Going Strong,” to the fi nancial sector. The article there. But they should also focus from IEEE MultiMedia’s April–June also outlines both adoption chal- on data integration because they 2017 issue, asks question about lenges and future opportunities. will have to use their cloud deploy- JPEG—which is celebrating its ment to share their information 25th anniversary as a standard this IEEE Micro among diff erent systems. year—such as where did it come from and what fundamental com- IEEE Micro’s May/June 2017 spe- IEEE Computer Graphics and ponents have given it longevity. cial issue features the top papers Applications from the various 2016 computer- IEEE Annals of the History of architecture conferences, as cho- Researchers have studied simula- Computing sen by a selection committee. tions of surgical cuts on deform- able bodies for more than two “At the Electronic Crossroads Computing Now decades. However, previous eff orts Once Again: The Myth of the Mod- based on fi nite element methods ern Computer Utility in the United The Computing Now website and mass spring meshes don’t States,” from IEEE Annals’ April– (computingnow.computer.org) scale to complex surgical sce- June 2017 issue, tries to determine features up-to-the-minute com- narios. “Effi cient Surgical Cutting whether past and present applica- puting news and blogs, along with with Position-Based Dynamics,” tions of the term “computer util- articles ranging from peer-reviewed from CG&A’s May/June 2017 issue, ity” share any commonalities. The research to opinion pieces by indus- presents a novel method that uses authors also question whether try leaders. www.computer.org/computingedge 5 SECTIONSPOTLIGHT TITLE ON TRANSACTIONS

arious techniques have been Computer Engineers’ investigated to mitigate, for example, recon gurable circuits’V aging problem. System-level Challenges for the Next approaches can also mitigate the problem at the OS level, where hardware resources are carefully selected to avoid Decade: The Triangle rapid circuit degradation. More accurate circuit-aging models are needed to allow high-level methods to operate of Power Density, on-chip systems at their real power- density limits (rather than costly con- servative ones), while continuously Circuit Degradation, and monitoring the tradeo between performance and thermally triggered aging mechanisms and their negative Reliability short- and long-term reliability effects. Involving all abstraction layers in the on-chip system’s design process Jörg Henkel, Karlsruhe Institute of Technology in this smart way will help extend DS. Paolo Montuschi, Polytechnic University of Turin The key is cross-layer approaches.

This installment highlighting the work published REFERENCES . H. Zhang et al., “Aging Resilience in IEEE Computer Society journals comes from and Fault Tolerance in Runtime Recon gurable Architectures,” IEEE IEEE Transactions on Computers. Trans. Computers, vol. , no. , , pp. –. or more than years, IEEE improved e ciency, power consump- . H. Khdr et al., “Power Density-Aware Transactions on Computers (TC) tion per area is sharply rising. The Resource Management for Hetero- has served the computing reason is that, after a long reign, DS geneous Tiled Multicores,” to be community with top-quality has ended because supply voltage has published in IEEE Trans. Computers, Fresearch contributions. We recently stopped scaling down. Some refer to vol. , no. , . played a stimulating game: we rolled this problem as “dark silicon,” infer- . S. Pagani et al., “Thermal Safe Power back + years, grabbed a few TC issues, ring that major parts of a chip would (TSP): E cient Power Budgeting for looked at the state of the art in a partic- need to remain idle (dark)—but expen- Heterogeneous Manycore Systems in ular eld, compared it with today’s pic- sive, highly integrated silicon clearly Dark Silicon,” IEEE Trans. Computers, ture, and re ected on the fact that “old” can’t stay idle. vol. , no. , , pp. ¢–. problems have been solved and “new” One promising solution is to tightly questions have emerged. First, there control power densities, operating JÖRG HENKEL is a professor at was Moore’s law. Then came Dennard close to or even temporarily exceeding Karlsruhe Institute of Technology. scaling (DS), which states that when recommended densities. To investi- Contact him at henkel@kit edu or moving from one technology node to gate the physical implications of high visit ces.itec.kit.edu/~henkel. the next, a transistor’s power density is power densities, we must distinguish constant—that is, independent of the among peak and average tempera- PAOLO MONTUSCHI is a profes- technology. Recently, DS was discontin- tures and temporal and spatial ther- sor of computer engineering at ued. So, what will happen in the future? mal gradients because they trigger Polytechnic University of Turin. It’s a fact that power density will be circuit-aging mechanisms such as Contact him at [email protected] or a major challenge for the foreseeable negative-bias temperature instability visit sta .polito.it/paolo.montuschi. future. Despite orders-of-magnitude and electromigration.

Computing in the Cloud

ince cloud computing began in earnest “Evidence and Forensics in the Cloud: Chal- about 20 years ago, researchers have lenges and Future Research Directions,” from Sadvanced the approach considerably. IEEE Cloud Computing, describes cloud forensics’ Cloud computing now involves technologies and challenges and opportunities. issues such as fog computing, the Internet of Things Managing resources in IoT edge devices (IoT), forensics in the cloud, edge computing, quality presents signifi cant challenges. The authors of of service, and standards. August’s ComputingEdge “FocusStack: Orchestrating Edge Clouds Using issue looks at these and other important matters. Focus of Attention,” from IEEE Internet Comput- The availability of many types of open source ing, propose managing edge devices as part of systems off ers aff ordable opportunities for organi- infrastructure-as-a-service clouds, employing their zations to build and adopt various kinds of cloud- FocusStack approach. computing environments, according to Computer’s The amount of data used in both cloud com- “Open Source Solutions for Cloud Computing.” puting and the IoT could easily grow to become Computing along the boundary between cloud unmanageable, which creates numerous prob- components and the world of humans and devices lems. “Connecting Fog and Cloud Computing,” entails connectedness that requires the development from IEEE Cloud Computing, looks at the problems of new standards, note the authors of IEEE Cloud and potential solutions. Computing’s “Standards at the Edge of the Cloud.” This ComputingEdge issue also includes arti- The author of IEEE Internet Computing’s “Qual- cles on topics other than cloud computing: ity of Cloud Services: Expect the Unexpected” presents experiences from several years of bench- • The ready availability of processing power is marking cloud services. He discusses how the altering the nature of software engineering, quality-related behaviors he observed aff ect cloud according to IEEE Software’s “How Abundance applications, for better or worse. Changes Software Engineering.” Computer’s “Extending the Cloud to the Net- • The author of IEEE MultiMedia’s “Multimedia work Edge” looks at the Telefónica telecommunica- Research: What Is the Right Approach?” asks tions company’s OnLife project, which virtualizes whether the multimedia community is suffi - the access network and gives cloud-computing ciently proactive in discussing how to select capabilities at the network edge to IoT application the correct research approaches for solving developers and content providers. various problems.

Open Source Solutions for Cloud Computing

G.R. Gangadharan, Institute for Development and Research in Banking Technology

The availability of many open source Communities of skilled volun- teers working together over the In- systems offers affordable opportunities for ternet collaboratively develop open organizations to build and adopt various types source, cloud-based infrastructure as a service (IaaS), platform as a of cloud computing environments. service (PaaS), software as a service (SaaS), and hypervisor systems, which continue to evolve and im- loud computing—which lets organizations prove. Their features and compatibility with existing in- access resources on demand via the Internet, frastructures have increased industry adoption by lead- rather than having to provide applications ing rms and start-ups. and services on their own—has emerged as I address these matters in this article, which is an Ca cost-e ective and e cient way to deploy IT solutions. abridged, updated version of a chapter that I co-wrote on However, commercial, proprietary cloud products are of- this topic published in Encyclopedia of Cloud Computing. ten expensive. Open source cloud-computing systems, on the other hand, o er cheaper, vendor-independent alter- OPEN SOURCE IS natives– that provide scalability, customizability, secu- IaaS provides customers with raw computing infrastruc- rity, interoperability, and easier migration. ture, including storage, processing, and networking

8 August 2017 Published by the IEEE Computer Society 2469-7087/17/$33.00 © 2017 IEEE 66 COMPUTER PUBLISHED BY THE IEEE COMPUTER SOCIETY 0018-9162/17/$33.00 © 2017 IEEE EDITOR SAN MURUGESAN CLOUD COVER BRITE Professional Services; [email protected]

TABLE 1. Comparison of open source infrastructure-as-a-service (IaaS) offerings. Criteria Eucalyptus OpenStack CloudStack

Ease of Amazon Web Services (AWS)– Automated deployment and Web-based management of server deployment and compatible API; web-based management using the open provisioning, hosts, storage, and management management console source Compass system other elements

Provisioning and Supported by cloud controller, Uses Heat Orchestration Supported by the CloudStack orchestration cluster controller, and node Template, via Representational orchestration engine controller State Transfer (REST) APIs to launch multiple composite-cloud applications

Monitoring and Supports tools like Ganglia and Supports tools like Nagios and Supports tools like Zenoss and alerts Nagios Ganglia Nagios Supported by Eucalyptus Stats and Supported by OpenStack Supports noti cations via email Amazon’s Cloudwatch Telemetry and management server

Interoperability Supports AWS API, EC2 (Elastic Supports RefStack and Citrix Supports NetScaler SD-WAN Compute Cloud), and S3 (Simple Systems’ NetScaler SD-WAN Storage Service)

Networking modes Edge, managed (virtual LAN/ API-driven networking service Basic (layer 3 ltering) zone and Open Source and services non-VLAN), virtual-private-cloud VLAN, ¡ at, generic routing advanced zone networking modes encapsulation, and virtual extensible LAN networking modes

Storage Allocation and deallocation of Supported by ephemeral le Primary storage: disk volumes Solutions for memory on demand by the storage system, block storage cinder, for guest virtual machines controller Object Storage (Swift), and shared associated with cluster l e s y s t e m Secondary storage: Network File Cloud Computing System-based storage Security Security groups (networking rules Components for identity Web sessions, Data Encryption applied to all associated instances provisioning, password Standard–encrypted tokens, G.R. Gangadharan, Institute for Development and Research in [layer 2 isolation]), identity access management, and authentication security groups Banking Technology control

Support Supported by knowledge base, Supported by Internet relay Supported by documentation, documentation, web-based issue chat (IRC), documentation and community support, and IRC The availability of many open source Communities of skilled volun- tracking, community forum, community forum, and vendor channel teers working together over the In- reference architectures, and support systems offers affordable opportunities for ternet collaboratively develop open expert support from Hewlett Packard Enterprise organizations to build and adopt various types source, cloud-based infrastructure as a service (IaaS), platform as a of cloud computing environments. service (PaaS), software as a service (SaaS), and hypervisor systems, resources. There are several popular major Windows and Linux distribu- Apache CloudStack (cloudstack which continue to evolve and im- open source IaaS systems (see Table ). tions; and integrates with Amazon’s S .apache.org) supports KVM, VM- loud computing—which lets organizations prove. Their features and compatibility with existing in- For example, Hewlett-Packard’s HPE (Simple Storage Service) and EC (Elas- ware’s vSphere, and Xen virtualiza- access resources on demand via the Internet, frastructures have increased industry adoption by lead- Helion Eucalyptus (www.eucalyptus tic Compute Cloud) public clouds. tion, and o ers a management server rather than having to provide applications ing rms and start-ups. .com)—elastic utility computing archi- OpenStack (www.openstack.org) with a web dashboard. and services on their own—has emerged as I address these matters in this article, which is an tecture for linking programs to useful o ers a modular architecture that Ca cost-e ective and e cient way to deploy IT solutions. abridged, updated version of a chapter that I co-wrote on systems—is a scalable IaaS framework provides a component-based way OPEN SOURCE P S However, commercial, proprietary cloud products are of- this topic published in Encyclopedia of Cloud Computing. that uses the Amazon Web Services to build clouds. It was developed by PaaS o ers development and middle- ten expensive. Open source cloud-computing systems, on (AWS) API to enable interoperability cloud-computing vendor Rackspace ware systems for designing and test- the other hand, o er cheaper, vendor-independent alter- OPEN SOURCE IS and cloud creation. It supports KVM Inc. and NASA, and is supported by ing software. There are several note- natives– that provide scalability, customizability, secu- IaaS provides customers with raw computing infrastruc- (kernel-based virtual machine), Xen, companies such as Hewlett-Packard, worthy open source PaaS platforms rity, interoperability, and easier migration. ture, including storage, processing, and networking and VMware virtualization; runs on IBM, and Intel. (see Table ).

TABLE 2. Comparison of open source platform-as-a-service (PaaS) solutions. Criteria Cloud Foundry Cloudify OpenShift

Ease of Provides services for easy Supports dynamic provisioning and Provides automatic deployment deployment development automatic resource scalability through Git pushes or the Red Hat Supports command-line interface Cloud (RHC) CLI tool (CLI) pushes and many languages

Security Provides account authentication as Supports role- and resource-based Uses rewalls, intrusion-detection an identity-management service authorization security systems, port monitoring, RPM (Red Hat’s RPM package manager) verication, and encrypted communication

Features Supports multiple languages Provides a simple orchestration Supports multiple languages and and frameworks with exible tool to congure and manage cloud databases congurations resources and to better support integration with external tools

Service Oers REST (Representational Supports third-party metering Provides microservice metering, metering State Transfer)-based metering and service for accounting of which measures units consumed at aggregation services via cf-abacus, pay-per-use resources the container level Cloud Foundry’s usage-metering engine

Resilience Provides virtual-machine Supports automatic resource Allows failovers and loose-service resurrection and cross availability- control coupling for resilience zone redundancy Allows both vertical (more resources) and horizontal (more instances) scaling via the open source HAProxy TCP/HTTP load balancer

VMware, Dell EMC, and General OPEN SOURCE SS environment. They also enable resource Electric developed Cloud Foundry SaaS oers software hosted on a pro- sharing. There are several leading open (www.cloudfoundry.org) as a self- vider’s infrastructure. Open source source hypervisors (see Table ). service application-execution engine, SaaS cloud systems allow rapid cus- KVM (www.linux-kvm.org) is a Li- automated deployment engine, and tomization and extension of the pro- nux kernel module that permits VM lifecycle manager, integrated with vided software. There are several hosting. It is a virtualization infra- various development tools. It uses a important open source SaaS cloud of- structure for the Linux kernel that scriptable command-line interface. ferings (see Table ). turns it into a hypervisor. Cloudify (getcloudify.org) is a Acquia (www.acquia.com) en- OpenVZ (openvz.org) allows OS- TOSCA (topology and orchestration ables hosting of the Drupal content- level virtualization by creating multi- specification for cloud applications)- management system on the Amazon ple secure, isolated Linux containers based cloud orchestration frame- EC cloud service to create a digital on a single server. This enables better work that models applications and foundation for delivering web content. server utilization and avoids applica- services, and automates their entire SuiteCRM (suitecrm.com) is a tion conicts. lifecycles. customer relationship management Xen (www.xenproject.org) uses a OpenShift (www.openshift.com), (CRM) application. microkernel to provide services that which open source software vendor Red Openbravo (openbravo.com) is enter- run a VM. It lets multiple OSs execute Hat developed, leverages both Kuber- prise resource planning (ERP) software. on the same hardware simultaneously netes, Google’s open source container and enables VM migration over a LAN. cluster manager, and Docker, an open OPEN SOURCE source system that automates the de- HYPERVISORS ployment of Linux applications within Hypervisors are the foundation of cloud he availability of many open containers. OpenShift adds DevOps computing, providing a way to create, source cloud oerings lets tools to improve deployed applications’ run, manage, and delete VMs on the organizations adopt a ho- development and maintenance. y without changing the hardware Tlistic cloud ecosystem in which

10 Computing Edge August 2017 68 COMPUTER WWW.COMPUTER.ORG/COMPUTER CLOUD COVER

TABLE 2. Comparison of open source platform-as-a-service (PaaS) solutions. TABLE 3. Comparison of open source software-as-a-service (SaaS) offerings. Criteria Cloud Foundry Cloudify OpenShift Criteria Acquia SuiteCRM Openbravo

Ease of Provides services for easy Supports dynamic provisioning and Provides automatic deployment Features and A content-management system A fully customizable customer A web-based enterprise deployment development automatic resource scalability through Git pushes or the Red Hat applicability that lets users add modules and relationship management resource-planning system that Supports command-line interface Cloud (RHC) CLI tool perform custom coding as needed system that lets users extend its automates most common business (CLI) pushes and many languages Provides a content-delivery functionality as needed processes network Security Provides account authentication as Supports role- and resource-based Uses rewalls, intrusion-detection an identity-management service authorization security systems, port monitoring, RPM Security Supports physical security, Supports role-based security Supports SSL and HTTPS (Red Hat’s RPM package manager) customer segregation, system- models with the conguration of verication, and encrypted access controls, OS and network security such as SSL and communication LAMP-stack security-patch HTTPS management, antivirus upload Provides OS security by including Features Supports multiple languages Provides a simple orchestration Supports multiple languages and scanning, le-system encryption, access control, le-system and frameworks with exible tool to congure and manage cloud databases SSL, HTTPS, data and physical encryption, and so on congurations resources and to better support media destruction, and logging integration with external tools Scalability Manages sudden trac spikes Uses the Network File System Uses model-view-controller Service Oers REST (Representational Supports third-party metering Provides microservice metering, programming to enable scalability metering State Transfer)-based metering and service for accounting of which measures units consumed at aggregation services via cf-abacus, pay-per-use resources the container level Support Includes community forum, Includes community forum and Includes community forum and Cloud Foundry’s usage-metering documentation, and Internet relay documentation documentation engine chat Resilience Provides virtual-machine Supports automatic resource Allows failovers and loose-service resurrection and cross availability- control coupling for resilience zone redundancy Allows both vertical (more resources) and horizontal (more Comparison of open source hypervisors. instances) scaling via the open TABLE 4. source HAProxy TCP/HTTP Criteria KVM OpenVZ Xen load balancer Supported hosts Supports x86 and x86-64 Supports x86 andx86-64 Available for ARM, IA-32, x86, and Ported to ARM, PowerPC, and x86-64 IA-64 VMware, Dell EMC, and General environment. They also enable resource OPEN SOURCE SS Features Provides emulation via QEMU, an Provides virtualization, Provides virtual machine Electric developed Cloud Foundry SaaS oers software hosted on a pro- sharing. There are several leading open open source hypervisor checkpointing, isolation, and migration over a LAN, hardware- (www.cloudfoundry.org) as a self- vider’s infrastructure. Open source source hypervisors (see Table ). Provides paravirtualization resource management assisted virtualization, and service application-execution engine, SaaS cloud systems allow rapid cus- KVM (www.linux-kvm.org) is a Li- paravirtualization automated deployment engine, and tomization and extension of the pro- nux kernel module that permits VM Supported guest OS Runs UNIX-like distributions, Supports only Linux distributions Runs most UNIX-like lifecycle manager, integrated with vided software. There are several hosting. It is a virtualization infra- Windows, OS X, Android, and distributions and runs Windows various development tools. It uses a important open source SaaS cloud of- structure for the Linux kernel that Solaris with virtualization support scriptable command-line interface. ferings (see Table ). turns it into a hypervisor. Support Includes forums and online Supported by online forums, as Includes a knowledge center, Cloudify (getcloudify.org) is a Acquia (www.acquia.com) en- OpenVZ (openvz.org) allows OS- tracking well as wiki and issue/source online forums, training, and paid TOSCA (topology and orchestration ables hosting of the Drupal content- level virtualization by creating multi- tracking via a Git repository Citrix Systems support specification for cloud applications)- management system on the Amazon ple secure, isolated Linux containers based cloud orchestration frame- EC cloud service to create a digital on a single server. This enables better work that models applications and foundation for delivering web content. server utilization and avoids applica- services, and automates their entire SuiteCRM (suitecrm.com) is a tion conicts. organizations can adopt dierent in these systems still limit adoption. REFERENCES lifecycles. customer relationship management Xen (www.xenproject.org) uses a solutions for dierent purposes, Thus, more research is needed in . I. Voras, B. Mihaljevic, and M. OpenShift (www.openshift.com), (CRM) application. microkernel to provide services that based on what’s best for them. Table these areas. Orlic, “Criteria for Evaluation of which open source software vendor Red Openbravo (openbravo.com) is enter- run a VM. It lets multiple OSs execute lists some of the organizations sup- Open Source Cloud Computing Hat developed, leverages both Kuber- prise resource planning (ERP) software. on the same hardware simultaneously porting open source cloud systems. ACKNOWLEDGMENTS Solutions,” Proc. rd Int’l Conf. In- netes, Google’s open source container and enables VM migration over a LAN. Open source cloud solutions of- I would like to thank University of Hy - formation Technology Interfaces (ITI cluster manager, and Docker, an open OPEN SOURCE fer freedom of reuse and promote derabad PhD student Deepnarayan Ti- ), , pp. –. source system that automates the de- HYPERVISORS innovation. Their vendor neutrality wari and Srinivas Komaragiri, a post- . P.T. Endo et al., “A Survey on Open ployment of Linux applications within Hypervisors are the foundation of cloud he availability of many open and interoperability make migration graduate student in banking technology Source Cloud Computing Solu- containers. OpenShift adds DevOps computing, providing a way to create, source cloud oerings lets cost-eective and easy. However, se- at the Institute for Development and Re- tions,” Proc. th Brazilian Symp. tools to improve deployed applications’ run, manage, and delete VMs on the organizations adopt a ho- curity issues, lack of service support, search in Banking Technology, for their Computer Networks and Distributed development and maintenance. y without changing the hardware Tlistic cloud ecosystem in which and the shortage of IT workers skilled useful insights. Systems (SBRC ), , pp. –.

www.computer.org/computingedge 11 68 COMPUTER WWW.COMPUTER.ORG/COMPUTER JANUARY 2017 69 CLOUD COVER

TABLE 5. Organizations supporting open source cloud systems. Category Open source cloud system Organizations

IaaS Eucalyptus Hewlett Packard Enterprise

OpenStack Best Buy, Bloomberg, Comcast, PayPal Holdings

CloudStack China Telecommunications, DataCentrix Holdings, University of Melbourne

PaaS CloudFoundry Cisco Systems, Hewlett Packard Enterprise, IBM, SAP

Cloudify GigaSpaces Technologies, VMware

OpenShift 6Fusion, Accenture, Vizuri

SaaS Acquia Australian government, BBC, Warner Music Group

SuiteCRM NHS England

Openbravo Decathlon, Home’s Up

Hypervisor KVM IBM, Linux

OpenVZ Virtuozzo

Xen Cavium, Intel

. C. Bryant, “A Guide to Open Source Cloud Computing Software.” Tom’s IT Pro, June ; www.tomsitpro .com/articles/open-source-cloud -computing-software,-.html. . G.R. Gangadharan et al., “Open- Source Cloud Software Solutions,” 2017 B. Ramakrishna Rau Award Encyclopedia of Cloud Computing, S. Call for Nominations Murugesan and I. Bojanova, eds., Wiley-IEEE Press, , pp. –. Honoring contributions to the computer microarchitecture field gotNew Deadline: flaws? 1 May 2017 Established in memory of Dr. B. (Bob) Ramakrishna Rau, the award recognizes his distinguished career in promoting and expanding the This article originally appeared in use of innovative computer microarchitecture techniques, including Computer, vol. 50, no. 1, 2017. his innovation in complier technology, his leadership in academic and industrial computer architecture, and his extremely high personal and ethical standards. WHO IS ELIGIBLE?: The candidate will have made an outstanding innovative contribution or contributions to microarchitecture, use of novel G.R. GANGADHARAN is an as- microarchitectural techniques or compiler/architecture interfacing. It is hoped, but not sociate professor at the Institute required, that the winner will have also contributed to the computer microarchitecture community through teaching, mentoring, or community service. for Development and Research in AWARD: Certificate and a $2,000 honorarium. Banking Technology, India. Contact PRESENTATION: AnnuallyFind presentedout more at the and ACM/IEEE get involved: International Symposium on him at [email protected]. Microarchitecture NOMINATION SUBMISSION:cybersecurity.ieee.org This award requires 3 endorsements. Nominations are being accepted electronically: www.computer.org/web/awards/rau CONTACT US: Send any award-related questions to [email protected] Read your subscriptions through the myCS www.computer.org/awards publications portal at http://mycs.computer.org

12 Computing Edge August 2017 70 COMPUTER WWW.COMPUTER.ORG/COMPUTER CLOUD CLOUD

CLOUD COMPUTING ASSUMES COMMUNI- CATION AMONG PARTICIPATING COMPO- NENTS. The boundary between the collection of Standards at these components and the world of humans and devices has acquired a set of names that encompass different concepts, including fog computing (implying a highly diffuse, distributed cloud), edge computing the Edge of the (implying a clean boundary between connected and non-connected devices), and the Internet of Things (IoT). These concepts all assume a degree of connectedness that requires development of standards. Cloud It’s intrinsically dif cult to restrain the scope of discussion when tackling topics related to the Internet of Things. The idea that a relatively small number of communication and automation methods can allow simple control over real-world devices is Even as individuals, people have a natural pre- compelling, and the power of this idea naturally leads disposition to pursue multiple options and to keep one to gloss over the many dif culties that come with their choices exible and variable. The Internet of implementing it. It’s good to look at some counter- Everything, if it comes to exist, will necessarily in- examples, therefore, from the outset. clude many such things chosen to be included in Even among humans, communication is not a this collection at various times and by different sets simple endeavor. Despite many attempts, some po- of people, all making choices according to their own litical, some altruistic, and most at their core eco- needs and circumstances. nomic, there has never been nor will there ever A large amount of device history and many most likely be a single standard spoken or written different physical and communication choices language that spans all of humanity and crowds all will be aggregated together. Some fog, edge, and other languages to non-existence. The closest we IoT aspects will be easier than others to include have come so far as a species may be HTML, and in the resulting aggregated collection, and some even this nearly-universal method shows the rapid will require special considerations. As covered evolution, fragmentation, and specialization that are elsewhere in this special issue, a wide variety characteristic of human endeavors. of sensors, inputs, and communication mecha- In the light of this historical failure, and for oth- nisms with an equally wide range of reliability er reasons that I’ll cover in this column, we should and security considerations will also have to be be modest in our expectations for a single unifying included. paradigm and a single simple set of standards to cover the concepts mentioned above.

The Internet of Everything A smooth intellectual transition can take place from observing that some real devices can be connected and automated to the assumption that everything can be treated as part of the same collection. It would not be correct, however, to assume that standards for communication protocols, hard- ALAN SILL ware, device management, data formats, security, or any of the other myriad aspects of the “things” Texas Tech University, in the IoT will all become uniform and simpli ed [email protected] on their own.

d2sta.indd 63 7/21/17 9:16 AM STANDARDS NOW

Table 1. A classiﬁcation of layers and settings for IoT, edge, and fog computing along with examples of relevant standards and protocols.

Layer Type Example Protocol, Standard, and/or Setting

Infrastructure 6LowPAN, IPv4/IPv6, RPL

Identiﬁcation EPC, uCode, IPv6, URIs

Communications / Transport Wiﬁ, Bluetooth, LPWAN

Discovery Physical Web, mDNS, DNS-SD

Data and Messaging Protocols MQTT, CoAP, AMQP, Websocket, Node

Device Management TR-069, OMA-DM

Semantic JSON-LD, Web Thing Model

Multi-layer Frameworks Alljoyn, IoTivity, Weave, Homekit

Security OTrP, X.509, Blockchain, OAuth, OpenID

Industry Vertical Connected Home, Industrial, Utility, Telecom

Fog, Edge, and Non-Centralized Computing these abbreviations. Some have been covered in These fog, edge, and IoT concepts share the basic previous columns in this series, and the rest can characteristic that they are not concentrated in a be found on the web site referenced. Instead, I’ll single location. They are intrinsically distributed, concentrate the rest of this column on standards with the characteristic assumption that they can in speci c to the edge of the cloud, and especially on principle be connected through intermediate mech- those that are receiving recent attention to adapt anisms. Such mechanisms may not be simple single them to such settings. interfaces, but may instead take place through multiple levels or layers, each of which is amenable to Edge-Speciﬁc Communication one or more standard speci cations. Technologies A useful breakdown of these multiple layers Communication technologies have experienced a has been compiled at the postscapes.com web site. burst of recent activity driven by the need to im- Again, because of history and the variety of com- prove speeds and reliability across a wide range of munication methods and devices, multiple protocols transmission methods. Among the standards that can be applicable at each layer. have seen rapid evolution are several in the IEEE A summary of these topics and typical stan- 802.x family of speci cations (www.ieee802.org). dards associated with each of them extended Although these are similarly named and differ in from the collection at this site is contained in designation only in the nal numbers and letters, Table 1. This collection is not at all exhaustive or they differ widely in data format and signaling be- definitive, but already serves to illustrate the va- havior and, for wireless speci cations, in frequency riety of existing specifications and considerations spectrum and physical range. for connecting devices and getting them to oper- For IoT applications, the standards that have ate in IoT settings. seen the most recent activity include 802.11ad, aimed To t the discussion of these topics in the to replace 802.11ac as the highest-speed short-range space available in this column, I won’t attempt to WLAN communications; 802.11af, which is being cover every topic in Table 1 or to expand each of proposed as a long-range wide-area protocol; and

14 Computing Edge August 2017 64 IEEE CLOUD COMPUTING WWW.COMPUTER.ORG/CLOUDCOMPUTING

d2sta.indd 64 7/21/17 9:52 AM STANDARDS NOW

Table 1. A classification of layers and settings for IoT, edge, and fog computing along with 802.11ah (also called “low power WiFi”), which is A wireless data networking technology based on examples of relevant standards and protocols. aimed at long-range but shorter-duration applica- the earlier Highway Addressable Remote Transduc- tions such as those for sensors and other sources of er Protocol (HART) digital instrumentation wired Layer Type Example Protocol, Standard, and/or Setting intermittent data. A standard designated as “WiFi” automation standard called WiHART (or Wire- Infrastructure 6LowPAN, IPv4/IPv6, RPL is differentiated from other wireless communica- lessHART) also emerged in 2004. It was adopted as tions in that it always incorporates use of the full IEC 62591 in 2010, which was replaced in 2016 by Identification EPC, uCode, IPv6, URIs TCP/IP protocols. an updated version.2 The radio communications are Communications / Transport Wifi, Bluetooth, LPWAN These various speci cations also use differ- de ned by IEEE 802.15.4, and operate as a mesh ent portions of the radio spectrum. For example, protocol in the 2.4 GHz band. Discovery Physical Web, mDNS, DNS-SD 802.11ad is designed for 60 GHz communications Data and Messaging Protocols MQTT, CoAP, AMQP, Websocket, Node using a region of the spectrum that has not been ex- Messaging Standards and Protocols ploited yet due to cost and technology limitations, Beyond the physical transmission layer, selection of Device Management TR-069, OMA-DM and that has not yet been agreed for use by interna- a data interchange method is also necessary. The Semantic JSON-LD, Web Thing Model tional standards bodies. most familiar of these is HTTP and its secure vari- Another, 802.11ah, uses portions of the spectrum ant (HTTPS), which are speci ed in a range of IETF Multi-layer Frameworks Alljoyn, IoTivity, Weave, Homekit between 54 and 790 MHz that have up to now been documents summarized at the working group web- Security OTrP, X.509, Blockchain, OAuth, OpenID used for broadcast television, and therefore must site (httpwg.org/specs). Several other application, be freed for other uses by individual governments transport, and link layer protocols are also useful. Industry Vertical Connected Home, Industrial, Utility, Telecom through regulatory processes. It uses the already- (see the May/June 2016 “Standards Now” column crowded 900 MHz band, which is in use also by com- for an overview of the de nitions of these network- peting approaches including some non-standardized ing layers). 3 proprietary WiFi devices. For communications that can be intermittent Fog, Edge, and Non-Centralized Computing these abbreviations. Some have been covered in Similar evolution is taking place in the Blue- or don’t have to be completely received, the User These fog, edge, and IoT concepts share the basic previous columns in this series, and the rest can tooth family of speci cations. Low-energy Bluetooth Datagram Protocol (UDP)4 is a useful method to characteristic that they are not concentrated in a be found on the web site referenced. Instead, I’ll is already built into almost all recent-generation carry out Internet communications. It can also be single location. They are intrinsically distributed, concentrate the rest of this column on standards smartphones, for example. Its advantages of low used to carry out IP communications in situations with the characteristic assumption that they can in speci c to the edge of the cloud, and especially on power requirements and inexpensive chip sets are in which handshaking and veri cation of receipt of principle be connected through intermediate mech- those that are receiving recent attention to adapt counterbalanced by limited range and complica- the individual message packets aren’t necessary. Al- anisms. Such mechanisms may not be simple single them to such settings. tions involving pairing and coordination of key sets ternatives to UDP are TCP and the Stream Control interfaces, but may instead take place through mul- among devices. Transmission Protocol (SCTP).5 tiple levels or layers, each of which is amenable to Edge-Specific Communication Emerging approaches to this problem com- Several IETF documents form the basis of the one or more standard speci cations. Technologies bine Bluetooth access to local devices that serve much more complex set of speci cations underlying A useful breakdown of these multiple layers Communication technologies have experienced a as bridges to other communication technologies. the Transmission Control Protocol (TCP)6, which has been compiled at the postscapes.com web site. burst of recent activity driven by the need to im- The Bluetooth standard itself is evolving to include continues to receive ongoing attention from the In- Again, because of history and the variety of com- prove speeds and reliability across a wide range of other variants, such as Bluetooth 4.2, which uti- ternet community due to its importance in various munication methods and devices, multiple protocols transmission methods. Among the standards that lizes the Internet Engineering Task Force (IETF) settings. can be applicable at each layer. have seen rapid evolution are several in the IEEE IPv6/6LoWPAN protocol1 to transmit IPv6 packets Methods to handle publish/subscribe messag- A summary of these topics and typical stan- 802.x family of speci cations (www.ieee802.org). and to form corresponding IPv6 link-local addresses ing, such as the Message Queuing Telemetry Trans- dards associated with each of them extended Although these are similarly named and differ in with stateless auto-con gured addresses on IEEE port (MQTT)7, can have advantages compared to from the collection at this site is contained in designation only in the nal numbers and letters, 802.15.4 networks. the previously described protocols when used for Table 1. This collection is not at all exhaustive or they differ widely in data format and signaling be- Cellular and mesh network communication ap- machine-to-machine communication at high speeds. definitive, but already serves to illustrate the va- havior and, for wireless speci cations, in frequency proaches are also being applied to machine-to- The Constrained Application Protocol (CoAP)8, riety of existing specifications and considerations spectrum and physical range. machine communications, and to overlay networks another manufacturing-relevant specialized trans- for connecting devices and getting them to oper- For IoT applications, the standards that have that can extend and enhance the range and reliability fer standard, provides a “request/response interac- ate in IoT settings. seen the most recent activity include 802.11ad, aimed of other networking methods. A wide variety of other tion model between application endpoints, supports To t the discussion of these topics in the to replace 802.11ac as the highest-speed short-range communication technologies aimed at reducing the built-in discovery of services and resources, and in- space available in this column, I won’t attempt to WLAN communications; 802.11af, which is being complexity and power requirements for dedicated includes key concepts of the Web such as URIs and cover every topic in Table 1 or to expand each of proposed as a long-range wide-area protocol; and dustrial IoT applications is also being pursued. Internet media types. CoAP is designed to easily

www.computer.org/computingedge 15 64 IEEE CLOUD COMPUTING WWW.COMPUTER.ORG/CLOUDCOMPUTING MARCH/APRIL 2017 IEEE CLOUD COMPUTING 65

d2sta.indd 64 7/21/17 9:52 AM d2sta.indd 65 7/21/17 9:16 AM STANDARDS NOW

interface with HTTP for integration with the Web An owner of a given device might want one set while meeting specialized requirements such as of restrictions to apply on one day to a given setting, multicast support, very low overhead, and simplicity but decide to change this to a different set of users for constrained environments.” or conditions on a different day or even within the The Advanced Message Queuing Protocol same day based on personal whim or variable needs. (AMQP)9 is also a middleware messaging standard This characteristic of security—that it is not a static set. It can be applied using either publish/subscribe concept but instead can be mutable and subject to or point-to-point communication patterns. AMQP complex decision-making characteristics—strikes has a layered architecture and is organized into dif- me as more important than the technical details of ferent parts to re ect that architecture. specic security protocols, which are well studied. A The Data Distribution Service (DDS)10 and re- larger discussion of this topic will have to wait until lated DDS Data Local Reconstruction Layer (DDS- a future issue. DLRL) specications handle data interchange tasks related to IoT systems. Unlike the other protocols mentioned here, DDS can handle content-aware net- AS ALWAYS, THIS DISCUSSION ONLY REP- work routing, data prioritization by transport priori- RESENTS MY OWN VIEWPOINT. I’d like to ties, and both unicast and multicast communications hear your opinions and experience in this area. within the methods dened by the standard set itself. I’m sure other readers of the magazine would also The most popular data formats in cloud com- appreciate additional information on this topic. puting are JavaScript Object Notation (JSON) and Please respond with your input on this or previous XML. JSON shows signicant evidence of adoption columns. Please include news you think the com- beyond the context of the JavaScript language, and munity should know about in the general areas of may outlast it in the long run. For IoT and manufac- cloud standards, compliance, or related topics. I’m turing settings, another interesting renement is the happy to review ideas for potential submissions to Sensor Network Object Notation (SNON)11, which the magazine or for proposed guest columns. I can is a representation based on JSON that includes be reached for this purpose at alan.sill@standards some predened elds that are especially useful in -now.org. dealing with sensor data. XML continues to receive attention and to be References adapted to different IoT-related settings. The XML- 1. IETF Datatracker. “Internet Engineering Task based Extensible Messaging and Presence Protocol Force (IETF) IPv6/6LoWPAN protocol,” https:// (XMPP) is designed for message-oriented middleware datatracker.ietf.org/wg/6lowpan/documents. communications (see http://xmpp.org/extensions). 2. Industrial Electrotechnical Commission, 2016; Beyond its applications to human-oriented communi- https://webstore.iec.ch/publication/24433. cations, XMPP is also used in smart electrical grid 3. Sill, Alan, “Standards Underlying Cloud applications and a variety of industrial applications. Network,”IEEE Cloud Computing, vol. 3, no. 3, Several extensions directly oriented toward use in IoT 2016, pp. 76–80. settings were published in late 2015. 4. “User Datagram Protocol (UDP),” https://tools. ietf.org/html/rfc768 Security in Edge and Distributed Settings 5. “Stream Control Transmission Protocol (SCTP),” Because of the huge range in types of input, physical https://tools.ietf.org/html/rfc4960 scale, frequency of communication and variety of us- 6. “Transmission Control Protocol,” https://tools. ers, it is nearly impossible to summarize the security ietf.org/html/rfc7414 considerations for IoT, fog and edge computing within 7. OASIS, 2017; “Message Queuing Telemetry a single set of paradigms. The variable that is hardest Transport (MQTT),” https://www.oasis-open to control, it seems to me, will be the degree to which .org/committees/mqtt/ human users wish to change their minds about the 8. “Constrained Application Protocol (CoAP),” security perimeter that applies to a given function. https://tools.ietf.org/html/rfc7252

16 Computing Edge August 2017 66 IEEE CLOUD COMPUTING WWW.COMPUTER.ORG/CLOUDCOMPUTING

d2sta.indd 66 7/21/17 9:16 AM STANDARDS NOW

interface with HTTP for integration with the Web An owner of a given device might want one set 9. “Advanced Message Queuing Protocol (AMQP),” for the Open Grid Forum and is an active member while meeting specialized requirements such as of restrictions to apply on one day to a given setting, https://www.amqp.org of IEEE, the Distributed Management Task Force, multicast support, very low overhead, and simplicity but decide to change this to a different set of users 10. Object Management Group, 2017; “Data Dis- and other cloud standards working groups, and for constrained environments.” or conditions on a different day or even within the tribution Service (DDS),” www.omg.org/spec/ serves on national and international computing The Advanced Message Queuing Protocol same day based on personal whim or variable needs. DDS standards roadmap committees. For further details, (AMQP)9 is also a middleware messaging standard This characteristic of security—that it is not a static 11. “Sensor Network Object Notation (SNON),” www visit http://nsfcac.org or contact him at alan.sill set. It can be applied using either publish/subscribe concept but instead can be mutable and subject to .snon.org @standards-now.org. or point-to-point communication patterns. AMQP complex decision-making characteristics—strikes has a layered architecture and is organized into dif- me as more important than the technical details of ferent parts to re ect that architecture. specic security protocols, which are well studied. A ALAN SILL is senior director of the High Perfor- 10 The Data Distribution Service (DDS) and re- larger discussion of this topic will have to wait until mance Computing Center and adjunct professor of This article originally appeared in lated DDS Data Local Reconstruction Layer (DDS- a future issue. physics at Texas Tech University. He also co-directs the IEEE Cloud Computing, vol. 4, no. 2, 2017. DLRL) specications handle data interchange tasks US National Science Foundation’s multi-university related to IoT systems. Unlike the other protocols “Cloud and Autonomic Computing” industry/univer- mentioned here, DDS can handle content-aware net- AS ALWAYS, THIS DISCUSSION ONLY REP- sity cooperative research center, and holds a posi- work routing, data prioritization by transport priori- RESENTS MY OWN VIEWPOINT. I’d like to tion as visiting professor of distributed computing at Read your subscriptions through the myCS publications portal at ties, and both unicast and multicast communications hear your opinions and experience in this area. the University of Derby. Sill has a PhD in physics http://mycs.computer.org. within the methods dened by the standard set itself. I’m sure other readers of the magazine would also from American University. He serves as president The most popular data formats in cloud com- appreciate additional information on this topic. puting are JavaScript Object Notation (JSON) and Please respond with your input on this or previous XML. JSON shows signicant evidence of adoption columns. Please include news you think the com- beyond the context of the JavaScript language, and munity should know about in the general areas of may outlast it in the long run. For IoT and manufac- cloud standards, compliance, or related topics. I’m turing settings, another interesting renement is the happy to review ideas for potential submissions to Sensor Network Object Notation (SNON)11, which the magazine or for proposed guest columns. I can is a representation based on JSON that includes be reached for this purpose at alan.sill@standards Are Enemy Hackers Slipping some predened elds that are especially useful in -now.org. through Your Team’s Defenses? dealing with sensor data. XML continues to receive attention and to be References Protect Your Organization adapted to different IoT-related settings. The XML- 1. IETF Datatracker. “Internet Engineering Task from Hackers based Extensible Messaging and Presence Protocol Force (IETF) IPv6/6LoWPAN protocol,” https:// by Thinking Like Them (XMPP) is designed for message-oriented middleware datatracker.ietf.org/wg/6lowpan/documents. communications (see http://xmpp.org/extensions). 2. Industrial Electrotechnical Commission, 2016; Take Our E-Learning Courses Beyond its applications to human-oriented communi- https://webstore.iec.ch/publication/24433. in the Art of Hacking cations, XMPP is also used in smart electrical grid 3. Sill, Alan, “Standards Underlying Cloud applications and a variety of industrial applications. Network,”IEEE Cloud Computing, vol. 3, no. 3, Several extensions directly oriented toward use in IoT 2016, pp. 76–80. settings were published in late 2015. 4. “User Datagram Protocol (UDP),” https://tools. You and your staff can take these courses where you are and at your own pace, getting hands-on, real- ietf.org/html/rfc768 world training that you can put to work immediately. Security in Edge and Distributed Settings 5. “Stream Control Transmission Protocol (SCTP),” Because of the huge range in types of input, physical https://tools.ietf.org/html/rfc4960 www.computer.org/artofhacking scale, frequency of communication and variety of us- 6. “Transmission Control Protocol,” https://tools. ers, it is nearly impossible to summarize the security ietf.org/html/rfc7414 considerations for IoT, fog and edge computing within 7. OASIS, 2017; “Message Queuing Telemetry a single set of paradigms. The variable that is hardest Transport (MQTT),” https://www.oasis-open to control, it seems to me, will be the degree to which .org/committees/mqtt/ human users wish to change their minds about the 8. “Constrained Application Protocol (CoAP),” security perimeter that applies to a given function. https://tools.ietf.org/html/rfc7252

www.computer.org/computingedge 17 66 IEEE CLOUD COMPUTING WWW.COMPUTER.ORG/CLOUDCOMPUTING MARCH/APRIL 2017 IEEE CLOUD COMPUTING 67

d2sta.indd 66 7/21/17 9:16 AM d2sta.indd 67 7/21/17 9:16 AM View from the Cloud Editor: George Pallis • [email protected] Quality of Cloud Services: Expect the Unexpected

David Bermbach • TU Berlin

Here, the author presents a number of experiences from several years of benchmarking cloud services. He discusses how the respectively observed quality behavior would have affected cloud applications or how cloud consumers could use the behavior to their advantage.

n the last few years, cloud computing has and to get insights into the actual quality of cloud found widespread adoption in companies of all services is through cloud service benchmarking I sizes. A core focus of these cloud consumers (see the related sidebar). is typically on cost savings, convenience of man- Thus, here I report on a number of experiences aged services, and on-demand capacity changes. from several years of benchmarking cloud services The quality of cloud services, however, is usually and briey discuss how the respectively observed taken “as-is”: based on documentation, advertise- quality behavior would have affected cloud appli- ments, but also past experiences from a non-cloud cations or how cloud consumers could use the world, cloud consumers typically have implicit behavior to their advantage. This article should assumptions. For instance, an eventually consis- be seen as a call: Don’t make assumptions, make tent storage system that claims to have triple rep- experiments. For this purpose, I also sketch out lication in close-by datacenters with high-speed how cloud consumers can use cloud service bench- network interconnection can be assumed to show marking in their application lifecycle. low millisecond staleness – that is, relatively good consistency behavior. As another example, virtual Performance of Virtual Machines machines (VMs) that come in sizes S, M, L, and XL can In 2011, Alexander Lenk and colleagues1 ran a num- be expected to grow in capacity for all resources – ber of performance experiments on top of Amazon network bandwidth, disk storage volume, RAM EC2 instances using the Phoronix test suite. Soon, size, CPU clock speed, or cores – when choosing a they discovered that they had a twin peak distri- bigger instance type. However, in both examples bution of compute performance results: For every this isn’t always the case – in fact, cloud consum- benchmark in the suite, there were some machines ers should always expect the unexpected. that showed a very good performance while oth- This isn’t meant to imply that the unexpected ers showed a rather poor performance. Through is always bad for the cloud consumer. Actually, in-depth analysis of results, they realized that the observable quality behavior is typically much bet- performance variance stayed constant over time ter than what’s guaranteed, for example, when and could also not be attributed to different instance cloud consumers only plan for what’s guaranteed, sizes; instead, Amazon had obviously deployed two they’ll never tap the full potential of cloud ser- different CPU types (AMD Opteron and Intel Xeon). vices. Furthermore, violations of guarantees might Depending on the CPU type, the machines either still occur. Therefore, not knowing about the qual- excelled at oating point or at integer operations. ity of the cloud services used will generally either However, both types came with the same price tag. lead to unexpected negative surprises through Furthermore, the performance difference could only (seemingly) obscure application behavior or to be identi ed after having provisioned the instance. inef ciencies when cloud consumers design their Obviously, this isn’t the anticipated behavior that applications only based on explicitly provided a cloud consumer would expect from a cloud pro- quality guarantees. The only way to avoid this vider that offers a standardized product.

68 Published by the IEEE Computer Society 1089-7801/17/$33.00 © 2017 IEEE IEEE INTERNET COMPUTING 18 August 2017 Published by the IEEE Computer Society 2469-7087/17/$33.00 © 2017 IEEE View from the Cloud Quality of Cloud Services: Expect the Unexpected Editor: George Pallis • [email protected] Quality of Cloud Services: Cloud Service Benchmarking o fully understand cloud service benchmarking, rst we driven by the deployment model and more by the usage model. T must consider what a cloud service is, and how to deter- Expect the Unexpected mine its qualities. What’s a Cloud Service Quality? A cloud service – that is, the software system behind the service What’s a Cloud Service? interface, will confront the cloud consumer with a particular David Bermbach • TU Berlin Much has been written about cloud computing, often focusing quality behavior: the cloud service might become unavailable, it on delivery models or a cloud computing stack. However, with might be slow to respond, or it might be limited with regards to Here, the author presents a number of experiences from several years of bench- the availability of container technologies or lambda services, a the number of requests that it can handle. These are all exam- differentiation into infrastructure as a service (IaaS) and plat- ples of qualities – namely, availability, latency, or scalability – marking cloud services. He discusses how the respectively observed quality form as a service (PaaS) seems somewhat outdated. At the same and an application using the respective service needs to have behavior would have affected cloud applications or how cloud consumers could time, Web APIs are widely used and NoSQL systems are much mechanisms in place to deal with these qualities (or, rather, deal use the behavior to their advantage. more similar to cloud storage services than the latter group is with poor quality). to virtual machines. For our purposes, a cloud service is, thus, a software system running in the cloud whose functionality is What’s Cloud Service Benchmarking? consumed programmatically by applications over Internet proto- Cloud service benchmarking is a way to systematically study the n the last few years, cloud computing has and to get insights into the actual quality of cloud cols. To applications, such cloud services appear like a black box, quality of cloud services based on experiments. For this purpose, found widespread adoption in companies of all services is through cloud service benchmarking independent of the deployment model used, which is expected the benchmark tool creates an arti cial load on the cloud service I sizes. A core focus of these cloud consumers (see the related sidebar). to adapt to application workloads while maintaining quality goals. under test while carefully tracking detailed quality metrics. A key is typically on cost savings, convenience of man- Thus, here I report on a number of experiences Speci cally, we consider an open source system such as Apache design goal of cloud service benchmarking is to mimic an appli- aged services, and on-demand capacity changes. from several years of benchmarking cloud services Kafka or Apache Cassandra, deployed on top of a compute ser- cation as closely as possible to get meaningful results; however, The quality of cloud services, however, is usually and briey discuss how the respectively observed vice to be a cloud service as long as it’s used/consumed like a ser- benchmark runs also aim to extensively stress the service, for taken “as-is”: based on documentation, advertise- quality behavior would have affected cloud appli- vice. This means that our understanding of cloud services is less example, through system load or even injected failures. ments, but also past experiences from a non-cloud cations or how cloud consumers could use the world, cloud consumers typically have implicit behavior to their advantage. This article should assumptions. For instance, an eventually consis- be seen as a call: Don’t make assumptions, make tent storage system that claims to have triple rep- experiments. For this purpose, I also sketch out These benchmark results are an sizes. What they discovered for the replicas. This implies two things: rst, lication in close-by datacenters with high-speed how cloud consumers can use cloud service bench- excellent example where cloud consum- SME provider was that – across differ- that other clients can read outdated network interconnection can be assumed to show marking in their application lifecycle. ers could bene t from their knowledge ent VM sizes – compute power, mem- data while updates are being propa- low millisecond staleness – that is, relatively good on cloud service quality: depending ory, and disk throughput increased as gated; and second, that several clients consistency behavior. As another example, virtual Performance of Virtual Machines on the performance requirements of expected – that is, an M instance gen- might write the same data item con- machines (VMs) that come in sizes S, M, L, and XL can In 2011, Alexander Lenk and colleagues1 ran a num- the respective cloud application, the erally showed better performance than currently, thereby leading to conicts. be expected to grow in capacity for all resources – ber of performance experiments on top of Amazon cloud consumer could simply start a an S instance. However, independent Especially in the presence of failures network bandwidth, disk storage volume, RAM EC2 instances using the Phoronix test suite. Soon, new instance, run a short test, and of the actual VM size, the available (such as message loss or crashed size, CPU clock speed, or cores – when choosing a they discovered that they had a twin peak distri- then determine whether they wanted to network bandwidth stayed constant. instances), this inconsistency window bigger instance type. However, in both examples bution of compute performance results: For every use that instance for their application Considering the cost of different VM – also called staleness – that is, the this isn’t always the case – in fact, cloud consum- benchmark in the suite, there were some machines or whether to repeat the provisioning sizes, this leads to an interesting situ- time during which outdated data might ers should always expect the unexpected. that showed a very good performance while oth- process. ation where for a number of applica- still be read, could become rather long. This isn’t meant to imply that the unexpected ers showed a rather poor performance. Through Such unexpected behavior isn’t a tions it will be much more attractive to In general, applications can often tol- is always bad for the cloud consumer. Actually, in-depth analysis of results, they realized that the thing of the past. In 2015, we men- scale-out using the smallest VM type erate staleness quite well; however, observable quality behavior is typically much bet- performance variance stayed constant over time tored a student project in which a instead of scaling up; especially so for this becomes much easier if staleness ter than what’s guaranteed, for example, when and could also not be attributed to different instance group of master’s students at Tech- network-bound applications. These is bounded. To our knowledge, even cloud consumers only plan for what’s guaranteed, sizes; instead, Amazon had obviously deployed two nische Universität Berlin ran a number effects would never have been discov- today there’s no cloud provider that they’ll never tap the full potential of cloud ser- different CPU types (AMD Opteron and Intel Xeon). of performance benchmarks on VMs. ered without benchmarking. guarantees upper bounds for staleness. vices. Furthermore, violations of guarantees might Depending on the CPU type, the machines either In their experiments, they compared Therefore, in 2011, we developed a still occur. Therefore, not knowing about the qual- excelled at oating point or at integer operations. an open stack-based SME cloud pro- Consistency of Cloud benchmarking approach for consis- ity of the cloud services used will generally either However, both types came with the same price tag. vider to Amazon’s EC2. For their mea- Storage Services tency and repeatedly measured consis- lead to unexpected negative surprises through Furthermore, the performance difference could only surements, they used a subset of the Cloud storage systems and services tency behavior of the Amazon Simple (seemingly) obscure application behavior or to be identi ed after having provisioned the instance. Phoronix test suite to quantify CPU are typically replicated; many of them Storage Service (S3) over the years.2 inef ciencies when cloud consumers design their Obviously, this isn’t the anticipated behavior that compute capacity, RAM, and disk guarantee so-called eventual con- Basically, this approach aims to pro- applications only based on explicitly provided a cloud consumer would expect from a cloud pro- throughput, but also network band- sistency. In such systems, an update voke the worst possible consistency quality guarantees. The only way to avoid this vider that offers a standardized product. width between different VMs of all operation terminates before writing all behavior so as to obtain probabilistic

Security of Cloud Storage 20,000 Services 15,000 Especially when dealing with sensitive data in cloud environments, security 10,000 becomes a key design goal – particularly for data-in-transit security, where

Staleness in ms 5,000 data are encrypted and hashed before being sent over the Internet. This, how- 0 ever, can be expected to come with 0 a performance impact – which has 50,000

100,000 150,000 200,000 250,000 300,000 350,000 400,000 450,000 500,000 550,000 largely been neglected by researchers (a) Seconds since experiment start so far: either researchers focus on security so that performance impacts are 12,000 largely disregarded, or they focus on 10,000 performance, then ignoring security or 8,000 choosing the weakest option available. 6,000 In recent experiments, I’ve worked 4,000 with colleagues to benchmark how

Staleness in ms enabling data-in-transit security (for 2,000 example, based on TLS) affects the 0 010203040506070 performance of cloud storage ser- (b) Test no. vices. Interestingly, though, there’s no clear result, as the impact completely depends on the concrete sys- Figure 1. Consistency of Amazon Simple Storage Service (S3) in a one-week tem. For instance, in previous work,4 benchmark run in 2011. (a) At night, S3 showed much lower and more predictable we described how Apache Cassandra staleness than during the day. (b) During the day, staleness of S3 followed a saw con gurations with TLS might, in pattern. This pattern was independent of the interval between individual tests. fact, outperform unsecured con gurations (essentially, this means that upper bounds on staleness, for example. night, it followed an obscure saw pat- the natural performance variability The measurement approach comprises tern during the day. Figures 1a and 1b, of cloud resources exceeds and hides a number of distributed machines (for taken from previous work,2 show this the performance impact of TLS); this, instance, 12 is a good number for three behavior measured during a one-week however, depends on the respective replicas) that continuously poll a tar- benchmark run: During the day, the con guration and setup details. Ama- get key. Another machine periodically rst update has a 2-second staleness, zon’s DynamoDB service, on the other updates that target key with the cur- the second one a 4-second staleness, hand, shows no performance impact rent timestamp and a version number and so on until it drops back down after at all – aside from computation over- (one test in Figure 1b). Correlation of close to 2 minutes and starts all over heads on the application machines, values read and the respective current again. Of course, we contacted Amazon the performance overhead is shoul- timestamp can then be used to deter- about this behavior and also continued dered and paid for by Amazon. On mine the staleness. Furthermore, this to benchmark S3 consistency behavior the other hand, we’ve seen in recent data also can be used to determine the over the years: Not only was the ini- experiments with Apache HBase that probability of reading stale data as a tial behavior totally unexpected – until enabling data-in-transit security function of the duration since the last our last benchmark run in late 2013, could have a catastrophic impact on update. it continued to change signi cantly performance, thereby also severely S3 guarantees eventual consistency (see Figure 22), thus providing further limiting scalability.5 For example, we based on at least three replicas located proof for our “expect the unexpected” could observe that a 12-node HBase in adjacent datacenters. What could be mantra. Without going into further cluster with data-in-transit security expected, hence, were staleness val- details, suf ce it to say that dealing enabled can sustain approximately ues in the lower two-digit millisecond with inconsistencies at the application the same throughput as an unsecured range. However, in our rst (repeated) level isn’t too dif cult – unless there’s 6-node cluster. experiments in 2011, we found that no information on the quality behavior For application developers, this while S3 had acceptable staleness at of underlying cloud services.3 should have a strong effect on the

70 www.computer.org/internet/ IEEE INTERNET COMPUTING 20 Computing Edge August 2017 View from the Cloud Quality of Cloud Services: Expect the Unexpected

Security of Cloud Storage service-selection process. For 20,000 20,000 Services instance, HBase should be avoided if Average Especially when dealing with sensitive security is necessary in cloud deploy- 10,000 4,000 15,000 Max data in cloud environments, security ments. On the other hand, a hosted 1,000 3,000 Min 10,000 becomes a key design goal – particu- service might be an excellent choice 100 2,000 larly for data-in-transit security, where where maximum security essentially Median 10 1,000 Standard Staleness in ms

Staleness in ms 5,000 data are encrypted and hashed before comes for free as long as you trust the Standard deviation Standard deviation being sent over the Internet. This, how- cloud provider. 1 0 12345678 0 ever, can be expected to come with Experiment 0 a performance impact – which has Availability of Web APIs 50,000

100,000 150,000 200,000 250,000 300,000 350,000 400,000 450,000 500,000 550,000 largely been neglected by researchers As a completely different example (a) Seconds since experiment start so far: either researchers focus on secu- of cloud services, we recently ran a Figure 2. Consistency behavior of Amazon S3, as determined through one- rity so that performance impacts are three-month experiment where we week benchmark runs in 2011–2013. Throughout the benchmark runs, 12,000 largely disregarded, or they focus on benchmarked performance and avail- behavior changed signi cantly. 10,000 performance, then ignoring security or ability of Web APIs which, as I previ- 8,000 choosing the weakest option available. ously described, we also consider cloud surprises. Another curious behavior stand the quality of all options. Of 6,000 In recent experiments, I’ve worked services due to their similarity from a was that there was approximately a 70 course, there are often interdepen- 4,000 with colleagues to benchmark how service consumption perspective. A percent chance of the HTTPS endpoint dencies – for instance, developers 6

Staleness in ms enabling data-in-transit security (for key aspect of our experiment was the of an API being available while the might need cloud services that are 2,000 example, based on TLS) affects the geodistribution of clients: because Web HTTP endpoint of the same API wasn’t offered by only a single provider, or 0 7 010203040506070 performance of cloud storage ser- and mobile applications are inherently (and vice versa). This indicates that a federated setup might be desir- (b) Test no. vices. Interestingly, though, there’s distributed – either through a global Web API providers often have separate able. This should lead to an environ- no clear result, as the impact com- user base or through the geomobil- front-end servers per protocol and only ment of handpicked cloud services, pletely depends on the concrete sys- ity of individual users – we deployed share the backend services. along with initial ideas for dealing Figure 1. Consistency of Amazon Simple Storage Service (S3) in a one-week tem. For instance, in previous work,4 our benchmarking clients all over the Both results can be dangerous for with quality problems. benchmark run in 2011. (a) At night, S3 showed much lower and more predictable we described how Apache Cassandra world. For the experiment, we selected application developers if they aren’t Afterward, in the (initial) devel- staleness than during the day. (b) During the day, staleness of S3 followed a saw con gurations with TLS might, in 15 hand-picked Web APIs so as to known. However, they also can be opment phase we would recommend pattern. This pattern was independent of the interval between individual tests. fact, outperform unsecured con gu- cover a wide variety of application leveraged, for example, by trying the implementing micro-benchmarks as rations (essentially, this means that areas, countries, provider sizes, and so respective other protocol in case of well for the application itself – simi- upper bounds on staleness, for example. night, it followed an obscure saw pat- the natural performance variability on. Each of the benchmarking clients unavailability or by tunneling requests lar to unit tests, benchmarks for testing The measurement approach comprises tern during the day. Figures 1a and 1b, of cloud resources exceeds and hides periodically called all 15 Web APIs over through additional backend servers in non-functional properties should a number of distributed machines (for taken from previous work,2 show this the performance impact of TLS); this, both HTTP and HTTPS, and also pinged other geographic regions. be part of the build process. This instance, 12 is a good number for three behavior measured during a one-week however, depends on the respective the API host. For these calls, we col- approach is especially well-suited replicas) that continuously poll a tar- benchmark run: During the day, the con guration and setup details. Ama- lected detailed results and thus could Cloud Service Benchmarking for microservice-based applications get key. Another machine periodically rst update has a 2-second staleness, zon’s DynamoDB service, on the other track latency and availability. for Developers where modules can be benchmarked updates that target key with the cur- the second one a 4-second staleness, hand, shows no performance impact What we expected to nd in the Now that we’ve seen how cloud ser- individually. During this phase, it also rent timestamp and a version number and so on until it drops back down after at all – aside from computation over- results was a performance variance vices show unexpected behavior again makes sense to periodically reassess the (one test in Figure 1b). Correlation of close to 2 minutes and starts all over heads on the application machines, depending on the geolocation of the and again, when and how should quality of underlying cloud services. values read and the respective current again. Of course, we contacted Amazon the performance overhead is shoul- client – this was typically the case. application developers use cloud ser- Finally, when the application goes into timestamp can then be used to deter- about this behavior and also continued dered and paid for by Amazon. On However, what we also expected was vice benchmarking? production, underlying cloud services mine the staleness. Furthermore, this to benchmark S3 consistency behavior the other hand, we’ve seen in recent that availability would be comparable Generally, a cloud migration or should be carefully monitored using data also can be used to determine the over the years: Not only was the ini- experiments with Apache HBase that across locations. This was absolutely the development of a cloud-native both monitoring, periodic benchmark- probability of reading stale data as a tial behavior totally unexpected – until enabling data-in-transit security not the case. We were surprised to nd application will begin with an initial ing, or indirect monitoring,2 where function of the duration since the last our last benchmark run in late 2013, could have a catastrophic impact on that there were several APIs that had assessment phase, where the devel- business key performance indicators update. it continued to change signi cantly performance, thereby also severely an availability of less than 50 percent opers decide on the target runtime (KPIs) gauge for quality changes in the S3 guarantees eventual consistency (see Figure 22), thus providing further limiting scalability.5 For example, we for most of the days of our experiment environment but also on the set of underlying cloud services. Whenever based on at least three replicas located proof for our “expect the unexpected” could observe that a 12-node HBase – however, this was true only in some cloud services that their application something unusual happens, develop- in adjacent datacenters. What could be mantra. Without going into further cluster with data-in-transit security regions while they were fully available will use. In this phase, it’s useful to ers should reassess the quality of the expected, hence, were staleness val- details, suf ce it to say that dealing enabled can sustain approximately in others. For an unknown reason, some select existing benchmark imple- cloud services used, but also adapt their ues in the lower two-digit millisecond with inconsistencies at the application the same throughput as an unsecured APIs don’t have the same availability mentations that are as similar as deployment decisions by, for example, range. However, in our rst (repeated) level isn’t too dif cult – unless there’s 6-node cluster. across geographic regions so that end possible to the application workload. switching providers. Of course, actually experiments in 2011, we found that no information on the quality behavior For application developers, this users of mobile applications built on Developers should then use these implementing this approach in practice while S3 had acceptable staleness at of underlying cloud services.3 should have a strong effect on the top might be confronted with negative benchmark tools to better under- comes with a number of challenges;

70 www.computer.org/internet/ IEEE INTERNET COMPUTING JANUARY/FEBRUARY 2017 71 www.computer.org/computingedge 21 View from the Cloud

however, these are beyond the scope of experimental results used as a basis for this 6. D. Bermbach and E. Wittern, “Benchmark- this article. article. ing Web API Quality,” Proc. Int’l Conf. Web Eng., 2016, pp. 188–206. References 7. T. Kurze et al., “Cloud Federation,” Proc. n all these examples, we have seen 1. A. Lenk et al., “What Are You Paying for? Int’l Conf. Clouds, Grids, and Virtualiza- I how completely unexpected behav- Performance Benchmarking for Infrastruc- tion, 2011, pp. 32–38. ior recurs in all kinds of cloud services. ture-as-a-Service Offerings,” Proc. IEEE Int’l Application developers should, there- Conf. Cloud Computing, 2011, pp. 484–491. David Bermbach is a senior researcher in the Infor- fore, never assume that cloud services 2. D. Bermbach and S. Tai, “Benchmarking mation Systems Engineering research group behave like traditional on-premises Eventual Consistency: Lessons Learned from of TU Berlin. His research interests include environments – instead, developers Long-Term Experimental Studies,” Proc. cloud service benchmarking, cloud applica- should expect the unexpected and IEEE Int’l Conf. Cloud Eng., 2014, pp. 47–56. tions, and IoT platforms, but also middleware prepare for it. This, however, is only 3. D. Bermbach, “Benchmarking Eventually and distributed systems in general. Bermbach possible through cloud service bench- Consistent Distributed Storage Systems,” PhD has a PhD with distinction in computer sci- marking: Don’t make assumptions, thesis, Dept. of Economics and Manage- ence from Karlsruhe Institute of Technology. make experiments. ment, Karlsruhe Inst. of Technology, 2014. Contact him at [email protected]. There are a number of open chal- 4. S. Müller et al., “Benchmarking the Perfor- lenges that would bene t from future mance Impact of Transport Layer Security research efforts. The first is that in Cloud Database Systems,” Proc. IEEE Read your subscriptions benchmarks are typically designed Int’l Conf. Cloud Eng., 2014, pp. 27–36. This article originally appeared in through the myCS pub- for reuse. However, especially in the 5. F. Pallas, J. Günther, and D. Bermbach, “Pick IEEE Internet Computing, vol. 21, lications portal at http:// context of custom microservices, cur- Your Choice in HBase: Security or Performance,” no. 1, 2017. mycs.computer.org. rently it’s unclear how benchmarks Proc. IEEE Int’l Conf. Big Data, to appear. that are part of the build process can be generalized and reused. After all, a speci c microservice is rather unique in its nature so that developing a “standard” benchmark that July–September 2016 IEEE MultiMedia doesn’t only test a minimum subset of features is quite challenging. The http://www.computer.org second aspect is moving from ne- 2017 B. Ramakrishna Rau Award granular benchmarks (or even micro- 2016 JULY–SEPTEMBER benchmarks) to more high-level Call for Nominations benchmarks. After all, application Honoring contributions to the computer microarchitecture field ❚ developers are often more interested Quality Modeling

in the overall quality of an entire New Deadline: 1 May 2017 cloud platform than in assessing Established in memory of Dr. B. (Bob) Ramakrishna Rau, the award individual cloud services – or even recognizes his distinguished career in promoting and expanding the worse: of small subsets of a service use of innovative computer microarchitecture techniques, including his innovation in complier technology, his leadership in academic

such as disk throughput of a VM. In and industrialVolume 23 Number 3 computer architecture, and his extremely high personal this area, identifying suitable, real- and ethical standards. istic application workloads but also WHO IS ELIGIBLE?: The candidate will have made an outstanding innovative contribution or contributions to microarchitecture, use of novel mult-22-03-c1 Cover-1 July 12, 2016 4:40 PM again the portability of benchmarking microarchitectural techniques or compiler/architecture interfacing. It is hoped, but not tools is an unsolved major challenge. required,IEEE MultiMedia that the winner serves will have the also community contributed to the of computer scholars, microarchitecture developers, community through teaching, mentoring, or community service. The third challenge is on develop- practitioners, and students who are interested in multiple media AWARD: Certificate and a $2,000 honorarium. ing benchmarks that assess multiple types and work in fields such as image and video processing, audio PRESENTATION: Annually presented at the ACM/IEEE International Symposium on qualities at the same time – currently, Microarchitectureanalysis, text retrieval, and data fusion. most benchmarks measure only one NOMINATION SUBMISSION: This award requires 3 endorsements. Nominations are quality, usually performance. being accepted electronically: www.computer.org/web/awards/rau CONTACT US: Send anyRead award-related questionsIt Today! to [email protected] Acknowledgments www.computer.org/multimediawww.computer.org/awards I thank Steffen Müller, Frank Pallas, Stefan Tai, and Erik Wittern for the joint work leading to the

72 www.computer.org/internet/ IEEE INTERNET COMPUTING 22 Computing Edge August 2017 EDITOR ROY WANT THE IOT CONNECTION Google; [email protected]

Extending the Cloud to the Network Edge

Rubén S. Montero, OpenNebula and Complutense University Elisa Rojas, Telcaria Ideas SL Alfonso A. Carrillo, Telefónica Ignacio M. Llorente, OpenNebula, Complutense University, and Harvard University

Telefónica’s OnLife project aims to virtualize the access network and give third-party Internet

of Things application developers and content To realize this vision, we need a providers cloud-computing capabilities at the new computing infrastructure that can cope with massive device con- network edge. nectivity and is exible enough to address the requirements of a diverse set of devices and their associated applications. Reducing and FROM THE EDITOR managing communication latency The Internet of Things presents many new challenges, including ensuring low- will de ne the future of IoT applica- latency interactions between users and devices. Low latency is particularly diﬃ - tions like video streaming, gaming, cult to achieve when cloud services are involved, as the cloud could be far away and many mobile apps. For example, from the devices it controls. In this article, the authors present a practical design voice-controlled smart-home sys- that dynamically migrates computing services closer to the user to solve this prob- tems bene t from content caching, lem. —Roy Want health devices require low latency to respond to emergencies in real time, connected cars might rely on the he Internet of Things (IoT) is drastically chang- collective processing of nearby vehicles’ sensor data, and ing our world by connecting every kind of device industrial robotics demand more computing capabilities to the Internet, from doorbells and sprinkler sys- with steady latency. tems to health sensors and tra c lights. Ideally, The geographical distance between IoT service pro- these devices will interconnect with other devices or ser- viders and users from a centralized cloud infrastruc- T vices to perform their tasks in smarter ways, forming the ture turns out to be an important issue. Centralized basis of an optimal environment that reacts to our needs clouds are appropriate for services with limited data and moods. communication— such as web services—or for batch

processing, but not for applications providers’ computing facilities to the by simplifying the implementation and that require moving large amounts of network edge. introducing an open framework to de- distributed data or those with interac - There are several initiatives to ploy edge applications. tive users that require low latency and achieve this goal, mainly built around real-time processing. Meeting these mobile-edge computing (MEC)—a net- ONLIFE latency demands requires bringing re- work architecture concept that enables The COdc’s functional goals are two- sources as close to IoT devices as phys- cloud-computing capabilities and an IT fold: rst, it must support current ically possible, as the response delay service environment at the edge of the residential services, such as Internet introduced by intercountry—or inter- cellular network. Among these, the access, voice calls, and Internet Proto- continental—round trips would make Central Oce Re-architected as a Data- col Television (IPTV); second, it must IoT applications unfeasible. Moreover, center (CORD) initiative seems to be allow the deployment of third-party edge solutions. While designing the COdc, we adhered to the following principles: use open source software We need a new computing infrastructure and open hardware specications, that can cope with massive device greeneld to avoid constraining new applications with current pro- connectivity and is ﬂexible enough to address tocols, and maintain simplicity by the requirements of a diverse set of devices not over-engineering an intrinsically and their associated applications. complex system. Figure shows the main components of OnLife’s architecture. In the if we consider the number of con- better suited for convergent telecoms, upper layer, the business support sys- nected IoT devices, centralized pro- as it integrates network function virtu- tem (BSS) provides the COdc with basic cessing of their generated data doesn’t alization (NFV) and software- dened user authentication, authorization, scale, and such processing needs to be networking (SDN). CORD aims to re- and accounting capabilities. Interac- distributed among resources close to duce costs while bringing agility and tion with Telefónica’s business logic the devices. These close-to-the-device rened control to the network. CORD’s is performed through a custom cap- resources also need to be provisioned reference architecture is based on tive portal that oers available edge in a cloud-like manner to support the three pillars: commodity hardware, an applications (for example, remotely various connected IoT devices, their SDN kernel to control the underlying controlling the lighting in a house), applications, and service providers. switching fabric, and a virtualization connectivity, and additional services. management platform to create and The SDN controller, based on the THE NEED FOR control the virtualized functions. Open Network Operating System EXTENDING THE CLOUD However, we believe that to support (ONOS; onos project.org) and respon- Most IoT service providers across dif- the various IoT devices and applications sible for executing the networking ferent industries have acknowledged of future cities and homes, the CO must logic that controls the switching fab- the latency issue and are building or be further re-architected as a cloud at ric in the CO, is in the lower layer. using distributed clouds to colocate the edge of the access network. This will There are two main network appli- their services across dierent geo- transform the CO into a multi tenant en- cations running in ONOS: vOLT and graphical areas to provide the required vironment where IoT service providers ClosFwd. The vOLT application re- quality of service (QoS) and function- can deploy elastic applications with a produces the behavior of an optical ality. Telecommunications companies great degree of control. line terminal (OLT) by redirecting are in a unique position to solve this Telefónica, one of the world’s largest trac to the captive portal by default problem because central oces (COs) telecoms, is exploring this approach (where clients can consult with and are usually located close to their cus- through the OnLife project, whose hire dierent services), and switching tomers’ premises and thus close to IoT main goal is to design a future-proof the inbound trac to the CO once the devices. In addition, COs can be trans- technology stack that could bring the client is subscribed to the network. formed into clouds. Similar to the benets of cloud computing and net- The ClosFwd application is in charge cloudlet concept, in which small-scale work programmability to the access of internally forwarding the CO and cloud datacenters at the edge of the network. OnLife’s technological core is creates the paths between the client, Internet are used to support resource- the CO datacenter (COdc), which builds the virtual subscriber gateway (vSG; intensive and interactive mobile ap- on some of CORD’s principles but takes the virtual replacement of the cus- plications, this can extend IoT service its disruptive approach a step further tomer premises equipment [CPE] that

24 Computing Edge August 2017 92 COMPUTER WWW.COMPUTER.ORG/COMPUTER THE IOT CONNECTION

processing, but not for applications providers’ computing facilities to the by simplifying the implementation and runs in a virtual machine [VM] and Business logic that require moving large amounts of network edge. introducing an open framework to de- provides basic routing and ltering), distributed data or those with interac - There are several initiatives to ploy edge applications. and the various services. Both appli- BSS Supervision tive users that require low latency and achieve this goal, mainly built around cations provide a RESTful API that is real-time processing. Meeting these mobile-edge computing (MEC)—a net- ONLIFE dynamically controlled by the cloud latency demands requires bringing re- work architecture concept that enables The COdc’s functional goals are two- management platform (CMP), which COdc (OnLife central of ce) sources as close to IoT devices as phys- cloud-computing capabilities and an IT fold: rst, it must support current is located at the same logical level ically possible, as the response delay service environment at the edge of the residential services, such as Internet of the SDN controller and built with Edge Edge Edge vSG introduced by intercountry—or inter- cellular network. Among these, the access, voice calls, and Internet Proto- OpenNebula (opennebula.org). app app app continental—round trips would make Central Oce Re-architected as a Data- col Television (IPTV); second, it must OpenNebula, a lightweight and IoT applications unfeasible. Moreover, center (CORD) initiative seems to be allow the deployment of third-party powerful CMP, is responsible for man- SDN Cloud management and app orchestration edge solutions. While designing the aging the virtualized resources that vOLT ClosFwd COdc, we adhered to the following implement the dierent NFVs and principles: use open source software edge applications. OpenNebula also COvr We need a new computing infrastructure and open hardware specications, interacts with the ONOS components that can cope with massive device greeneld to avoid constraining to establish the network connectiv- new applications with current pro- ity for each VM. Additionally, it pro- Clos fabric connectivity and is flexible enough to address tocols, and maintain simplicity by vides the orchestration functionality the requirements of a diverse set of devices not over-engineering an intrinsically needed to manage multiple- VM ap- and their associated applications. complex system. plications that might include inter- Computing infrastructure Figure shows the main compo- dependencies and elasticity rules to nents of OnLife’s architecture. In the dynamically adjust the number of if we consider the number of con- better suited for convergent telecoms, upper layer, the business support sys- VMs based on the application load. Figure 1. OnLife architecture. The business support system (BSS) in the upper layer nected IoT devices, centralized pro- as it integrates network function virtu- tem (BSS) provides the COdc with basic Finally, several virtualized com- provides the central office datacenter (COdc) with basic user authentication, authoriza- cessing of their generated data doesn’t alization (NFV) and software- dened user authentication, authorization, ponents of the architecture are im- tion, and accounting capabilities. The software-defined networking controller, based on scale, and such processing needs to be networking (SDN). CORD aims to re- and accounting capabilities. Interac- plemented as either VMs or SDN ap- the Open Network Operating System (ONOS), is responsible for executing the network- distributed among resources close to duce costs while bringing agility and tion with Telefónica’s business logic plications: a vSG; a CO virtual router ing logic that controls the Clos switching fabric in the CO. The cloud manager, based on the devices. These close-to-the-device rened control to the network. CORD’s is performed through a custom cap- (COvr; in charge of routing trac to OpenNebula, is responsible for managing the virtualized resources that implement the resources also need to be provisioned reference architecture is based on tive portal that oers available edge Telefónica’s transport networks), and different network function virtualizations and edge applications. COvr: CO virtual router; in a cloud-like manner to support the three pillars: commodity hardware, an applications (for example, remotely the edge applications to support IoT vOLT: network application that replicates the behavior of an optical line terminal; vSG: various connected IoT devices, their SDN kernel to control the underlying controlling the lighting in a house), devices that are deployed in indepen- virtual subscriber gateway. applications, and service providers. switching fabric, and a virtualization connectivity, and additional services. dent VMs in an isolated network. The management platform to create and The SDN controller, based on the idea behind edge applications is that THE NEED FOR control the virtualized functions. Open Network Operating System third-party companies can develop edge computing applications, similar persistency information at the edge. EXTENDING THE CLOUD However, we believe that to support (ONOS; onos project.org) and respon- their own appliances to implement the to the infrastructure as a service (IaaS) This includes the application logic Most IoT service providers across dif- the various IoT devices and applications sible for executing the networking associated edge logic for the devices. model, which opens the datacenter to itself, so edge applications also need ferent industries have acknowledged of future cities and homes, the CO must logic that controls the switching fab- The OnLife architecture has been external workloads. The ability to pro- to be able to autocongure. The auto- the latency issue and are building or be further re-architected as a cloud at ric in the CO, is in the lower layer. implemented in a proof-of-concept vide this edge-computing platform in conguration process is performed using distributed clouds to colocate the edge of the access network. This will There are two main network appli- CO, based on compute nodes with a pay-as-you-go model (similar to IaaS) using specic information passed to their services across dierent geo- transform the CO into a multi tenant en- cations running in ONOS: vOLT and CPUs and Gbytes of RAM, using a opens up avenues in both innovative the edge application upon bootup. The graphical areas to provide the required vironment where IoT service providers ClosFwd. The vOLT application re- virtualized Clos fabric consisting of use cases and business models. context could include user data, con- quality of service (QoS) and function- can deploy elastic applications with a produces the behavior of an optical x leaf-spine OpenFlow switches and However, given the CO’s specic guration parameters, or additional ality. Telecommunications companies great degree of control. line terminal (OLT) by redirecting an emulated OLT. This setup allowed characteristics in terms of computa- resources to install the application. are in a unique position to solve this Telefónica, one of the world’s largest trac to the captive portal by default us to showcase a complete workow tional and storage resources—in ad- We envision a wide range of edge problem because central oces (COs) telecoms, is exploring this approach (where clients can consult with and from customer authentication to the dition to the environment’s security applications that will work with On- are usually located close to their cus- through the OnLife project, whose hire dierent services), and switching deployment of associated edge appli- constraints—a well-dened frame- Life, from single-component instances tomers’ premises and thus close to IoT main goal is to design a future-proof the inbound trac to the CO once the cations. As test cases, we implemented work to develop such edge applica- to app lications that require the de- devices. In addition, COs can be trans- technology stack that could bring the client is subscribed to the network. basic connectivity applications (Inter- tions is required. For example, an ployment of multiple VMs. An edge formed into clouds. Similar to the benets of cloud computing and net- The ClosFwd application is in charge net access and video on demand) and a application deployed at the network application in the COdc provides this cloudlet concept, in which small-scale work programmability to the access of internally forwarding the CO and content delivery network (CDN). edge has to be rapidly reallocated capability and includes deployment cloud datacenters at the edge of the network. OnLife’s technological core is creates the paths between the client, when the user moves across the ac- dependencies between the VMs. The Internet are used to support resource- the CO datacenter (COdc), which builds the virtual subscriber gateway (vSG; EDGE APPLICATIONS cess network (for instance, from home interconnection of the VMs for each intensive and interactive mobile ap- on some of CORD’s principles but takes the virtual replacement of the cus- One of OnLife’s main challenges is to to oce). Therefore, we require edge edge application happens in a separate plications, this can extend IoT service its disruptive approach a step further tomer premises equipment [CPE] that make the CO available for third-party applications to not store any state or private network.

www.computer.org/computingedge 25 92 COMPUTER WWW.COMPUTER.ORG/COMPUTER APRIL 2017 93 THE IOT CONNECTION

Backbone network ONOS to deploy the edge applications serv2serv1 COvr and provide them with the features mentioned earlier.

MOVING SERVICES Clos Access FROM CUSTOMER PREMISES portal s1 s2 Internet TO THE CENTRAL OFFICE The initial functional and perfor- L1 L2 L3 L4 mance analyses made using the proof-of-concept and demo applica- vOLT tions are very promising, and show us how to move other Telefónica solutions (currently deployed in the CPE or vSG CDN in expensive centralized locations) to the network edge. CPEs have limited Central of ce capacity to host new IoT services such as internal security, access control, and energy management, which currently require the installation of additional physical equipment. The COdc Edge application allows us to host these services within a vSG built for the specic needs of the product oering. In particular, the fol- VLAN for each app lowing Telefónica solutions and ser- HGU vices are being considered:

Residential access › Inmótica Hydra. This energy- Figure 2. Service architecture for OnLife edge applications. Applications are deployed eciency enterprise solution as virtual machines by OpenNebula and interconnected in the Clos through specif- helps customers manage and re- ic switching circuits installed by ONOS. Internet of Things devices connect to each duce their energy consumption application in the COdc through the residential access network. Together with the edge and requires the installation of applications, standard services applications are also deployed in the COdc (for example, on- premise servers that occupy Internet access or VoIP) and accessed in the same way. CDN: content delivery network; oor space and remote mainte- HGU: home gateway unit. nance for each customer facility. Telefónica aims to remove this equipment and host all functionality within the COdc, leaving Figure depicts the deployment of as a trac jam, a large event, or an only the system’s meters and several applications in the COdc. IoT emergency in a neighborhood might sensors on premises. devices use the residential network to require allocating additional com- › On the Spot. This retail connect to the edge applications de- putational resources to the associ- commerce– oriented solution, ployed in the COdc. Within the COdc, ated edge application. The COdc can which provides small businesses the IoT trac is then forwarded to the increase (or decrease) the number of with in-store music, digital target application through specic VMs considering application-specic signage, and customer Wi-Fi, switching circuits in the Clos, which performance metrics; for example, to also requires the installation of eventually could send the data to the add more VMs at specic times and on-premise servers that oc- ISP backbone network. Apart from IoT dates or when the number of requests cupy oor space. On the Spot’s applications, basic service apps are are above a given threshold. maintenance cost and contin- also deployed for each customer and The COdc also provides a well- uous software updates pose a accessed in the same way (for example, dened API to manage edge ap- challenge that the COdc is well vSG for Internet access). plications. This API resembles the suited to help resolve. Edge applications are tied to the classical IaaS API to control a VM’s › FAAST Vulnerability. This resi- environmental conditions where lifecycle. The COdc uses the func- dential solution, which provides IoT devices operate. A problem such tionality exposed by OpenNebula and protection against IoT threats,

26 Computing Edge August 2017 94 COMPUTER WWW.COMPUTER.ORG/COMPUTER THE IOT CONNECTION

Backbone network ONOS to deploy the edge applications requires an agent in the CPE. portal.etsi.org/portals//tbpages serv2serv1 COvr and provide them with the features However, most home CPEs don’t /mec/docs/mobile-edge_computing mentioned earlier. have the capabilities to host it. _-_introductory_technical_white PURPOSE: The IEEE Computer Society is thePURPOSE: world’s Thelargest IEEE association Computer of Society computing is The COdc is the only way to de- _paper_v©--.pdf. professionalsthe world’s largest and is association the leading of provider computing of technicalprofessionals information and is the in leading the eld. provider Visit our of MOVING SERVICES liver this service without replac- . L. Peterson et al., “Central Oªce technical information in the eld. Visit our Clos website at www.computer.org. Access OMBUDSMAN:website at www.computer.org. Email ombudsman@ Internet FROM CUSTOMER PREMISES ing or upgrading the residential Re-Architected as a Data Center,” OMBUDSMAN: Email ombudsman@ portal s1 s2 computer.org. TO THE CENTRAL OFFICE CPE-installed base. IEEE Comm. Magazine, vol. , no. , computer.org. Next Board Meeting: 12–13 November 2017, The initial functional and perfor- , pp. –. Phoenix,Next Board AZ, Meeting: USA 12–17 June 2017, Phoenix, AZ, USA L1 L2 L3 L4 mance analyses made using the . B. Sotomayor et al., “Virtual Infra- EXECUTIVE COMMITTEE EXECUTIVE COMMITTEE President: Jean-Luc Gaudiot proof-of-concept and demo applica- aking use of NFV and SDN, structure Management in Private President: Jean-Luc Gaudiot President-Elect: Hironori Kasahara; Past President-Elect: Hironori Kasahara; Past tions are very promising, and show exible datacenters built on and Hybrid Clouds,” IEEE Internet President: Roger U. Fujii; Secretary: Forrest vOLT President: Roger U. Fujii; Secretary: Forrest Shull; First VP, Treasurer: David Lomet; Second us how to move other Telefónica solu- commodity hardware can Computing, vol. , no. , , Shull; First VP, Treasurer: David Lomet; Second VP, Publications: Gregory T. Byrd; VP, Member VP, Publications: Gregory T. Byrd; VP, Member tions (currently deployed in the CPE or Mnow be deployed in telecom COs. Fur- pp. –. & Geographic Activities: Cecilia Metra; V P, & Geographic Activities: Cecilia Metra; V P, vSG CDN Professional & Educational Activities: Andy T. Professional & Educational Activities: Andy T. in expensive centralized locations) to thermore, it’s been shown that the Chen; VP, Standards Activities: Jon Rosdahl; Chen; VP, Standards Activities: Jon Rosdahl; VP, Technical & Conference Activities: Hausi the network edge. CPEs have limited open source ONOS and OpenNebula VP, Technical & Conference Activities: Hausi A. Müller; 2017–2018 IEEE Director & Delegate A. Müller; 2017–2018 IEEE Director & Delegate capacity to host new IoT services such projects can be adapted to dierent Division VIII: Dejan S. Milojičić; 2016–2017 IEEE Central of ce Division VIII: Dejan S. Milojičić; 2016–2017 IEEE Director & Delegate Division V: Harold Javid; 2017 as internal security, access control, application scenarios and support new Director & Delegate Division V: Harold Javid; 2017 IEEE Director-Elect & Delegate Division V-Elect: IEEE Director-Elect & Delegate Division V-Elect: John W. Walz and energy management, which cur- requirements, while allowing for fast John W. Walz BOARD OF GOVERNORS rently require the installation of addi- and inexpensive prototyping. This article originally appeared in BOARD OF GOVERNORS Term Expiring 2017: Alfredo Benso, Sy-Yen Kuo, Term Expiring 2017: Alfredo Benso, Sy-Yen Kuo, tional physical equipment. The COdc Next steps for the OnLife project Computer, vol. 50, no. 4, 2017. Ming C. Lin, Fabrizio Lombardi, Hausi A. Müller, Ming C. Lin, Fabrizio Lombardi, Hausi A. Müller, Edge application Dimitrios Serpanos, Forrest J. Shull allows us to host these services within will consist of migrating and adapt- Dimitrios Serpanos, Forrest J. Shull Term Expiring 2018: Ann DeMarle, Fred Douglis, Term Expiring 2018: Ann DeMarle, Fred Douglis, a vSG built for the specic needs of the ing the solution to a production-ready Vladimir Getov, Bruce M. McMillin, Cecilia Metra, Vladimir Getov, Bruce M. McMillin, Cecilia Metra, Kunio Uchiyama, Stefano Zanero product oering. In particular, the fol- hardware infrastructure and replacing Kunio Uchiyama, Stefano Zanero Term Expiring 2019: Saurabh Bagchi, Leila De VLAN for each app Term Expiring 2019: Saurabh Bagchi, Leila De Floriani, David S. Ebert, Jill I. Gostin, William lowing Telefónica solutions and ser- the emulated elements (such as vOLT) RUBÉN S. MONTERO is chief archi- Floriani, David S. Ebert, Jill I. Gostin, William Gropp, Sumi Helal, Avi Mendelson HGU vices are being considered: with actual equipment. We also aim tect at OpenNebula and an associate Gropp, Sumi Helal, Avi Mendelson to replace our current gigabit passive EXECUTIVE STAFF professor of computer architecture Executive Director: Angela R. Burgess; Director, Residential access Executive Director: Angela R. Burgess; Director, Governance & Associate Executive Director: Anne › Inmótica Hydra. This energy- optical network (GPON) access tech- at Complutense University. Contact Governance & Associate Executive Director: Anne Marie Kelly; Director, Finance & Accounting: Marie Kelly; Director, Finance & Accounting: Figure 2. Service architecture for OnLife edge applications. Applications are deployed eciency enterprise solution nology with the latest XGS-PON and Sunny Hwang; Director, Information Technology him at [email protected]. Sunny Hwang; Director, Information Technology & Services: Sumit Kacker; Director, Membership as virtual machines by OpenNebula and interconnected in the Clos through specif- helps customers manage and re- NG-PON technologies, without mod- & Services: Sumit Kacker; Director, Membership Development: Eric Berkowitz; Director, Products ic switching circuits installed by ONOS. Internet of Things devices connect to each duce their energy consumption ifying the COdc software solution and Development: Eric Berkowitz; Director, Products ELISA ROJAS is a research director & Services: EvanEvan M. Butter eld; Director, Sales & Marketing: Chris Jensen application in the COdc through the residential access network. Together with the edge and requires the installation of at a reduced capital expenditure. at Telcaria Ideas S.L. Contact her at Marketing: Chris Jensen applications, standard services applications are also deployed in the COdc (for example, on- premise servers that occupy COMPUTER SOCIETY OFFICES [email protected]. Washington, D.C.: 2001 L St., Ste. 700, Internet access or VoIP) and accessed in the same way. CDN: content delivery network; Washington, D.C.: 2001 L St., Ste. 700, oor space and remote mainte- Washington, D.C. 20036-4928 Phone: +1 202 371 0101 • Fax: +1 202 728 9614 HGU: home gateway unit. nance for each customer facility. ALFONSO A. CARRILLO manages Phone: +1 202 371 0101 • Fax: +1 202 728 9614 Email: [email protected]@computer.org Telefónica aims to remove this REFERENCES the COdc architecture at Telefónica. Los Alamitos: 10662 Los Vaqueros Circle, Los Alamitos, CA 90720 • Phone: +1 714 821 8380 • equipment and host all function- . K. Xu, X. Wang, and W. Wei, “Toward Alamitos, CA 90720 • Phone: +1 714 821 8380 • Contact him at alfonsoaurelio Email: [email protected]@computer.org ality within the COdc, leaving Software Dened Smart Home,” IEEE [email protected]. Membership & Publication Orders Phone: +1+1 800800 272 6657 • Fax: +1 714 821 4641 • Figure depicts the deployment of as a trac jam, a large event, or an only the system’s meters and Comm. Magazine, vol. , no. , , Email: [email protected]@computer.org Asia/Paci c: Watanabe Building, 1-4-2 Minami- several applications in the COdc. IoT emergency in a neighborhood might sensors on premises. pp. –. is a project Asia/Paci c: Watanabe Building, 1-4-2 Minami- IGNACIO M. LLORENTE Aoyama, Minato-ku, Tokyo 107-0062, Japan • devices use the residential network to require allocating additional com- › On the Spot. This retail . M. Satyanarayanan et al., “The Case director at OpenNebula and a full Phone: +81+81 33 34083408 3118 • Fax: +81 3 3408 3553 • Email: [email protected]@computer.org connect to the edge applications de- putational resources to the associ- commerce– oriented solution, for VM-Based Cloudlets in Mobile professor of computer architecture ployed in the COdc. Within the COdc, ated edge application. The COdc can which provides small businesses Computing,” IEEE Pervasive Comput- IEEEIEEE BOARDBOARD OF DIRECTORS as well as head of the Distributed President & CEO: Karen Bartleson; President- the IoT trac is then forwarded to the increase (or decrease) the number of with in-store music, digital ing, vol. , no. , , pp. –. Systems Architecture Research Elect: JamesJames Jefferies; Past President: Barry L. Shoop; Secretary: William Walsh; Treasurer: signage, and customer Wi-Fi, . S. Clinch et al., “How Close Is Close Shoop; Secretary: William Walsh; Treasurer: target application through specic VMs considering application-specic Group at Complutense University. JohnJohn W.W. Walz; Director & President, IEEE-USA: Karen Pedersen; Director & President, Standards switching circuits in the Clos, which performance metrics; for example, to also requires the installation of Enough? Understanding the Role He is also a visiting scholar at Karen Pedersen; Director & President, Standards Association: Forrest Don Wright; Director & VP, eventually could send the data to the add more VMs at specic times and on-premise servers that oc- of Cloudlets in Supporting Display Harvard University. Contact him at Educational Activities: S.K. Ramesh; Director & ISP backbone network. Apart from IoT dates or when the number of requests cupy oor space. On the Spot’s Appropriation by Mobile Users,” VP, Membership and Geographic Activities: Mary [email protected]. Ellen Randall; Director & VP, Publication Services applications, basic service apps are are above a given threshold. maintenance cost and contin- Proc. IEEE Int’l Conf. Pervasive and Products: Samir El-Ghazaly; Director & VP, Technical Activities: Marina Ruggieri; Director also deployed for each customer and The COdc also provides a well- uous software updates pose a Computing and Comm. (PerCom ), & Delegate Division V: Harold Javid; Director & accessed in the same way (for example, dened API to manage edge ap- challenge that the COdc is well ; elijah.cs.cmu.edu/DOCS Delegate Division VIII: Dejan S. Milojičić vSG for Internet access). plications. This API resembles the suited to help resolve. /clinch-percom--CAMERA Read your subscriptions Edge applications are tied to the classical IaaS API to control a VM’s › FAAST Vulnerability. This resi- -READY.pdf. through the myCS publications portal at revised 31 May 2017 environmental conditions where lifecycle. The COdc uses the func- dential solution, which provides . M. Patel et al., Mobile-Edge Comput- revised 26 Jan. 2017 http://mycs.computer.org IoT devices operate. A problem such tionality exposed by OpenNebula and protection against IoT threats, ing, white paper, ETSI, Sept. ;

www.computer.org/computingedge 27 94 COMPUTER WWW.COMPUTER.ORG/COMPUTER APRIL 2017 95 CLOUD AND THE LAW

al information stored and outsourced to the cloud, to frauds that are more sophisticated, and to attacks Evidence and that are disruptive, such as compromising a company’s day-to-day operations. Cloud storage services can also be abused by criminals, who use it to store and hide incriminating and illegal materials or to Forensics in 1 distribute copyright materials. There have been several concerted efforts by cloud service providers to prevent their services the Cloud: from being criminally exploited. For example, Drop- box has implemented a child abuse material detection software, whose details are not publically available, which allows searching within the les Challenges and stored on Dropbox to identify breaches of the Terms of Use and Acceptable Use Policy. Similarly, Micro- soft’s PhotoDNA is designed to identify child abuse materials from the les stored by companies on their Future Research servers, and used in its cloud storage product. Another commonly seen criminal exploitation of the cloud is to support the execution of large-scale Directions and distributed attacks, for example by compromising some instances of virtual machines within a cloud infrastructure to launch Distributed De- nial-of-Service (DDoS) attacks against third-party Kim-Kwang Raymond Choo, University of Texas at San Antonio websites, portals or platforms. In 2012, a group of cyber-criminals exploited the CVE-2014-3120 Elas- Christian Esposito and Aniello Castiglione, University of Salerno ticsearch 1.1.x vulnerability, in order to compromise virtual machines within Amazon EC2, and launched a UDP based DDoS attack. Predictably, most cloud service providers have platform-wide ALTHOUGH IT IS POPULAR WITH COMPA- DDoS protection systems that monitor incoming NIES AND PRIVATE USERS, CLOUD COM- and outgoing traf c in order to prevent DDoS at- PUTING CAN BE ABUSED OR TARGETED BY tack against their platform or to avoid being used to CRIMINALS. This can range from stealing person- launch such attacks. A number of other security solutions have been proposed for the cloud in the literature, ranging from access control to crypto primitives to intrusion detection to privacy-preserving, and so forth. Despite the existence and deployment of various security solutions, there will be times EDITOR: where digital investigation is needed. As noted in a previous column,2 to successfully prosecute in- KIM-KWANG dividuals who commit crimes involving digital evi- RAYMOND CHOO dence, one must be able to gather evidence of an incident or crime that has involved cloud servers University of Texas at San Antonio as well as the client devices that have been used [email protected] to access the cloud services, a process known as digital forensics (or cloud forensics).

d3law.indd 14 7/21/17 9:20 AM CLOUD AND THE LAW

al information stored and outsourced to the cloud, Cloud Forensics sic tools in terms of their evidential data collection to frauds that are more sophisticated, and to attacks In a cloud forensic investigation, it is necessary and analysis capabilities. It may also identify types Evidence and that are disruptive, such as compromising a com- to analyze the data ow, commonly at three main of evidence available on computing devices that fo- pany’s day-to-day operations. Cloud storage services stages, data-at-rest on the client device(s), data-in- rensic investigators would not have otherwise known can also be abused by criminals, who use it to store transit, and data-at-rest on the server(s). Therefore, it were available. and hide incriminating and illegal materials or to is important to conduct static analysis and dynamic However, existing techniques may not be appli- Forensics in 1 distribute copyright materials. (binary code) analysis of apps installed on the client cable in cloud forensics. For example, investigators There have been several concerted efforts by device, analysis of data communication and exltra- may not have physical access to the evidence, and cloud service providers to prevent their services tion channels and techniques, and investigation and a corrupted insider from the cloud service provider the Cloud: from being criminally exploited. For example, Drop- validation of techniques to locate and recover public can easily alter the evidence. Roussev et al. also not- box has implemented a child abuse material de- and private keys, authentication tokens, encrypted ed that in software as a service (SaaS) forensics, ``... tection software, whose details are not publically blocks, and other data of interest in the network traf- the use of traditional forensic tools results in acqui- available, which allows searching within the les c and on the client device and server (e.g. memory sition and analysis [that] is inherently incomplete’’.5 Challenges and stored on Dropbox to identify breaches of the Terms dumps). For example, a number of re- of Use and Acceptable Use Policy. Similarly, Micro- searchers have examined the potential soft’s PhotoDNA is designed to identify child abuse to recover data remnants from client materials from the les stored by companies on their devices, such as Android and iOS devic- Future Research servers, and used in its cloud storage product. es, that have been used to access cloud Another commonly seen criminal exploitation of services (such as the potential to recover There have been several concerted the cloud is to support the execution of large-scale forensic artefacts from an OS X PC af- efforts by cloud service providers and distributed attacks, for example by compro- ter it had been used to access Apple’s to prevent their services from being Directions 3 mising some instances of virtual machines within iCloud). In a recent investigation of the a cloud infrastructure to launch Distributed De- implementation of the OAuth protocol, a criminally exploited. nial-of-Service (DDoS) attacks against third-party commonly used token-based authentica- Kim-Kwang Raymond Choo, University of Texas at San Antonio websites, portals or platforms. In 2012, a group of tion system in mobile apps, the research- cyber-criminals exploited the CVE-2014-3120 Elas- ers demonstrated how one can intercept Christian Esposito and Aniello Castiglione, University of Salerno ticsearch 1.1.x vulnerability, in order to compro- and recover security tokens (e.g. access mise virtual machines within Amazon EC2, and and refresh tokens used to authenticate the user) Infrastructure such as distributed lesystems launched a UDP based DDoS attack. Predictably, from the device’s memory heap. This would allow can support Infrastructure as a Service (IaaS) and most cloud service providers have platform-wide forensic investigators having obtained the security other cloud computing environments by providing ALTHOUGH IT IS POPULAR WITH COMPA- DDoS protection systems that monitor incoming tokens to access a user account even after the user data fragmentation and distribution, potentially NIES AND PRIVATE USERS, CLOUD COM- and outgoing traf c in order to prevent DDoS at- has changed his/her password-based credentials (de- between countries and within datacentres. This PUTING CAN BE ABUSED OR TARGETED BY tack against their platform or to avoid being used to pending on the service provider’s implementation).4 results in signicant technical, jurisdictional and CRIMINALS. This can range from stealing person- launch such attacks. Data may not initially be in a format appropri- operational challenges in the collection of eviden- A number of other security solutions have ate for collection as digital evidence, and as such, tial data for analysis in both criminal investigations been proposed for the cloud in the literature, it becomes necessary to “decode” the protocol used and civil litigation matters.6,7 For example, a Brit- ranging from access control to crypto primitives by the application or operating system for data stor- ish barrister and a Senior Policy Advisor and Crown to intrusion detection to privacy-preserving, and age and/or transit. Thus, it is important to conduct Advocate with UK Government Crown Prosecution so forth. Despite the existence and deployment a comprehensive, empirical investigation of a range Service predicted that the evidence obtained from of various security solutions, there will be times of client devices and cloud servers against existing the cloud will play a more signicant role in the EDITOR: where digital investigation is needed. As noted in techniques and commercial and open source digital foreseeable future.8 a previous column,2 to successfully prosecute in- forensic tools, in order to make a detailed determi- In addition, as explained by Martini and Choo, KIM-KWANG dividuals who commit crimes involving digital evi- nation of the limitations of existing techniques and investigators must trust the cloud service provider RAYMOND CHOO dence, one must be able to gather evidence of an forensic tools when collecting data from client de- to maintaining trustworthy logs about the cloud incident or crime that has involved cloud servers vices and cloud servers. activity, and providing reports about the activities University of Texas at San Antonio as well as the client devices that have been used It is expected that such technical investigations of user(s) of interest upon request (e.g. a court or- [email protected] to access the cloud services, a process known as will clearly demonstrate the strengths and weak- der).9 Zawoad, Dutta and Hasan presented a solu- digital forensics (or cloud forensics). nesses of current techniques and the various foren- tion for logging the activities within the cloud, and

d3law.indd 14 7/21/17 9:20 AM d3law.indd 15 7/21/17 9:20 AM CLOUD AND THE LAW

to ensure the integrity and con dentiality of such previous versions), they were able to use the authen- logs.10 Speci cally, they propose a Proof of Past Log tication credentials collected from the client to de- (PPL) scheme to avoid tampering of the logs after crypt les stored on the server. This demonstrated their generation, and to encrypt some crucial in- the utility of the client followed by server forensic formation within the logs so as to protect the user’s investigation approach. In another work, the same privacy. The proposed solution also facilitates the authors designed a process for remote programmatic presentation of the collected evidence for veri ca- collection of evidence from an IaaS cloud service, tion in the court. which would provide forensic researchers and prac- Dykstra and Sherman described a method to titioners a tool (for instance collecting data via API) collect forensic artifacts from Amazon’s EC2 ser- to collect evidential data using a repeatable and fo- vice. They also used Eucalyptus (which operates rensically sound process.15 similarly from a client point of view to EC2) for the purposes of injecting forensic tools into running Forensic-by-Design and Forensic-as-a- VMs via the hypervisor layer.11 Using conventional Service forensic tools (such as Guidance Software EnCase Ab Rahman and colleagues proposed an alternative and AccessData FTK), the authors were success- forensic readiness strategy, referred to as forensic- ful in collecting evidence from EC2 and Eucalyp- by-design.16 Conceptually, forensic-by-design is similar to security-by-design and privacy-by-design, where requirements for forensics are integrated into relevant phases of the system development life- Conceptually, forensic-as-a-service cycle, with the objective of developing is similar to software-as-a-service forensic-ready systems. The utility of such an approach is demonstrated in a where forensic applications and latter work.17 services are being moved to the cloud. There has also been research into offering forensic-as-a-service. Concep- tually, forensic-as-a-service is similar to software-as-a-service where forensic applications and services are being tus. The level of trust required to execute each of moved to the cloud. For example, Castiglione and the collection procedures was also reported in the colleagues presented a cloud-based methodology to study. In a latter work, the same authors contributed acquire forensic evidence from online services, such a forensic toolkit for the OpenStack cloud platform as webpages, chats, documents, photos and videos.18 – FROST.12 FROST allows a remote user to collect A cloud-based solution hosts a network trusted ser- an image of the users’ VMs hosted in OpenStack, vice used to acquire evidence for subsequent analy- and retrieve log events for all API requests made by sis. Such an acquisition can be undertaken using a the user and rewall logs for all of the users’ VMs. HTTPS proxy (capable of recording activities at the FROST is integrated with several OpenStack Dash- network level, such as IP, when an online service is board and Compute components. accessed), or a software agent for the collection of in- Martini and Choo presented a four-stage cloud formation obtained by the targeted online service in a forensic framework, and used it to guide their server What You See Is What You Get (WYSIWYG) manner. and client analysis of the ownCloud private Stor- Along with his colleagues, van Beek proposed a age.13,14 The authors successfully recovered a range cloud-based approach which allows one to process of artifacts, including le data, metadata and au- and investigate the large volume of seized digital thentication credentials. Then they analyzed the materials,19 typically of a criminal investigation. server component of ownCloud. In addition to locat- This was also coined big data forensics by Quick ing a range of metadata and uploaded les (including and Choo.20 Speci cally, digital evidence obtained

30 Computing Edge August 2017 16 IEEE CLOUD COMPUTING WWW.COMPUTER.ORG/CLOUDCOMPUTING

d3law.indd 16 7/21/17 9:20 AM CLOUD AND THE LAW

to ensure the integrity and con dentiality of such previous versions), they were able to use the authen- during the investigation are outsourced to the cloud to collect evidence from devices that are encrypted logs.10 Speci cally, they propose a Proof of Past Log tication credentials collected from the client to de- by creating forensic copies, and later examined us- using strong passwords. Therefore, it is crucial to (PPL) scheme to avoid tampering of the logs after crypt les stored on the server. This demonstrated ing a standard set of tools. Thus, evidence copies develop, validate and re ne novel evidence-based their generation, and to encrypt some crucial in- the utility of the client followed by server forensic can be created and stored in a centralized and ac- data collection techniques to obtain evidential data formation within the logs so as to protect the user’s investigation approach. In another work, the same cessible location. Fu and his colleagues presented a from cloud computing (and other computing) de- privacy. The proposed solution also facilitates the authors designed a process for remote programmatic cloud-based distributed solution for tracing Internet vices in crimes that make use of sophisticated and presentation of the collected evidence for veri ca- collection of evidence from an IaaS cloud service, criminals using high-bandwidth sentinels within secure technologies, for example, the use of strong tion in the court. which would provide forensic researchers and prac- anonymous networks, such as Tor.21 This allows the encryption to secure both data-at-rest and data-in- Dykstra and Sherman described a method to titioners a tool (for instance collecting data via API) capturing of (criminal) communications for analysis. transit, as well as anti-forensic techniques. These collect forensic artifacts from Amazon’s EC2 ser- to collect evidential data using a repeatable and fo- novel evidence-based data collection techniques vice. They also used Eucalyptus (which operates rensically sound process.15 need to be designed to circumvent advanced secu- similarly from a client point of view to EC2) for the HARNESSING TECHNOLOGICAL ADVANCES rity features (such as developing low-level exploits purposes of injecting forensic tools into running Forensic-by-Design and Forensic-as-a- FOR VARIOUS ASPECTS OF POLICING HAS and undertaking physical hardware analysis) and VMs via the hypervisor layer.11 Using conventional Service BEEN A KEY OPERATIONAL OBJECTIVE obtain evidential data from cloud computing devic- forensic tools (such as Guidance Software EnCase Ab Rahman and colleagues proposed an alternative IN MANY GOVERNMENTS AND LAW EN- es, without compromising the evidence’s integrity. and AccessData FTK), the authors were success- forensic readiness strategy, referred to as forensic- FORCEMENT AGENCIES.22 Examples include These techniques will enhance “guardianship” and ful in collecting evidence from EC2 and Eucalyp- by-design.16 Conceptually, forensic-by-design is modernizing communications between eld inves- the “deterrent” effect in policing. similar to security-by-design and priva- tigators, such as crime scene analysis personal and A recent literature survey also shows that there cy-by-design, where requirements for investigators, forensic laboratories, and the digi- is a need for effective visualization of evidential data forensics are integrated into relevant tal archives, using cloud computing. For example, for forensic practitioners and investigators, as point- phases of the system development life- Schiliro and Choo presented a cloud-based interac- ed out by the authors “while many researchers have Conceptually, forensic-as-a-service cycle, with the objective of developing tive constable on patrol system, which allows a law made progress towards a model for visualizing foren- is similar to software-as-a-service forensic-ready systems. The utility of enforcement agency (or any other private sector or- sic data, there continue to be gaps in this research such an approach is demonstrated in a ganization) to deliver the organization’s capabilities area which need to be addressed”.24,25 where forensic applications and latter work.17 to the frontline of cer via a mobile app.23 This in- When designing cloud forensic techniques, it services are being moved to the cloud. There has also been research into cludes the capability to connect and pull/push infor- is also important to balance the need for a secure offering forensic-as-a-service. Concep- mation and intelligence from a wide range of public mobile telecommunications system and the rights of tually, forensic-as-a-service is similar and private databases (for example CCTV systems in individuals to privacy against the need to protect the to software-as-a-service where foren- a particular city, such as San Antonio), employing community from serious and organized crimes and sic applications and services are being data-mining and other big data analytical technolo- cyber and national security interests. This issue has tus. The level of trust required to execute each of moved to the cloud. For example, Castiglione and gies, and so on. serious implications on the ability of governments to the collection procedures was also reported in the colleagues presented a cloud-based methodology to As cloud and related technologies advance, fo- protect their citizens against serious and organized study. In a latter work, the same authors contributed acquire forensic evidence from online services, such rensic investigators will nd it challenging to keep crimes. However, it remains an under-researched a forensic toolkit for the OpenStack cloud platform as webpages, chats, documents, photos and videos.18 pace, in the sense of identifying new forensic ar- area due to the interdisciplinary challenges speci c – FROST.12 FROST allows a remote user to collect A cloud-based solution hosts a network trusted ser- tifacts. Thus, there is a need for ongoing research to cloud (and digital) forensics. Thus, it is important an image of the users’ VMs hosted in OpenStack, vice used to acquire evidence for subsequent analy- into identifying new forensic artefacts in the cloud to bring together approaches from different disci- and retrieve log events for all API requests made by sis. Such an acquisition can be undertaken using a and related environment (for example multi-cloud plines to address the major contemporary challenges the user and rewall logs for all of the users’ VMs. HTTPS proxy (capable of recording activities at the and federated cloud, fog computing, edge comput- associated with cloud forensics. For instance, to en- FROST is integrated with several OpenStack Dash- network level, such as IP, when an online service is ing, and Internet of Things, such as Internet of Bat- sure individual privacy, the techniques developed by board and Compute components. accessed), or a software agent for the collection of in- tle eld Things), considering both data-at-rest and forensic researchers should focus on individual sus- Martini and Choo presented a four-stage cloud formation obtained by the targeted online service in a data-in-transit, as well as developing new forensi- pect devices under direct judicial oversight (for ex- forensic framework, and used it to guide their server What You See Is What You Get (WYSIWYG) manner. cally sound data collection techniques. ample under a search warrant), as opposed to broad and client analysis of the ownCloud private Stor- Along with his colleagues, van Beek proposed a Current forensic techniques generally make spectrum surveillance, such as the NSA incident re- age.13,14 The authors successfully recovered a range cloud-based approach which allows one to process use of vendor data communication facilities built vealed by Snowden in 2013. of artifacts, including le data, metadata and au- and investigate the large volume of seized digital into the mobile devices (such as iTunes backups thentication credentials. Then they analyzed the materials,19 typically of a criminal investigation. for iOS devices) for the purpose of forensic extrac- References server component of ownCloud. In addition to locat- This was also coined big data forensics by Quick tion. Often this limits the potential for data extrac- 1. D. Ó Coileáin, and D. O’mahony, “Accounting ing a range of metadata and uploaded les (including and Choo.20 Speci cally, digital evidence obtained tion. For example, current tools would not be able and Accountability in Content Distribution

www.computer.org/computingedge 31 16 IEEE CLOUD COMPUTING WWW.COMPUTER.ORG/CLOUDCOMPUTING MAY/JUNE 2017 IEEE CLOUD COMPUTING 17

d3law.indd 16 7/21/17 9:20 AM d3law.indd 17 7/21/17 9:20 AM CLOUD AND THE LAW

Architectures: A Survey”, ACM Computing Sur- tal Investigation, vol. 10, Supplement, pp. S87- veys, vol. 47, no. 4, art. 59, May 2015. S95, 2013. 2. B. Martini, and K.-K. R. Choo, Cloud Forensic 13. B. Martini, and K.-K. R. Choo, Cloud storage fo- Technical Challenges and Solutions: A Snapshot, rensics: ownCloud as a case study, Digital Investi- IEEE Cloud Computing, vol. 1, no. 4, pp. 20-25, gation, vol. 10, no. 4, pp: 287-299, December 2013. 2014. 14. B. Martini, K.-K. R. Choo, An integrated concep- 3. K. Oestreicher, “A forensically robust method for tual digital forensic framework for cloud comput- acquisition of iCloud data”, Digital Investigation, ing, Digital Investigation, vol. 9, no. 2, pp: 71-80, vol. 11, Supplement 2, pp: S106-S113, August November 2012. 2014. 15. B. Martini and K.-K. R. Choo, Remote Program- 4. B. Martini, Q. Do and K.-K. R. Choo, “Digital fo- matic vCloud Forensics: A Six-Step Collection rensics in the cloud era: The decline of passwords Process and a Proof of Concept, Proceedings of and the need for legal reform”. Trends & Issues in the 2014 IEEE 13th International Conference Crime and Criminal Justice, vol. 512, pp. 1–16, on Trust, Security and Privacy in Computing and 2016. Communications (TRUSTCOM ‘14), pp: 935- 5. V. Roussev, I. Ahmed, A. Barreto, S. McCulley, 942, 2014. and V. Shanmughan, “Cloud forensics–Tool de- 16. N. H. Ab Rahman, W. B. Glisson, Y. Yang, K.- velopment studies & future outlook”, Digital In- K. R. Choo, Forensic-by-Design Framework vestigation, vol. 18, pp: 79-95, 2016. for Cyber-Physical Cloud Systems, IEEE Cloud 6. C. Hooper, B. Martini, K.-K. R. Choo, Cloud Computing, vol. 3, no. 1, pp: 50-59, 2016. computing and its implications for cybercrime 17. N. H. Ab Rahman, N. D. W. Cahyani, and K.-K. investigations in Australia, Computer Law & Se- R. Choo, “Cloud incident handling and forensic- curity Review, vol. 29, no. 2, pp. 152-163, 2013. by-design: cloud storage as a case study”, Con- 7. D. J. B. Svantesson, and L. van Zwieten, Law en- currency and Computation: Practice and Experi- forcement access to evidence via direct contact ence, 2017. http://dx.doi.org/10.1002/cpe.3868 with cloud providers – identifying the contours 18. A. Castiglione, G. Cattaneo, G. De Maio, A. De of a solution, Computer Law & Security Review, Santis, and G. Roscigno, “A Novel Methodol- vol. 32, no. 5, pp. 671-682, 2016. ogy to Acquire Live Big Data Evidence from the 8. S. Mason, and E. George, Digital evidence and Cloud”, IEEE Transactions on Big Data, 2017. ‘cloud’ computing, Computer Law & Security Re- https://doi.org/10.1109/TBDATA.2017.2683521 view, vol. 27, no. 5, pp. 524-528, 2011. 19. H.M.A. van Beek, E.J. van Eijk, R.B. van Baar, 9. B. Martini, and K.-K. R. Choo, An integrated M. Ugen, J.N.C. Bodde, and A.J. Siemelink, conceptual digital forensic framework for cloud “Digital forensics as a service: Game on”, Digital computing, Digital Investigation, vol. 9, no. 2, pp. Investigation, vol. 15, pp: 20-38, December 2015. 71-80, 2012. 20. D. Quick, and K.-K. R. Choo, “Big Forensic Data 10. S. Zawoad, A. K. Dutta and R. Hasan, “Towards Management in Heterogeneous Distributed Sys- Building Forensics Enabled Cloud Through Se- tems: Quick Analysis of Multimedia Forensic cure Logging-as-a-Service”, IEEE Transactions Data”, Software: Practice and Experience, 2017. on Dependable and Secure Computing, vol. 13, http://dx.doi.org/10.1002/spe.2429 no. 2, pp. 148-162, 2016. 21. X. Fu, Z. Ling, W. Yu, and J. Luo, “Cyber Crime 11. J. Dykstra, and A. T. Sherman, Acquiring fo- Scene Investigations (C²SI) through Cloud Com- rensic evidence from infrastructure-as-a-service puting”, Proceedings of the IEEE 30th Interna- cloud computing: Exploring and evaluating tools, tional Conference on Distributed Computing Sys- trust, and techniques, Digital Investigation, vol. tems Workshops, pp: 26-31, 2010. 9, Supplement, pp. S90-S98, 2012. 22. K.-K. R. Choo, Harnessing information and com- 12. J. Dykstra, and A. T. Sherman, Design and im- munications technologies in community polic- plementation of FROST: Digital forensic tools for ing, in Judy Putt, editor, Community policing in the OpenStack cloud computing platform, Digi- Australia, Research and Public Policy, vol. 111, pp.

32 Computing Edge August 2017 18 IEEE CLOUD COMPUTING WWW.COMPUTER.ORG/CLOUDCOMPUTING

d3law.indd 18 7/21/17 9:20 AM CLOUD AND THE LAW

Architectures: A Survey”, ACM Computing Sur- tal Investigation, vol. 10, Supplement, pp. S87- 67–75, available at http://www.aic.gov.au/media ANIELLO CASTIGLIONE is an adjunct professor at veys, vol. 47, no. 4, art. 59, May 2015. S95, 2013. _librar y/publications/rpp/111/rpp111.pdf, 2011. the University of Salerno (Italy) and at the University 2. B. Martini, and K.-K. R. Choo, Cloud Forensic 13. B. Martini, and K.-K. R. Choo, Cloud storage fo- 23. F. Schiliro, and K.-K. R. Choo, The Role of Mo- of Naples “Federico II” (Italy). His research interests Technical Challenges and Solutions: A Snapshot, rensics: ownCloud as a case study, Digital Investi- bile Devices in Enhancing the Policing System to include security, communication networks, infor- IEEE Cloud Computing, vol. 1, no. 4, pp. 20-25, gation, vol. 10, no. 4, pp: 287-299, December 2013. Improve Ef ciency and Effectiveness: A Practi- mation forensics and security, applied cryptography. 2014. 14. B. Martini, K.-K. R. Choo, An integrated concep- tioner’s Perspective,. In Au M H and Choo K-K Castiglione has a PhD in computer science from the 3. K. Oestreicher, “A forensically robust method for tual digital forensic framework for cloud comput- R, editors, Mobile Security and Privacy: Advances, University of Salerno, Italy. He is member of several acquisition of iCloud data”, Digital Investigation, ing, Digital Investigation, vol. 9, no. 2, pp: 71-80, Challenges and Future Research Directions, pp. associations, including IEEE and ACM. Contact him vol. 11, Supplement 2, pp: S106-S113, August November 2012. 85–99, Syngress, an Imprint of Elsevier. http:// at [email protected]. 2014. 15. B. Martini and K.-K. R. Choo, Remote Program- dx.doi.org/10.1016/B978-0-12-804629-6.00005-5. 4. B. Martini, Q. Do and K.-K. R. Choo, “Digital fo- matic vCloud Forensics: A Six-Step Collection 24. C. Tassone, B. Martini, and K.-K. R. Choo. Fo- rensics in the cloud era: The decline of passwords Process and a Proof of Concept, Proceedings of rensic Visualization: Survey and Future Research Read your subscriptions through and the need for legal reform”. Trends & Issues in the 2014 IEEE 13th International Conference Directions. In Choo K-K R and Dehghantanha This article originally appeared in Crime and Criminal Justice, vol. 512, pp. 1–16, on Trust, Security and Privacy in Computing and A, editors, Contemporary Digital Forensic Inves- the myCS publications portal at IEEE Cloud Computinghttp://mycs.computer.org., vol. 4, no. 3, 2017. 2016. Communications (TRUSTCOM ‘14), pp: 935- tigations of Cloud and Mobile Applications, pp. 5. V. Roussev, I. Ahmed, A. Barreto, S. McCulley, 942, 2014. 163–184, Syngress, an Imprint of Elsevier. http:// and V. Shanmughan, “Cloud forensics–Tool de- 16. N. H. Ab Rahman, W. B. Glisson, Y. Yang, K.- dx.doi.org/10.1016/B978-0-12-805303-4.00011-3. velopment studies & future outlook”, Digital In- K. R. Choo, Forensic-by-Design Framework 25. C. Tassone, B. Martini, and K.-K. R. Choo, “Vi- vestigation, vol. 18, pp: 79-95, 2016. for Cyber-Physical Cloud Systems, IEEE Cloud sualizing Digital Forensic Datasets: A Proof of 6. C. Hooper, B. Martini, K.-K. R. Choo, Cloud Computing, vol. 3, no. 1, pp: 50-59, 2016. Concept”, Journal of Forensic Sciences,2017. IEEE-CSIEEE-CS computing and its implications for cybercrime 17. N. H. Ab Rahman, N. D. W. Cahyani, and K.-K. http://dx.doi.org/10.1111/1556-4029.13431 investigations in Australia, Computer Law & Se- R. Choo, “Cloud incident handling and forensic- CHARLESCHARLES BABBAGEBABBAGE curity Review, vol. 29, no. 2, pp. 152-163, 2013. by-design: cloud storage as a case study”, Con- 7. D. J. B. Svantesson, and L. van Zwieten, Law en- currency and Computation: Practice and Experi- AWARDAWARD forcement access to evidence via direct contact ence, 2017. http://dx.doi.org/10.1002/cpe.3868 KIM-KWANG RAYMOND CHOO holds the CALLCALL FOR FOR AWARD AWARD NOMINATIONS NOMINATIONS with cloud providers – identifying the contours 18. A. Castiglione, G. Cattaneo, G. De Maio, A. De Cloud Technology Endowed Professorship in the De- DeadlineDeadline 1 15 October October 2017 2017 of a solution, Computer Law & Security Review, Santis, and G. Roscigno, “A Novel Methodol- partment of Information Systems and Cyber Security vol. 32, no. 5, pp. 671-682, 2016. ogy to Acquire Live Big Data Evidence from the at the University of Texas at San Antonio. His research ABOUTABOUT THE THE IEEE-CS IEEE-CS CHARLES CHARLES BABBAGE BABBAGE 8. S. Mason, and E. George, Digital evidence and Cloud”, IEEE Transactions on Big Data, 2017. interests include cyber and information security and AWARDAWARD ‘cloud’ computing, Computer Law & Security Re- https://doi.org/10.1109/TBDATA.2017.2683521 digital forensics. He is a senior member of IEEE, a EstablishedEstablished in in memory memory of of Charles Charles Babbage Babbage in in view, vol. 27, no. 5, pp. 524-528, 2011. 19. H.M.A. van Beek, E.J. van Eijk, R.B. van Baar, Fellow of the Australian Computer Society, an Hon- recognitionrecognition of of significant significant contributionscontributions in in the the fieldfield of parallel computation. The candidate would have 9. B. Martini, and K.-K. R. Choo, An integrated M. Ugen, J.N.C. Bodde, and A.J. Siemelink, orary Commander, 502nd Air Base Wing, Joint Base of parallel computation. The candidate would have mademade an an outstanding, outstanding, innovative innovative contribution contribution or or conceptual digital forensic framework for cloud “Digital forensics as a service: Game on”, Digital San Antonio-Fort Sam Houston, USA, and has a contributionscontributions to to parallel parallel computation. computation. It It is is hoped, hoped, but but computing, Digital Investigation, vol. 9, no. 2, pp. Investigation, vol. 15, pp: 20-38, December 2015. PhD in information security from Queensland Univer- notnot required, required, that that the the winner winner will will have have also also contributed contributed to the parallel computation community through 71-80, 2012. 20. D. Quick, and K.-K. R. Choo, “Big Forensic Data sity of Technology, Australia. Contact him at raymond to the parallel computation community through teaching,teaching, mentoring, mentoring, or or community community service. service. 10. S. Zawoad, A. K. Dutta and R. Hasan, “Towards Management in Heterogeneous Distributed Sys- [email protected]. Building Forensics Enabled Cloud Through Se- tems: Quick Analysis of Multimedia Forensic AWARDAWARD & & PRESENTATION PRESENTATION A certificate and a $1,000 honorarium presented to a cure Logging-as-a-Service”, IEEE Transactions Data”, Software: Practice and Experience, 2017. A certificate and a $1,000 honorarium presented to a singlesingle recipient. recipient. The The on Dependable and Secure Computing, vol. 13, http://dx.doi.org/10.1002/spe.2429 CHRISTIAN ESPOSITO received the Ph.D. de- winner will be invited winner will be invited NOMINATION SITE toto present present a a paper paper NOMINATION SITE no. 2, pp. 148-162, 2016. 21. X. Fu, Z. Ling, W. Yu, and J. Luo, “Cyber Crime gree in computer engineering and automation from awards.computer.org and/orand/or presentation presentation awards.computer.org 11. J. Dykstra, and A. T. Sherman, Acquiring fo- Scene Investigations (C²SI) through Cloud Com- the University of Napoli “Federico II”, Italy. He is an at the annual IEEE- at the annual IEEE- AWARDS HOMEPAGE rensic evidence from infrastructure-as-a-service puting”, Proceedings of the IEEE 30th Interna- adjunct professor at the University of Naples “Federi- CSCS International International AWARDS HOMEPAGE www.computer.org/awardswww.computer.org/awards cloud computing: Exploring and evaluating tools, tional Conference on Distributed Computing Sys- co II”, Italy, and at the University of Salerno, Italy, ParallelParallel and and Distributed Processing trust, and techniques, Digital Investigation, vol. tems Workshops, pp: 26-31, 2010. where he is also a research fellow. His research in- Distributed Processing CONTACTCONTACT US US SymposiumSymposium [email protected]@computer.org 9, Supplement, pp. S90-S98, 2012. 22. K.-K. R. Choo, Harnessing information and com- terests include reliable and secure communications, (IPDPS(IPDPS 2017). 2017). 12. J. Dykstra, and A. T. Sherman, Design and im- munications technologies in community polic- middleware, distributed systems, positioning systems, plementation of FROST: Digital forensic tools for ing, in Judy Putt, editor, Community policing in multi-objective optimization, and game theory. Con- the OpenStack cloud computing platform, Digi- Australia, Research and Public Policy, vol. 111, pp. tact him at [email protected].

www.computer.org/computingedge 33 18 IEEE CLOUD COMPUTING WWW.COMPUTER.ORG/CLOUDCOMPUTING MAY/JUNE 2017 IEEE CLOUD COMPUTING 19

d3law.indd 18 7/21/17 9:20 AM d3law.indd 19 7/21/17 9:20 AM Beyond Wires Editor: Yih-Farn Robin Chen • [email protected] FocusStack: Orchestrating Edge Clouds Using Focus of Attention

Brian Amento, Robert J. Hall, Kaustubh Joshi, and K. Hal Purdy • AT&T Labs Research

Managing resources in Internet of Things (IoT) edge devices presents new challenges: massive scale, novel security issues, and new resource types. We propose that edge devices can be managed as part of Infrastructure-as-a-Service clouds. Our approach, FocusStack, uses situational awareness to solve problems of inef cient messaging and mixed-initiative control that IoT device management raises for traditional cloud platforms.

dge computing (also known as cloudlets or tially from different developers, are deployed to fog computing) has traditionally focused on these devices. These application instances coor- E small cloud data centers associated with the dinate with other instances to perform tasks last mile of the Internet. Increasingly, however, with local or platform-wide scope. Control plane even the small form factor devices that connect nodes in the cloud orchestrate the management to the last mile, such as television set-top boxes, of these “distributed virtual data centers” of edge network gateways, WiFi access points, cars, and devices, and allow both tenants and administra- drones, present an interesting target for building tors to interact with this edge cloud. Through this a managed cloud computing platform that can interface, tenants deploy and update applications, serve a rich set of new applications. Applications configure secure application networking, and range from traditional network edge services provide access to storage resources. such as content caches or WAN accelerators, to Despite similarities to the traditional cloud more novel ones such as privacy-preserving big IaaS model, such an approach presents unique data analytics on set-top boxes, connected cars, challenges. First, edge devices often have lim- Internet of Things (IoT) sensor-sharing applica- ited compute and memory resources, and in the tions, and applications allowing users to lease case of drones, limited energy as well. Second, sensors and computation resources in drones. the network environment is dramatically differ- We propose that distributed edge devices ent from the typical data center, complete with should be managed similarly to an Infrastructure- nodes that might be moving constantly and as-a-Service (IaaS) cloud computing data center have intermittent connectivity with scarce and by extending traditional cloud orchestration variable available bandwidth. Third, the ratio tools. We assume that the devices are managed, of compute to control nodes is dramatically at least in part, by a controlling entity, such as a different. With potentially millions of devices cable provider with a collection of set-top boxes, attaching to a small set of cloud controllers, an operator deploying a eet of drones, or an auto the control plane must be highly efcient. Last, manufacturer providing a managed computing these devices often need mixed-initiative man- platform in its cars. This conceptual framework agement that’s distinct from traditional cloud considers each mobile edge device as analogous provider/tenant separation (we discuss this to a compute server. Tenant applications, poten- more later).

56 Published by the IEEE Computer Society 1089-7801/17/$33.00 © 2017 IEEE IEEE INTERNET COMPUTING 34 August 2017 Published by the IEEE Computer Society 2469-7087/17/$33.00 © 2017 IEEE Beyond Wires FocusStack: Orchestrating Edge Clouds Using Focus of Attention Editor: Yih-Farn Robin Chen • [email protected]

We address these challenges using teristics: as a rst example, consider Drones for hire. RentMyEyes ies a FocusStack: Orchestrating an intelligent approach to messag- customer premise devices such as set- eet of connected drones equipped ing based on the concept of focus of top boxes, edge routers, or WiFi access with cameras and environmen- attention. The cloud control plane’s points. These devices could provide tal sensors. These drones wait for Edge Clouds Using Focus operations are scoped based on con- a number of services ranging from remote sensing jobs to be submitted text that includes the edge device’s usage analytics to environment sens- over the Internet. Each job is rep- location, health, capabilities, and ing. As a second example, consider resented by a target area the drone of Attention user authorization preferences. We that cars are rich sensor platforms must y to, along with an app that call this capability location-based not just due to the wealth of data they the drone should run once there. situational awareness. Devices that collect about themselves, but also for Once at the target, the app is autho- Brian Amento, Robert J. Hall, Kaustubh Joshi, aren’t in the current focus of atten- their ability to measure their environ- rized to collect and analyze data and K. Hal Purdy • AT&T Labs Research tion are neither tracked by the cloud ment and the driving habits of those from the drone’s camera and sen- control plane, nor participate in any who operate them. In addition to sors in real time, and potentially Managing resources in Internet of Things (IoT) edge devices presents new chal- control plane protocols. Doing so not constraints on compute and memory, adjust the drone’s ight plan based only minimizes the resource uti- cars have additional challenges due to on its analysis. On receiving the job, lenges: massive scale, novel security issues, and new resource types. We pro- lization of the edge devices, since mobility and variable network con- RentMyEyes can use FocusStack to pose that edge devices can be managed as part of Infrastructure-as-a-Service they don’t need to provide periodic nectivity. A third example, drones, are identify a drone close to the target clouds. Our approach, FocusStack, uses situational awareness to solve problems updates to the cloud, but it also the ultimate mobile platforms experi- area with suf cient energy left, and allows the cloud control plane to be encing severe energy constraints and deploy the app to it. of inef cient messaging and mixed-initiative control that IoT device management more ef cient and scalable, since it extreme variability in network condi- raises for traditional cloud platforms. only needs to handle a small subset tions, as they y in and out of radio Need for Situational Awareness of devices at any one time. Finally, range. General-purpose drone plat- In the aforementioned scenarios, such dynamic scoping is essential forms can be useful as a platform as situational awareness plays a key for handling edge devices that are well as for environmental sensing and role in not just the semantics of dge computing (also known as cloudlets or tially from different developers, are deployed to constantly moving and might be dis- tracking. The following are additional the service, but also in enabling fog computing) has traditionally focused on these devices. These application instances coor- connected from the network at any examples. ef ciency and scalability. Assum- E small cloud data centers associated with the dinate with other instances to perform tasks given time; such devices are simply ing that the cloud can’t afford to last mile of the Internet. Increasingly, however, with local or platform-wide scope. Control plane excluded from the focus of attention Viewership analytics. CableCorp actively monitor the operations of even the small form factor devices that connect nodes in the cloud orchestrate the management of the current orchestration task. wants to understand differences every edge device at all times, we to the last mile, such as television set-top boxes, of these “distributed virtual data centers” of edge We have built such a cloud, between TV viewing habits in Los need an architecture that can focus network gateways, WiFi access points, cars, and devices, and allow both tenants and administra- called FocusStack, by combining Angeles and New York City. Using attention on the devices in an area of drones, present an interesting target for building tors to interact with this edge cloud. Through this OpenStack (see www.openstack. FocusStack’s geoaddressing primi- interest at a time of interest, extract a managed cloud computing platform that can interface, tenants deploy and update applications, org) — one of the most popular open tives, CableCorp can identify and information, and take action on that serve a rich set of new applications. Applications configure secure application networking, and source cloud management platforms deploy two Hadoop instances in the information. For example, Highway range from traditional network edge services provide access to storage resources. — with the AT&T Labs Geocast Sys- target regions. Thus, CableCorp can I-5 is a long, straight road run- such as content caches or WAN accelerators, to Despite similarities to the traditional cloud tem (ALGS),1 a multi-tiered geo- compute the aggregate results they ning the length of California along more novel ones such as privacy-preserving big IaaS model, such an approach presents unique graphic addressing (GA) network need without ever collecting the which, most of the time, nothing data analytics on set-top boxes, connected cars, challenges. First, edge devices often have lim- subsystem that allows packets to individual users’ TV viewing history. interesting happens. While thou- Internet of Things (IoT) sensor-sharing applica- ited compute and memory resources, and in the be sent to devices in a geographic sands of cars drive on I-5 each day, tions, and applications allowing users to lease case of drones, limited energy as well. Second, region instead of a speci c set of IP Car diagnostics. Fast Motors Inc. wishes only a tiny fraction of them would sensors and computation resources in drones. the network environment is dramatically differ- endpoints, as in IP unicast or mul- to understand how cold ambient temper- be of interest to the Car Diagnostics We propose that distributed edge devices ent from the typical data center, complete with ticast. FocusStack can be deployed ature affects engine performance. While service. The health, computational should be managed similarly to an Infrastructure- nodes that might be moving constantly and on an unmodi ed installation of it isn’t feasible (for volume and privacy state, and opt-in state of the rest as-a-Service (IaaS) cloud computing data center have intermittent connectivity with scarce and OpenStack, and can deploy appli- reasons) to continuously upload detailed are irrelevant to the Car Diagnostics by extending traditional cloud orchestration variable available bandwidth. Third, the ratio cations that are packaged as light- diagnostics data from all cars at all times, application and would require sig- tools. We assume that the devices are managed, of compute to control nodes is dramatically weight Docker (see www.docker.com) it’s possible to write a simple one-time ni cant cellular data bandwidth to at least in part, by a controlling entity, such as a different. With potentially millions of devices OS container instances to “compute app to read internal data from speci c report at all times. For the tiny frac- cable provider with a collection of set-top boxes, attaching to a small set of cloud controllers, nodes” running on edge devices. cars and run analytics to access engine tion of cars of interest, we require an operator deploying a eet of drones, or an auto the control plane must be highly efcient. Last, performance. They use FocusStack a method for gaining up-to-date manufacturer providing a managed computing these devices often need mixed-initiative man- Motivating Examples to deploy the app to a small sample of intelligence on their computational platform in its cars. This conceptual framework agement that’s distinct from traditional cloud FocusStack can be used to manage cars in New England. When the study and resource states. considers each mobile edge device as analogous provider/tenant separation (we discuss this clouds that comprise a variety of end- is complete, the app is no longer needed Motivated by these consid- to a compute server. Tenant applications, poten- more later). point types with a range of charac- and can be removed from the cars. erations, our primary requirement

The AT&T Labs Geocast System

e detail a schematic of the AT&T Labs Geocast System near the destination region, where it will be relayed again across W (ALGS) in Figure A.1 The ALGS implements a seam- the ad hoc WiFi tier to devices in the region. lessly integrated, two-tier network geographic addressing (GA) Because the car diagnostics service makes use of edge devices service. A packet’s address, referred to as its geocast region, is in connected vehicles, they don’t have WiFi capability, so that de ned by a circle, where the packet header contains latitude service depends entirely upon ALGS’s long-range tier. See else- and longitude of the center of the circle and the radius in meters. where for more details about ALGS and its use by FocusStack.1,2 Packets sent via the ALGS can transit either an ad hoc WiFi tier or a long-range tier mediated by an Internet-based georouting References service accessed through the 3G/4G/LTE/GSM (Global System 1. R. Hall et al, “Scaling up a Geographic Addressing System,” Proc. 2013 IEEE for Mobile Communications) system. Packets can be relayed Military Comm. Conf., 2013; doi:10.1109/MILCOM.2013.34. across either tier or both tiers; in some cases, a packet originat- 2. B. Amento et al., “FocusStack: Orchestrating Edge Clouds Using Loca- ing in one ad hoc tier can be transferred to a long-range-capable tion-Based Focus of Attention,” Proc. 2016 IEEE/ACM Symp. Edge Com- device, which will relay it over the long-range tier to a device puting, 2016, pp. 179–191.

Long-range extension server

Virtual Botswana server Georouter server GRDB

Bot

Bot Internet

GSM GSM GSM (3G/4G/LTE) (3G/4G/LTE) (3G/4G/LTE)

Smartphone Smartphone Smartphone WiFi device

802.11 ad hoc WiFi SAGP geocast

Figure A. The AT&T Labs Geocast System. GRDB = georouter database; GSM = Global System for Mobile Communications; and SAGP = Scalable ad hoc Geocast Protocol.

for FocusStack’s awareness func- where the application needs to be devices in a particular area. The Car tion is to obtain awareness infor- deployed and active on edge devices Diagnostics application is only inter- mation when attention is focused located within a circumscribed geo- ested in a subset of a manufacturer’s on a geographic area. Consequently, graphical area. cars on the road. A remote sens- applications well-suited for use with A second key requirement is moti- ing application is interested only in FocusStack are like those described vated by the observation that an appli- the environmental sensors provid- in the previous examples — that is, cation clearly isn’t interested in all IoT ing information of interest. Thus,

58 www.computer.org/internet/ IEEE INTERNET COMPUTING 36 Computing Edge August 2017 Beyond Wires FocusStack: Orchestrating Edge Clouds Using Focus of Attention

The AT&T Labs Geocast System FocusStack’s awareness component must be capable of limiting the scope Geocast query e detail a schematic of the AT&T Labs Geocast System near the destination region, where it will be relayed again across of queries so that only a narrow subset Containers Cloud-based (ALGS) in Figure A.1 The ALGS implements a seam- the ad hoc WiFi tier to devices in the region. Geocast W of all IoT devices in an area will even Georouter Nova Compute node lessly integrated, two-tier network geographic addressing (GA) Because the car diagnostics service makes use of edge devices reply. GClib Nova service. A packet’s address, referred to as its geocast region, is in connected vehicles, they don’t have WiFi capability, so that Finally, different applications VMs de ned by a circle, where the packet header contains latitude service depends entirely upon ALGS’s long-range tier. See else- require different awareness informa- Edge devices FocusStack API SAMonitor and longitude of the center of the circle and the radius in meters. where for more details about ALGS and its use by FocusStack.1,2 tion. The Drones For Hire application Apps

Packets sent via the ALGS can transit either an ad hoc WiFi tier is interested only in the computa- OpenStack OpenStack LSA subsystem Containers nova control or a long-range tier mediated by an Internet-based georouting References tional, communications, energy, and OSE subsystem service accessed through the 3G/4G/LTE/GSM (Global System 1. R. Hall et al, “Scaling up a Geographic Addressing System,” Proc. 2013 IEEE opt-in state of participating drones. OpenStack control LSA awareness messages for Mobile Communications) system. Packets can be relayed Military Comm. Conf., 2013; doi:10.1109/MILCOM.2013.34. It isn’t interested in information rel- GClib Nova OSE messages when in across either tier or both tiers; in some cases, a packet originat- 2. B. Amento et al., “FocusStack: Orchestrating Edge Clouds Using Loca- evant to other applications, such as focus of aGenHon Edge devices ing in one ad hoc tier can be transferred to a long-range-capable tion-Based Focus of Attention,” Proc. 2016 IEEE/ACM Symp. Edge Com- remote auto maintenance data in OSE always-on messages device, which will relay it over the long-range tier to a device puting, 2016, pp. 179–191. road vehicles, humidity data from nearby farm eld sensors, bat- Figure 1. FocusStack architecture. LSA = Location-based Situational Awareness; tery level information from nearby OSE = OpenStack Extension; and VMs = virtual machines. Long-range extension server smartphones, and so on. However, other applications might want these Virtual Botswana server Georouter server GRDB other types of information. Thus, picture to our OpenStack Extension framework, whose monitoring com- FocusStack must be able to query for (OSE) subsystem that allows deploy- ponent, SAMonitor, is based on the Bot custom sets of application-speci c ment, execution, and management Field Common Operating Picture awareness information. of containers on small edge comput- (FCOP) algorithm,2 which is a dis- Bot When multiple edge devices meet ing devices with limited network- tributed algorithm using geographi- the criteria relevant to a particular ing capabilities. Figure 1 shows the cally addressed messaging. Bot Internet application, the set of edge devices overall architecture of FocusStack, meeting the application criteria which forms a hybrid cloud con- • In geographic addressing (GA), a is presented to the application for sisting of both edge devices run- packet’s address identi es a subset GSM GSM GSM further application-speci c selec- ning lightweight Linux containers of physical space. When sent, (3G/4G/LTE) (3G/4G/LTE) (3G/4G/LTE) tion. The individual application then (based on Docker), and cloud-based this means that the packet will decides on which of the edge devices compute nodes that can run virtual be transferred to all devices cur- to invoke application elements. machines (VMs) as with a traditional rently in that space. A GA service Smartphone Smartphone IaaS cloud. is implemented in the network and Smartphone WiFi device FocusStack Architecture When a cloud operation (such as appears to the programmer as an FocusStack is an architecture deploying a new container instance) API analogous to (and in paral- that supports deploying heteroge- is invoked by calling the appropriate lel with) the IP stack. FocusStack neous applications to a diverse set FocusStack API, the LSA subsystem uses the ALGS1 for GA messaging 802.11 ad hoc WiFi of IoT edge devices. These devices is rst used to scope this request by (see the related sidebar). SAGP geocast are potentially limited in compute building an operating picture (we • The FCOP algorithm2 is a GA- power, energy, and connectivity and discuss this more in the following based distributed algorithm are frequently mobile. Our platform section) for use in seeding the appro- designed to enable each device to Figure A. The AT&T Labs Geocast System. GRDB = georouter database; GSM = Global System for Mobile enables developers to focus on their priate OpenStack operation to be update all others on its current Communications; and SAGP = Scalable ad hoc Geocast Protocol. application rather than on nding carried out by OSE. awareness information in an ef - and tracking the various edge com- cient and scalable manner. puting devices where they will be LSA Subsystem • GCLib is the software framework deployed. The LSA subsystem allows FocusStack supporting LSA that provides for FocusStack’s awareness func- where the application needs to be devices in a particular area. The Car There are two major architec- to obtain awareness information on- components access to GA mes- tion is to obtain awareness infor- deployed and active on edge devices Diagnostics application is only inter- tural components that together com- demand when the focus of attention saging, access to sharing of arbi- mation when attention is focused located within a circumscribed geo- ested in a subset of a manufacturer’s prise the FocusStack platform. The is directed to a particular geographic trary data within the device (car, on a geographic area. Consequently, graphical area. cars on the road. A remote sens- Location-based Situational Aware- area and, equally importantly, to drone, and so on), and automatic applications well-suited for use with A second key requirement is moti- ing application is interested only in ness (LSA) subsystem is based on the stop the awareness messaging once support for the query/response FocusStack are like those described vated by the observation that an appli- the environmental sensors provid- AT&T Labs Geocast System (ALGS).1 focus is no longer directed there. awareness function. Full details in the previous examples — that is, cation clearly isn’t interested in all IoT ing information of interest. Thus, It provides an awareness operating LSA is implemented in the GCLib are provided elsewhere.3

58 www.computer.org/internet/ IEEE INTERNET COMPUTING JANUARY/FEBRUARY 2017 59 www.computer.org/computingedge 37 Beyond Wires

• The SAMonitor component is the acts with a local Docker instance to Regarding mixed-initiative con- heart of our situational aware- launch and manage containers. Con- trol — the presence of stakeholders ness technique. When attention is tainers running on the edge nodes like the device’s owner or opera- focused on an area, here mean- are provided full OpenStack ser- tor, in addition to traditional cloud ing a circle de ned by center vices, including access to con gu- operators and application owners, latitude/longitude and radius in rable virtual networks. These virtual requires changes to management meters, an SAMonitor component networks are implemented using and access control. We’ve taken rst is created that periodically sends OpenStack’s standard LinuxBridge steps in addressing the problem of the awareness query determined neutron plugin. Connectivity across multiple stakeholders managing the by the application, and collects the LTE cellular network between same device by our focus of atten- an operating picture consisting of the edge nodes and layer 3 network tion concept. A FocusStack control the timestamped query responses nodes in the cloud occurs over IP plane, F, by rebuilding its aware- sent back by all devices report- Security (IPsec) tunnels. ness operating picture each time ing from the monitored area. attention is focused on an area, can This periodic querying is main- make accurate management deci- tained throughout the time that e proposed to treat huge num- sions even when other stakeholders attention is focused, so that new W bers of real-world IoT devices might have carried out their own devices entering the area during as members of a cloud, so that the management operations on devices this time are added to the oper- rich set of tools and applications in the area while those devices were ating picture and become avail- developed for IaaS cloud comput- out of F’s focus. However, in addi- able to the application or service. ing can be brought to bear for the tion, the device must also be pro- The awareness query is program- IoT. We postulate that control plane tected from cloud-initiated actions. mable and is tailored to the needs scalability and communication For example, when the application of the application or service using complexity are major challenges in owner asks the cloud to deploy an a particular SAMonitor instance. doing so. In response, we proposed a application on an edge device, the Example query terms include solution based on dynamic focus of edge node owner’s preferences must location, velocity, computational attention, implemented within our be accounted for. In the future, we state, and battery level. location-based situational aware- intend to develop a policy frame- ness technique. Our initial studies work that allows a device owner Now that we’ve detailed the LSA sub- show this can dramatically reduce to express conditions under which system, let’s look at the OSE subsystem. message traf c and control plane applications are authorized to exe- overhead by more than four orders cute orchestration actions on the OpenStack Extensions of magnitude.3 device. (OSE) Subsystem However, challenges remain that With the resolution of these chal- In a standard OpenStack environ- must be tackled in the future; namely, lenges, we believe that an IoT edge ment, VMs are deployed and man- security and mixed-initiative con- cloud can provide the means to cre- aged on compute nodes comprised trol. Regarding security — unlike ate a rich and diverse ecosystem for of traditional data center server a traditional cloud data center that IoT applications similar to the one machines. To incorporate relatively relies on physical security to ensure that exists for IaaS clouds. limited edge device compute nodes, that servers and hypervisors can be we opt instead to integrate light- trusted, FocusStack must assume References weight Docker containers into the that edge nodes can be compromised 1. R. Hall et al, “Scaling up a Geographic OpenStack platform. This enables the at any time, because they’re under Addressing System,” Proc. 2013 IEEE portability, security, and application physical control by end users. This Military Comm. Conf., 2013; doi:10.1109/ isolation of Docker containers while assumption necessitates a revisiting MILCOM.2013.34. still sharing the rich set of orchestra- of trust models in the cloud control 2. R. Hall, “A Geocast-Based Algorithm for tion and management tools available plane, which today might lead to the a Field Common Operating Picture,” Proc. in OpenStack with other typical data entire cloud infrastructure being 2012 IEEE Military Comm. Conf, 2012; center applications. compromised because of a single doi:10.1109/MILCOM.2012.6415848. Edge compute nodes require sev- compute node compromise.4 Future 3. B. Amento et al., “FocusStack: Orchestrat- eral components to interact with our work will address this issue by limiting Edge Clouds Using Location-Based architecture. Nodes run a custom ing the scope of operations that can Focus of Attention,” Proc. 2016 IEEE/ACM version of Nova Compute that inter- be initiated from an edge node. Symp. Edge Computing, 2016, pp. 179–191.

60 www.computer.org/internet/ IEEE INTERNET COMPUTING 38 Computing Edge August 2017 Beyond Wires FocusStack: Orchestrating Edge Clouds Using Focus of Attention

• The SAMonitor component is the acts with a local Docker instance to Regarding mixed-initiative con- 4. W.K. Sze, A. Srivastava, and R. Sekar, formance engineering. Hall has a PhD in tion, and networking. Joshi has a PhD in heart of our situational aware- launch and manage containers. Con- trol — the presence of stakeholders “Hardening Openstack Cloud Platforms electrical engineering and computer sci- computer science from the University of ness technique. When attention is tainers running on the edge nodes like the device’s owner or opera- against Compute Node Compromises,” ence from the Massachusetts Institute of Illinois at Urbana-Champaign. Contact focused on an area, here mean- are provided full OpenStack ser- tor, in addition to traditional cloud Proc. 11th ACM on Asia Conf. Computer Technology. He is a Fellow of Automated him at [email protected]. ing a circle de ned by center vices, including access to con gu- operators and application owners, and Comm. Security, 2016, pp. 341–352. Software Engineering and member of latitude/longitude and radius in rable virtual networks. These virtual requires changes to management the Steering Committee of the IEEE/ACM K. Hal Purdy is a lead inventive scientist at meters, an SAMonitor component networks are implemented using and access control. We’ve taken rst Brian Amento is a principal inventive scientist International Conferences on Automated AT&T Labs Research. His current inter- is created that periodically sends OpenStack’s standard LinuxBridge steps in addressing the problem of at AT&T Labs Research. His research inter- Software Engineering. He serves as Edi- ests include cloud computing, connected the awareness query determined neutron plugin. Connectivity across multiple stakeholders managing the ests include novel interaction techniques, tor in Chief of Automated Software Engi- car technology, and general-purpose GPU by the application, and collects the LTE cellular network between same device by our focus of atten- the Internet of Things, general-purpose neering, an international journal, and is computing. Purdy has an MS in computer an operating picture consisting of the edge nodes and layer 3 network tion concept. A FocusStack control GPU computing, and edge devices. an ACM Distinguished Scientist. Contact science from Rutgers University. Contact the timestamped query responses nodes in the cloud occurs over IP plane, F, by rebuilding its aware- Amento has a PhD in computer science him at [email protected]. him at [email protected]. sent back by all devices report- Security (IPsec) tunnels. ness operating picture each time from Virginia Polytechnic and State Uni- ing from the monitored area. attention is focused on an area, can versity with a specialty in human-com- Kaustubh Joshi is a lead inventive scientist This periodic querying is main- make accurate management deci- puter interaction. Contact him at brian@ at AT&T Labs Research, where he leads tained throughout the time that e proposed to treat huge num- sions even when other stakeholders research.att.com. research efforts on new cloud manage- attention is focused, so that new W bers of real-world IoT devices might have carried out their own ment and network dataplane technolo- devices entering the area during as members of a cloud, so that the management operations on devices Robert J. Hall is a principal investigator at gies needed to pave the way for AT&T’s this time are added to the oper- rich set of tools and applications in the area while those devices were AT&T Labs Research. His work focuses network function virtualization (NFV) Read your subscriptions ating picture and become avail- developed for IaaS cloud comput- out of F’s focus. However, in addi- in the areas of automated software vision of virtualizing the majority of its This article originally appeared in through the myCS publi- able to the application or service. ing can be brought to bear for the tion, the device must also be pro- engineering, requirements engineering, network by 2020. His expertise is in the IEEE Internet Computing, vol. 21, cations portal at http:// The awareness query is program- IoT. We postulate that control plane tected from cloud-initiated actions. modeling and simulation, scalable wire- areas of adaptable and dependable dis- no. 1, 2017. mycs.computer.org. mable and is tailored to the needs scalability and communication For example, when the application less network protocols, and cloud per- tributed systems, the cloud, virtualiza- of the application or service using complexity are major challenges in owner asks the cloud to deploy an a particular SAMonitor instance. doing so. In response, we proposed a application on an edge device, the Example query terms include solution based on dynamic focus of edge node owner’s preferences must location, velocity, computational attention, implemented within our be accounted for. In the future, we state, and battery level. location-based situational aware- intend to develop a policy frame- ADVERTISERADVERTISER INFORMATION INFORMATION • AUGUST 2017 ness technique. Our initial studies work that allows a device owner Now that we’ve detailed the LSA sub- show this can dramatically reduce to express conditions under which system, let’s look at the OSE subsystem. message traf c and control plane applications are authorized to exe- Advertising Personnel Southwest, California: overhead by more than four orders cute orchestration actions on the Mike Hughes OpenStack Extensions of magnitude.3 device. Marian Anderson: Sr. Advertising Coordinator Email: [email protected] Advertising Personnel Southwest, California: (OSE) Subsystem However, challenges remain that With the resolution of these chal- Email: [email protected] Phone: +1 805 529 6790 Phone: +1 714 816 2139 | Fax: +1 714 821 4010 Mike Hughes In a standard OpenStack environ- must be tackled in the future; namely, lenges, we believe that an IoT edge Debbie Sims: Advertising Coordinator Email:Southeast: [email protected] ment, VMs are deployed and man- security and mixed-initiative con- cloud can provide the means to cre- Email:Sandy [email protected] Brown: Sr. Business Development Mgr. Phone:Heather +1 Buonadies 805 529 6790 aged on compute nodes comprised trol. Regarding security — unlike ate a rich and diverse ecosystem for Phone:Email +1 [email protected] 714 816 2138 | Fax: +1 714 821 4010 Email: [email protected] of traditional data center server a traditional cloud data center that IoT applications similar to the one Phone: +1 714 816 2144 | Fax: +1 714 821 4010 Phone: +1 973 304 4123 Fax: +1 973 585 7071 machines. To incorporate relatively relies on physical security to ensure that exists for IaaS clouds. Advertising Sales Representative (Classi eds & Jobs Board) Advertising Sales Representatives (display) limited edge device compute nodes, that servers and hypervisors can be Advertising Sales Representatives (display) Advertising Sales Representatives (Classified Line) References we opt instead to integrate light- trusted, FocusStack must assume Central, Northwest, Far East: Heather Buonadies Central, Northwest, Southeast, Far East: Email: [email protected] weight Docker containers into the that edge nodes can be compromised 1. R. Hall et al, “Scaling up a Geographic Eric Kincaid Heather Buonadies OpenStack platform. This enables the at any time, because they’re under Addressing System,” Proc. 2013 IEEE EricEmail: Kincaid [email protected] Phone:Email: [email protected] 201 887 1703 portability, security, and application physical control by end users. This Military Comm. Conf., 2013; doi:10.1109/ Email:Phone: [email protected] +1 214 673 3742 Phone: +1 973 304 4123 isolation of Docker containers while assumption necessitates a revisiting MILCOM.2013.34. Phone:Fax: +1+1 888214 886673 85993742 Fax: +1 973 585 7071 Fax: +1 888 886 8599 still sharing the rich set of orchestra- of trust models in the cloud control 2. R. Hall, “A Geocast-Based Algorithm for Advertising Sales Representative (Jobs Board) Northeast, Midwest, Europe, Middle East: Advertising Sales Representatives (Jobs Board) tion and management tools available plane, which today might lead to the a Field Common Operating Picture,” Proc. Northeast,Ann & David Midwest, Schissler Europe, Middle East: Email: [email protected], [email protected] Marie Thompson in OpenStack with other typical data entire cloud infrastructure being 2012 IEEE Military Comm. Conf, 2012; David Schissler Phone: +1 508 394 4026 Email:Heather [email protected] Buonadies center applications. compromised because of a single doi:10.1109/MILCOM.2012.6415848. Email: [email protected] Email: [email protected] 4 Fax: +1 508 394 1707 Phone: 714-813-5094 Edge compute nodes require sev- compute node compromise. Future 3. B. Amento et al., “FocusStack: Orchestrat- Phone: +1 508 394 4026 Phone: +1 973 304 4123 eral components to interact with our work will address this issue by limiting Edge Clouds Using Location-Based Fax: +1 508 394 1707 Fax: +1 973 585 7071 architecture. Nodes run a custom ing the scope of operations that can Focus of Attention,” Proc. 2016 IEEE/ACM version of Nova Compute that inter- be initiated from an edge node. Symp. Edge Computing, 2016, pp. 179–191.

60 www.computer.org/internet/ IEEE INTERNET COMPUTING JANUARY/FEBRUARY 2017 61 www.computer.org/computingedge 39 NEW PREFERRED PLUS MEMBERSHIP

OPTIONS TRAINING & DEVELOPMENT FOR A BETTER FIT. RESEARCH

BASIC

STUDENT

And a better match for your career goals. IEEE Computer Society lets you choose your membership — and the benefits it provides — to fit your specific career needs. With four professional membership categories and one student package, you can select the precise industry resources, offered exclusively through the Computer Society, that will help you achieve your goals.

Learn more at www.computer.org/membership. Achieve your career goals with the fit that’s right for you. Explore your options below.

Training & Preferred Plus Research Basic Student Select your Development $60 $126 $55 $115 $55 $115 $40 $99 $8 membership IEEE Affiliate IEEE Affiliate IEEE Affiliate IEEE Affiliate Does not include Member Member Member Member Member Member Member Member IEEE membership

Computer magazine (12 digital issues)*

ComputingEdge magazine (12 issues)

Members-only discounts on conferences and events

Members-only webinars

Unlimited access to Computing Now, computer.org, and the new mobile-ready myCS

Local chapter membership

Skillsoft’s Skillchoice™ Complete with 67,000+ books, videos, courses, practice exams and mentorship resources Books24x7 on-demand access to 15,000 technical and business resources

Two complimentary Computer Society magazine subscriptions

myComputer mobile app 30 tokens 30 tokens 30 tokens

12 FREE 12 FREE Computer Society Digital Library Member pricing Member pricing Included downloads downloads

3 FREE 3 FREE Training webinars Member pricing Member pricing Member pricing webinars webinars

Priority registration to Computer Society events

Right to vote and hold office

One-time 20% Computer Society online store discount

* Print publications are available for an additional fee. See catalog for details.

www.computer.org/membership CLOUD TIDBITS

With all that said, we do have challenges to consider. The challenges include how we can get data Connecting processed from so many external devices. According Cisco, cloud traf c is likely to rise 3.7-fold by 2020, increasing 3.9 zettabytes (ZB) per year in 2015 (the latest full year for which data is available) to 14.1 ZB Fog and Cloud per year by 2020.2 Moreover, big data-associated Internet of Things devices are a large cause of this growth. In- deed, by 2020, database, analytics and IoT work- Computing loads will account for 22 percent of total business workloads, compared to 20 percent in 2015. The total volume of data generated by IoT will reach 600 ZB per year by 2020, which is 275 times higher than projected traf c going from data centers to I ATTENDED THE INTERNET OF THINGS end users/devices (2.2 ZB); 39 times higher than WORLD 2016 EVENT IN SANTA CLARA, CA- total projected data center traf c (15.3 ZB), accord- LIF., last year and served as the track chairman for, you ing to the same Cisco report. guessed it, cloud and IoT. The feeling I got throughout Thus, we have the perfect storm of the use of the event was one of confusion: IoT seems to be so sys- cloud computing, and the growth of IoT. IoT is about temic, yet is dif cult to de ne. As one presenter put it, processing data that comes from devices in some “It’s like plastic. It’s going to be a part of everything.” way that’s meaningful, and cloud computing is about According to Research Nester, “The Global In- leveraging data from centralized computing and ternet of Things (IoT) market reached USD 598.2 storage. Growth rates of both can easily become un- billion in 2015 and the market is expected to reach manageable. We have some problems to solve. USD 724.2 billion by 2023. Further, the market is projected to register a CAGR of 13.2 percent during Deﬁne the Problem, and the Solution the forecast period 2016-2023 globally.”1 In the context of the Internet of Things, the trouble But you don’t have to tell us. Everything in our with the cloud is that data needs to be sent back from lives from the cars we drive, to the thermostats on the sensors gathering info, such as a Nest thermostat the wall, to our refrigerators, literally, has a mind or a Fitbit wristband, to a database in a remote public of their own these days. So much so, that we have cloud. The time that it takes for the data to be trans- ridiculous examples, everything from connected egg ferred from the device or sensor to the remote public trays to Bluetooth-enabled toilets. cloud, that is the latency, is often too great to meet the requirements of the IoT system. We need to do something different, and we can start by doing IoT applications at the cloud’s edge. This means that we avoid sending all the data from sensors and devices back to the cloud, but instead build data and applications on the edge of the network that can handle most of the data gathering and processing. DAVID S. Recently, I published an article in Computer LINTHICUM about Edge computing3. In that article I de ned an architecture called Responsive Data Architecture, Cloud Technology Partners in which I mentioned that IoT brings this issue of [email protected] moving some computing to the edge again. For example, say there is a machine on a factory oor that

d2tid.indd 18 7/21/17 9:22 AM CLOUD TIDBITS

With all that said, we do have challenges to con- analyzes the quality of an auto part that it makes. up the OpenFog Consortium to promote its view. sider. The challenges include how we can get data If the part is not up to quality, as determined by an Whatever it ends up being called and de ned, the Connecting processed from so many external devices. According optical scanner, then it is automatically rejected. key is to reduce latency for response-critical applica- Cisco, cloud traf c is likely to rise 3.7-fold by 2020, While this keeps a human from looking at the tions by moving the data transfer and processing to increasing 3.9 zettabytes (ZB) per year in 2015 (the part, and thus slowing down the process, it also the edge of the cloud, closer to the IoT device. latest full year for which data is available) to 14.1 ZB takes a great deal of time to transmit the data and I’ve been involved in dozens of systems where Fog and Cloud per year by 2020.2 image back to the centralized database and compute the data and applications were placed near the Moreover, big data-associated Internet of engine, where a determination is made as to the suc- source, yet still working with centralized data and Things devices are a large cause of this growth. In- cess of the manufacturing process, and then com- applications. While it’s a bit tricky, it’s not that hard deed, by 2020, database, analytics and IoT work- municated back to the machine. to do. So, what value does OpenFog bring? Computing loads will account for 22 percent of total business The cloud complicates this process even more. There are a few bene ts that I see, including: workloads, compared to 20 percent in 2015. The We’re focused on centralized computing, thus total volume of data generated by IoT will reach there will be latency. Now, instead of sending the • A standard architecture and enabling technol- 600 ZB per year by 2020, which is 275 times higher data back to the data center on the other side of ogy that allows you to approach edge computing than projected traf c going from data centers to the factory, we send it to a remote cloud server in a simple but consistent way. I ATTENDED THE INTERNET OF THINGS end users/devices (2.2 ZB); 39 times higher than that can be thousands of miles away. WORLD 2016 EVENT IN SANTA CLARA, CA- total projected data center traf c (15.3 ZB), accord- To make things worse, we send it over LIF., last year and served as the track chairman for, you ing to the same Cisco report. the open Internet. However, consid- guessed it, cloud and IoT. The feeling I got throughout Thus, we have the perfect storm of the use of ering the amount of processing that Computing at the edge of the network the event was one of confusion: IoT seems to be so sys- cloud computing, and the growth of IoT. IoT is about needs to occur, the cloud is typically is, of course, nothing new – we’ve been temic, yet is dif cult to de ne. As one presenter put it, processing data that comes from devices in some more ef cient. “It’s like plastic. It’s going to be a part of everything.” way that’s meaningful, and cloud computing is about So what do we do? How do we solve doing it for years to solve the same issue According to Research Nester, “The Global In- leveraging data from centralized computing and the problem? We already know that with other kinds of computing. ternet of Things (IoT) market reached USD 598.2 storage. Growth rates of both can easily become un- computing at the edge pushes most of billion in 2015 and the market is expected to reach manageable. We have some problems to solve. the data processes out to the edge of the USD 724.2 billion by 2023. Further, the market is network, close to the source. Then it’s a projected to register a CAGR of 13.2 percent during Deﬁne the Problem, and the Solution matter of dividing the processing between data and • The ability to provide a good product develop- the forecast period 2016-2023 globally.”1 In the context of the Internet of Things, the trouble processing at the edge, versus data and processing in ment framework that network devices and soft- But you don’t have to tell us. Everything in our with the cloud is that data needs to be sent back from the centralized system, meaning a public cloud such ware builders can follow, as well as inuence. lives from the cars we drive, to the thermostats on the sensors gathering info, such as a Nest thermostat as Amazon Web Services. • The ability to deal with security in a consistent the wall, to our refrigerators, literally, has a mind or a Fitbit wristband, to a database in a remote public The concept is to process the data that needs to way. Last year DDOS attacks took over devices, of their own these days. So much so, that we have cloud. The time that it takes for the data to be trans- quickly return to the device. In this case, the pass/ not computers, and now that everything is smart ridiculous examples, everything from connected egg ferred from the device or sensor to the remote public fail data that indicates the success or failure of the and has an OS, this will be a fact of life going trays to Bluetooth-enabled toilets. cloud, that is the latency, is often too great to meet physical manufacturing of the auto part. However, forward. the requirements of the IoT system. the data should also be centrally stored, and, ulti- We need to do something different, and we can mately, all of the data is sent back to the centralized OpenFog recently published a reference archi- start by doing IoT applications at the cloud’s edge. system, cloud or not, for permanent storage and for tecture that covers pretty much everything from Se- This means that we avoid sending all the data from future processing. curity to Programmability (see Figure). If this looks sensors and devices back to the cloud, but instead The bene t is better performance and ef ciency. like it’s been designed by committee, it’s because it build data and applications on the edge of the net- IoT applications need to react almost instantly to the has. I did not nd it useful. work that can handle most of the data gathering and data generated by a sensor or device, such as stop- Like other open standards, OpenFog gets things processing. ping a train, if sensors have reported problems with done through workgroups and committees. The DAVID S. Recently, I published an article in Computer the track switch a few miles ahead, or shutting down danger here is that OpenFog could suffer from “too LINTHICUM about Edge computing3. In that article I de ned an an industrial machine that is about to overheat and many cooks in the kitchen.” The lack of interest in architecture called Responsive Data Architecture, explode. There are hundreds of use cases where re- many standards came about due to lack of speed. Cloud Technology Partners in which I mentioned that IoT brings this issue of action time is the key value of the IoT system. However, OpenFog does have a good list of mem- [email protected] moving some computing to the edge again. For ex- Of course, we have to give this a name. Cisco ber companies (see www.openfogconsortium.org/ ample, say there is a machine on a factory oor that Systems has tried to brand it fog computing and set what-we-do/).

d2tid.indd 18 7/21/17 9:22 AM d2tid.indd 19 7/21/17 9:22 AM CLOUD TIDBITS This article originally appeared in IEEE Cloud Computing, vol. 4, no. 2, 2017.

Security Scalability Open AutonomyRAS Agility Hierarchy Programmability

FIGURE 1. The OpenFog Reference Architecture is based on eight pillars.

What does this mean? At their core, the value of all of these concepts is With the Internet of Things, the latency issue is that we’re considering alternatives to placing every- more acute and more widespread than it is for other thing in the public cloud. Why? Because the public kinds of computing. That’s why putting IoT at the cloud does not make sense, in some cases. IoT will edge of the cloud is such an important concept. challenge us to think differently, and the use of edge Again, it’s not that hard of a concept to carry out. computing, or fog computing, all combined with Most distributed computing developers are very fa- cloud computing, is the likely path that we will nd miliar with the concept of placing the processing as ourselves upon. close to the source as you can. No matter how speedy the networks get, latency References will always be something that developers and ad- 1. Internet of Things (IoT) Market: Global De- mins will try to manage. While we can certainly toss mand, Growth Analysis & Opportunity Outlook new equipment at the problem, I’ve found that most 2023. Published 1 February 2017, http://www performance issues need to be solved by changing .researchnester.com/reports/internet-of-things the design, and not the infrastructure. This is the -iot-market-global-demand-growth-analysis only way you can truly solve the problem. -opportunity-outlook-2023/216 So, the concept is sound, and OpenFog, and 2. VNI Global Fixed and Mobile Internet Traf c Fog computing, is attempting to formalize it, lead- Forecasts. 2017. http://www.cisco.com/c/en/us/ ing thought and promoting the notion of computing solutions/service-provider/visual-networking at the edge for cloud and non-cloud deployments. If -index-vni the Cisco standard is successful, then OpenFog will 3. David Linthicum, “Responsive Data Archi- have accomplished its objective. tecture for the Internet of Things”, Computer, That said, standards seem to fail, and this stan- vol. 49, no. , pp. 72-75, Oct. 2016, doi:10.1109/ dard could be no exception. The fact of the matter is MC.2016.302 that they fail because so many of the member companies have their own agendas, which may not line up with the agendas of the other members. Thus, DAVID S. LINTHICUM is senior vice president of not much gets done, and the fruit of the standard Cloud Technology Partners. He’s also Gigaom’s re- dies on the vine. Fair warning, OpenFog. search analyst and frequently writes for InfoWorld on If IoT and cloud are in your future (who does not deep technology subjects. His research interests in- have them in their future?), then you need to study this clude complex distributed systems, including cloud issue. This means reading my other article, Responsive computing, data integration, service-oriented archi- Data Architecture (RDA), as well as understanding what tecture, Internet of Things, and big data systems. OpenFog has to offer with an eye on what’s realistic. Contact him at [email protected].

44 Computing Edge August 2017 20 IEEE CLOUD COMPUTING WWW.COMPUTER.ORG/CLOUDCOMPUTING

d2tid.indd 20 7/21/17 9:22 AM TECHNOLOGY Help build the next generation of systems behind Facebook's products. Facebook, Inc. currently has multiple openings in Menlo Park, CA (various levels/types): Production Engineer (PEB0717J) Participate in the design, implementation and ongoing management of major site applications and subsystems. Bachelor’s degree required. Exp. may be required depending on level/type. Production Engineer (PEM0717J) Participate in the design, implementation and ongoing management of major site applications and subsystems. Master’s degree required. Exp. may be required depending on level/type. Data Scientist (9794J) Apply your expertise in quantitative analysis, data mining, and the presentation of data to see beyond the numbers and understand how our users interact with our core products. Community Operations Specialist, Instagram (4680J) Process and analyze data to develop operational strategies for improving community support experience; define clear business problems and prioritize solutions using data-driven analytics. Product Manager (411J) Plan business objectives, develop product strategies and establish responsibilities across product area. Product Manager (8022J) Engage in product design and development of digital products. Partner Solutions Manager (7344J) Lead technical implementation & execution for multiple data partners whose data is stored on the platform. Technical Program Manager (7865J) Lead the development of products to support the Infrastructure Engineering organization, whose responsibilities include the growth, management and 24x7 upkeep of the Facebook website. Application Product Manager (8763J) Develop innovative solutions by re-engineering business processes. Decision Scientist (9960J) Partner with marketing, research and product organizations to design, execute, measure and improve the impact of marketing efforts. eBusiness Program Manager (9795J) Plan business systems architecture automation for customer-facing eBusiness functions, including B2C eCommerce, CRM, and B2B data integration. Position requires occasional domestic and international travel to unanticipated locations. Research Scientist (7764J) Research, design, and develop new optimization algorithms and techniques to improve the efficiency and performance of Facebook’s platforms. Solutions Engineering Manager (6786J) Drive engineering effort, communicate cross-functionality, and be a subject matter expert. Technical Program Manager (1908J) Manage cross-functional Solutions Engineering programs in a matrix organization covering a range of ad tech products across all lines of the business. Engineering Manager (3844J) Drive engineering effort, communicate cross-functionality, and be a subject matter expert. Front End Engineer (9047J) Work with Product Designers to implement the next generation of Company’s products. Application Engineer (9049J) Design and develop Hyperion systems. Enhance Hyperion applications for budget, forecast and long range plan for financial planning and analysis (FP&A). Systems Engineer (8952J) Build test benches and regression tests for network operating system dependencies with focus on hardware-software interactions. Data Engineer (8810J) Build, scale, and administer Facebook’s internal enterprise RDBMS databases Oracle along with enterprise applications such as Oracle E-business suite, Oracle Fusion Middle- ware, Microstrategy, and Tableau. Technical Program Manager (10243J) Coordinate cross-functional infrastructure software engineering programs in a matrix organization covering a range of areas (Network, Content Distribution Network, Security, Performance). Analyst, New Products (8542J) Conduct in-depth investigations leveraging large and complex data sets using advanced statistical methodologies and tools. Technical Program Manager (10861J) Coordinate cross-functional site infrastructure projects in a matrix organization covering a range of areas (data center, office, PoP, datacenter network & backbone network, CDN, hardware systems, capacity management). UX Researcher (10391J) Oversee and design the user experience component to generate actionable insights. Design research studies that address both user behavior and attitudes. Electrical Engineer (9831J) Design, prototype, implement, and validate electrical systems including power systems, sensor systems, microcontroller systems, RF systems, and other sub systems to enable virtual reality headsets. Position requires occasional domestic and international travel.

Openings in Redmond, WA (multiple openings, various levels/types): Optical Scientist (7275J) Development of novel algorithms and simulation methods for design of advanced polarization optics for use in future wearable display systems.

Openings in Seattle, WA (multiple openings, various levels/types): Software Engineer (SWE717-BJ) Create web and/or mobile applications that reach over one billion people & build high volume servers to support our content. Bachelor’s degree required. Exp. may be required depending on level/type. Software Engineer (SWE717-MJ) Create web and/or mobile applications that reach over one billion people & build high-volume servers to support our content, utilizing graduate level knowledge. Master’s degree required. Exp. may be required depending on level/type. Software Engineer (7085J) Create web and/or mobile applications that reach over one billion people, and develop highly scalable tools leveraging machine learning, data regression, and rule based models.

Mail resume to: Facebook, Inc. Attn: SB-GIM, 1 Hacker Way, Menlo Park, CA 94025. Must reference job title & job# shown above, when applying.

How Abundance Changes Software Engineering

Diomidis Spinellis

IN THE END, it seems it was all about age units, petabyte databases, immense numbers. Consider the marvelous be- cloud-based datacenters, millions of havior of a humble honeybee. It ies and software components, and, for some for- navigates; it communicates and exhibits tunate companies, billions of users. social behavior; it perceives shapes, col- This abundance is changing the na- ors, patterns, odors, and movements. ture of software engineering. First, by Achieving these tasks with a computer reducing the cost of failure, abundance has been challenging scientists for de- changes how we developers use comput- cades. Yet, over the past few years we’ve ing technologies. Second, abundance been conquering one tough problem af- changes our role by moving the focus ter another. Technologies such as self- from technology to management. driving cars, automatic translation, speech recognition, and face tagging are When Failure Is an Option entering the mainstream. The rise of processing power is letting Although algorithmic innovation has us adopt different ways to ensure a pro- played an important role, the key en- gram’s correctness. So, compile-time type abler has been raw processing power. checking is giving way to languages with A honeybee’s brain contains about a dynamic type systems, such as JavaScript million neurons and a billion synapses, and Python, even for use in production. while modern CPUs contain a few bil- Such systems are obviously wasteful; lion transistors. Granted, a synapse is a their inability (failure) to verify types up lot more complex than a logic gate, and front means they might pay the cost of silicon gates switch much faster than type checking every time they execute a neurons, but on a rough scale, you could statement. Yet their versatility often justi- argue that computers are now achieving es their price. Similarly, although signif- some sort of parity with tiny biologi- icant progress has been made in formally cal brains. We see a similar abundance ensuring a program’s correctness, nowa- of resources in other areas of comput- days we often prefer to run thousands of ing: gigabit networking, terabyte stor- unit and regression tests every time we

IEEE Software To be the best source of reliable, useful, peer-reviewed information for leading software practitioners— Mission Statement the developers and managers who want to keep up with rapid technology change.

464 IEEE SOFTWAREAugust 2017 | PUBLISHED BY THE IEEE COMPUTERPublished by the SOCIETY IEEE Computer Society 0740-7459/17/$33.002469-7087/17/$33.00 © 2017 © IEEE 2017 IEEE FROM THE EDITOR Editor in Chief: Diomidis Spinellis Athens University of Economics and Business, [email protected]

EDITORIALSUBMIT STAFFTODAY change as little as the name of a sin- Many users can also bring with gle variable. Thus, ample processing them millions of service requests per Lead Editor: Meghan O’Dell, m.odell@ How Abundance computer.org power has reduced production system minute. With such numbers of re- IEEE TRANSACTIONS ON failures to failed test cases. quests, detailed performance char- Content Editor: Dennis Taylor Staff Editors: Lee Garber and The abundance of central mem- acterization and service provisioning RebeccaBIG Torres DATA Changes Software ory capacity has created a similar are often dif cult or unnecessary. In- Publications Coordinator: shift. Many modern systems eschew stead, we accept that servicing some [email protected] the problems of explicit manual requests will be slightly delayed, and Lead Designer: Jennie Zhu-Mai AND SUBMIT Engineering memory management by adopting we use these delays as a signaling Production Editor: Monette Velasco dynamic garbage collection. Their mechanism for dynamic load balanc- Webmaster: Brandi Ortega overhead (the cost of failing to man- ing and the elastic provision of addi- MultimediaFor more Editor: information Erica Hardison Diomidis Spinellis ually manage every allocated byte) tional computing resources. Illustrators:on paper Annie submission, Jiu, Robert Stack, can be as much as the program’s On the software side we now and featuredAlex Torres articles, calls for whole working set and one CPU core have thousands of components just Coverpapers, Artist: Peterand Bollingersubscription devoted to the task. But with mem- a mouse click away. From HTML Director,links Products visit: & Services: IN THE END, it seems it was all about age units, petabyte databases, immense ory capacity measured in gigabytes parsing to QR code scanning and Evan Butter eld numbers. Consider the marvelous be- cloud-based datacenters, millions of and multicore CPUs available even from cryptographic protocols to Seniorwww.computer.org/tbd Manager, Editorial Services: Robin Baldwin havior of a humble honeybee. It ies and software components, and, for some for- on low-end smartphones, this is a full text search, it’s all there. This Manager, Editorial Content: navigates; it communicates and exhibits tunate companies, billions of users. small price to pay for the increased wealth of elements is letting us move Carrie Clark social behavior; it perceives shapes, col- This abundance is changing the na- productivity and reliability that such from generic, elaborately designed Senior Business Development Manager: ors, patterns, odors, and movements. ture of software engineering. First, by systems bring us. frameworks to organic ecosystems Sandra Brown Achieving these tasks with a computer reducing the cost of failure, abundance We often don’t think of our soft- in which our systems can gradually Senior Advertising Coordinators: has been challenging scientists for de- changes how we developers use comput- ware’s users as a resource, but they grow according to our speci c needs. Marian Anderson, [email protected] cades. Yet, over the past few years we’ve ing technologies. Second, abundance are. Globalization, the distribution Debbie Sims, [email protected] been conquering one tough problem af- changes our role by moving the focus of software as a service, and network From Technology ter another. Technologies such as self- from technology to management. effects often endow our organiza- to Management CS PUBLICATIONS BOARD driving cars, automatic translation, tions with millions of users. When The profusion of easily available Greg Byrd (VP for Publications), Alfredo Benso, Irena Bojanova, Robert Dupuis, David S. Ebert, speech recognition, and face tagging are When Failure Is an Option users are plentiful, we can some- software components and systems is Davide Falessi, Vladimir Getov, José Martínez, entering the mainstream. The rise of processing power is letting times do without detailed require- also changing our focus as software Forrest Shull, George K. Thiruvathukal Although algorithmic innovation has us adopt different ways to ensure a pro- ments analysis and instead experi- developers: from specifying, design- CS MAGAZINE OPERATIONS played an important role, the key en- gram’s correctness. So, compile-time type ment with various options through ing, implementing, and maintain- COMMITTEE TBD is financially cosponsored abler has been raw processing power. checking is giving way to languages with A/B testing. We simply divide our ing code to selecting, integrating, George byK. IEEEThiruvathukal Computer (Chair), Society, Gul IEEE Agha, Communications Society, IEEE A honeybee’s brain contains about a dynamic type systems, such as JavaScript users into groups and try different using, managing, and contributing M. BrianComputational Blake, Jim X. Intelligence Chen, Maria Society, Ebling, million neurons and a billion synapses, and Python, even for use in production. versions of the software on them to software components. This entails LievenIEEE Eeckhout, Sensors MiguelCouncil, Encarnação, IEEE Consumer Nathan Ensmenger,Electronics Sumi Helal, Society, San IEEE Murugesan, Signal Yong while modern CPUs contain a few bil- Such systems are obviously wasteful; decide which features to adopt and learning how to nd and choose Processing Society, IEEE Systems, Rui, Ahmad-RezaMan & Cybernetics Sadeghi, Society, Diomidis IEEE Spinellis, lion transistors. Granted, a synapse is a their inability (failure) to verify types up which failed ones to axe. components on the basis of their VS Subrahmanian,Systems Council, Mazin IEEEYousif Vehicular Technology Society lot more complex than a logic gate, and front means they might pay the cost of A lavish user base is also allow- quality and ability to meet our sys- Editorial:TBD All submissionsis technically are cosponsoredsubject to editing by for silicon gates switch much faster than type checking every time they execute a ing us (or forcing us) to reduce an tem’s requirements. We must also clarity, IEEEstyle, Controland space. Systems Unless otherwiseSociety, IEEEstated, bylined neurons, but on a rough scale, you could statement. Yet their versatility often justi- application’s feature set to the lowest become skilled in keeping track of articlesPhotonics and departments, Society, as IEEE well as Engineering product and service descriptions,in Medicine re ect the& Biology author’s Society,or rm’s opinion.IEEE argue that computers are now achieving es their price. Similarly, although signif- common denominator. Our mandate the selected components’ evolution InclusionPower in IEEE & Software Energy does Society, not necessarily and IEEE constitute endorsement by BiometricsIEEE or the IEEE Council Computer Society. some sort of parity with tiny biologi- icant progress has been made in formally is no longer to stuff an application in a way that keeps our systems se- To Submit: Access the IEEE Computer Society’s Web- cal brains. We see a similar abundance ensuring a program’s correctness, nowa- with features to satisfy every one of cure, reliable, and maintainable. based system, ScholarOne, at http://mc.manuscript central.com/sw-cs. Be sure to select the right manuscript of resources in other areas of comput- days we often prefer to run thousands of its (in the past, few) users but to se- And, because ecosystems die when type when submitting. Articles must be original and not ing: gigabit networking, terabyte stor- unit and regression tests every time we lect carefully those features that will we all derive value from them with- exceed 4,700 words including gures and tables, which count for 200 words each. satisfy the majority in our large user out also giving back, we must con- IEEE prohibits discrimination, harassment and bullying: group. This phenomenon is most tribute to the ecosystems we use. For more information, visit www.ieee.org pronounced in minimal but handy This means expanding our attention /web/aboutus/whatis/policies/p9-26.html. IEEE Software To be the best source of reliable, useful, peer-reviewed information for leading software practitioners— apps that run in widespread devices from managing our organization’s Mission Statement the developers and managers who want to keep up with rapid technology change. such as tablets. teams to successfully participating

CONTACT US WELCOME NEW AUTHORS EDITORIAL BOARD MEMBERS For detailed information on submitting articles, access www.computer.org /software/author.htm. Sarah C. Gregory is a senior methodologist in re- LETTERS TO THE EDITOR quirements engineering (RE) at Intel. She develops Send letters to and mentors RE subject matter experts across Intel worldwide; conducts training; and supports individu- Editor, IEEE Software 10662 Los Vaqueros Circle als, teams, and leaders who seek to improve their Los Alamitos, CA 90720 personal, group, or business unit RE practice. She’s [email protected] deeply engaged with the IEEE International Require- Please provide an email address ments Engineering Conference, having served as an or daytime phone number with your letter. industry track reviewer, an industry cochair, and the industry representative to the conference steering committee. Gregory’s academic background includes ON THE WEB graduate degrees in law, information science, and systematic theology, and www.computer.org/software she’s pursuing a doctorate in social theory. Starting with the Sept./Oct. issue SUBSCRIBE of IEEE Software, she’ll be the editor of the Requirements department. Contact www.computer.org/software/subscribe her at [email protected].

SUBSCRIPTION Didar Zowghi is a professor of software engineer- CHANGE OF ADDRESS ing at the University of Technology Sydney (UTS) and [email protected]. an adjunct professor of software engineering at the Please specify IEEE Software. Auckland University of Technology. Her research ad- MEMBERSHIP dresses the issues and challenges of requirements CHANGE OF ADDRESS engineering. Previously she was the director of the [email protected]. Centre for Human Centred Technology Design and the associate dean of research at the UTS Faculty of In- MISSING formation Technology. She has worked in the software industry in the UK and OR is DAMAGED series of in-depth COPIES interviews Australia as a programmer, software engineer, analyst, consultant, and project [email protected] prominent security experts features Gary McGraw as manager. Zowghi received a PhD in software engineering from Macquarie Uni- REPRINTSanchor. IEEE OF Security ARTICLES & Privacy versity. She’s a member of the program committee and the chair of the steer- magazineFor price information publishes or toexcerpts order reprints, of the ing committee of the IEEE International Conference on Requirements Engineer- email [email protected] ing. She is the regional editor of the Requirements Engineering Journal and is or fax20-minute +1 714 821 4010. conversations in article format each issue. on the editorial board of IET Software. She is IEEE Software’s new associate REPRINT PERMISSION editor for Software Requirements. Contact her at [email protected]. To obtain permission to reprint an article, contact the Intellectual Property Rights Of ce at [email protected].

in larger open source and commer- novative way. For example, in the cial communities. 1980s the Lotus Corporation prof- www.computer.org/ Moreover, the abundance of com- ited mightily by managing to cram puting resources is changing our a fully featured and blindingly re- silverbullet goals. Often we’re interested not in sponsive spreadsheet program into a *Also available at iTunes developing software that can run 4.77-MHz IBM PC with 256 Kbytes on constrained resources but in uti- of RAM. Nowadays, instead of lizing the available resources in the struggling to shoehorn applications most productive, pro table, and in- into constrained hardware, Wall

486 IEEE SOFTWAREComputing | Edge WWW.COMPUTER.ORG/SOFTWARE | @IEEESOFTWARE August 2017 FROM THE EDITOR FROM THE EDITOR

This article originally appeared in CONTACT IEEE Software, vol. 34, no. 3, 2017. US Street’s technology darlings are com- to do from now on is to sail on the chitecture, and operations, plus the WELCOME NEW ing up with ways to use the wide- tailwinds of abundance. However, design of the neural network itself, AUTHORS EDITORIAL BOARD MEMBERS spread broadband connectivity and this isn’t the whole story. To take it for which advances come from select For detailed information on submitting vast cloud-based infrastructures to to an extreme, it’s like arguing that groups around the world. The same articles, access www.computer.org offer undreamed-of services. Often, the moon landings were a matter of applies, for example, to testing. We /software/author.htm. Sarah C. Gregory is a senior methodologist in re- our dif cult task is no longer to de- having high-energy combustion fuel. now have the power to test thou- LETTERS TO THE EDITOR quirements engineering (RE) at Intel. She develops sign algorithms, data structures, and Yes, the feats we’ve seen in the past sands of components and users, but Send letters to and mentors RE subject matter experts across Intel schemata but to manage immense couple of years have been the result if we don’t know what we’re doing, worldwide; conducts training; and supports individu- datacenters and data stores. of increasing raw computing power. all tests are worthless. Editor, IEEE Software 10662 Los Vaqueros Circle als, teams, and leaders who seek to improve their Finally, the shift to cloud com- However, that power isn’t enough. So, in the end, was it all about Los Alamitos, CA 90720 personal, group, or business unit RE practice. She’s puting and the provision of services And it doesn’t come alone, nor will it numbers? Well, yes, but to get to the [email protected] deeply engaged with the IEEE International Require- on a global scale is shifting our re- increase forever. numbers we also need a lot of good Please provide an email address ments Engineering Conference, having served as an sponsibility from working on soft- To realize the systems I’ve de- old-fashioned engineering, algorith- or daytime phone number with your letter. industry track reviewer, an industry cochair, and the industry representative to ware with clearly de ned boundar- scribed, in both software and hard- mic thinking, and sweat. the conference steering committee. Gregory’s academic background includes ies to managing planet-wide system ware, scores of very intelligent ON THE WEB graduate degrees in law, information science, and systematic theology, and deployments. Our objective isn’t so people have devoted their lives to www.computer.org/software she’s pursuing a doctorate in social theory. Starting with the Sept./Oct. issue much to deliver quality software but delivering ingenious designs and im- Correction SUBSCRIBE of IEEE Software, she’ll be the editor of the Requirements department. Contact to offer a correspondingly reliable, plementations. Perhaps someday the In “App Store 2.0: From Crowdsourced her at [email protected]. secure, ef cient, and maintainable singularity will arrive and machines www.computer.org/software/subscribe Information to Actionable Feedback in software-based service. will design machines. Until then, Mobile Ecosystems” (Mar./Apr. 2017, SUBSCRIPTION Didar Zowghi is a professor of software engineer- the enormous processing power we CHANGE OF ADDRESS pp. 81–89), in the fth line of the sec- ing at the University of Technology Sydney (UTS) and use requires similar brawn to cre- ond column on p. 83, “see sia” should [email protected]. an adjunct professor of software engineering at the n the basis of what I’ve ar- ate it. A neural network might per- Please specify IEEE Software. be “see Figure 2a.” IEEE Software Auckland University of Technology. Her research ad- gued here, someone might form marvels, but that requires raw regrets the error. MEMBERSHIP dresses the issues and challenges of requirements think that all we need processing power, infrastructure ar- CHANGE OF ADDRESS O engineering. Previously she was the director of the [email protected]. Centre for Human Centred Technology Design and the associate dean of research at the UTS Faculty of In- MISSING OR DAMAGED COPIES formation Technology. She has worked in the software industry in the UK and Australia as a programmer, software engineer, analyst, consultant, and project stay connected. [email protected]. manager. Zowghi received a PhD in software engineering from Macquarie Uni- Keep up with the latest REPRINTS OF ARTICLES versity. She’s a member of the program committee and the chair of the steer- IEEE Computer Society For price information or to order reprints, ing committee of the IEEE International Conference on Requirements Engineer- email [email protected] ing. She is the regional editor of the Requirements Engineering Journal and is publications and activities or fax +1 714 821 4010. on the editorial board of IET Software. She is IEEE Software’s new associate wherever you are. REPRINT PERMISSION editor for Software Requirements. Contact her at [email protected]. To obtain permission to reprint an article, contact the Intellectual Property Rights Of ce at [email protected]. in larger open source and commer- novative way. For example, in the IEEE Pervasive Computing explores the many facets of pervasive and ubiquitous cial communities. 1980s the Lotus Corporation prof- computing with research articles, case studies, product reviews, conference reports, Moreover, the abundance of com- ited mightily by managing to cram departments covering wearable and mobile technologies, and much more. puting resources is changing our a fully featured and blindingly re- goals. Often we’re interested not in sponsive spreadsheet program into a Keep abreast of rapid technology change by subscribing today! | @ComputerSociety | facebook.com/IEEEComputerSociety developing software that can run 4.77-MHz IBM PC with 256 Kbytes | @ComputingNow | facebook.com/ComputingNow on constrained resources but in uti- of RAM. Nowadays, instead of | IEEE Computer Society | youtube.com/ieeecomputersociety lizing the available resources in the struggling to shoehorn applications www.computer.org/pervasive| Computing Now most productive, pro table, and in- into constrained hardware, Wall

6 IEEE SOFTWARE | WWW.COMPUTER.ORG/SOFTWARE | @IEEESOFTWARE www.computer.org/computingedge MAY/JUNE 2017 | IEEE SOFTWARE 7 49 EIC’s Message

Multimedia Research: What Is the Right Approach?

Alan Hanjalic ur multimedia research community has sibly critical aspects, unanswered. Now that Associate EIC O become increasingly open to and proac- multimedia technology has reached the level of Delft University of tive in addressing the needs and concerns of large-scale, real-world deployment, our users Technology people living in a world dominated by big (mul- will increasingly be confronted with these timedia) data. Initiatives coming from recent unanswered aspects. multimedia conferences—such as the call for As indicated by Moshe Vardi in his Editor’s “novel topics” at ACM Multimedia 2017 (www. Letter, “Technology for the Most Effective Use acmmm.org/2017/program/novel-topics) or of Mankind,” appearing in the January 2017 the focus on societal impact in the “Brave New issue of Communications of the ACM, IT is Ideas” session at ACM Multimedia 2016— “changing the world, but not always for the clearly show a focus shift. We’ve moved from better.” He states that “deploying technology user-agnostic problems (such as semantic image without understanding its societal context may interpretation) to user-centric problems (such have adverse societal consequences.” He also as investigating whether an image is relevant, gives the example of the “frictionless sharing” interesting, or useful). technology that eventually gave rise to the The quality of the solutions we offer for these “fake-news phenomenon.” Speciﬁcally con- problems largely depends on the approach we cerning multimedia technology—such as auto- choose, including the method, algorithm, and matic algorithmic solutions for multimedia dataset. The question I would like to raise, based content indexing, recommendation, and distri- on my observations of the recent developments bution—the implications of algorithms failing in our community, is are we as open and proac- in a real-world setting might also be signiﬁcant. tive when it comes to discussing whether the Consider, for example, an algorithm that auto- approaches we choose to address these problems matically assigns wrong or offensive labels to are the right ones? What are our guiding princi- images. ples in this choice? Do we have a philosophy However, instead of taking these implica- underlying these principles, and, if so, what is tions more and more seriously, quite an op- this philosophy based on? How does it help us posite development can be observed in our optimize the mapping between the problems community. We’re witnessing an enormous we try to solve and the solutions we offer? What technology push (through evaluation bench- are the long-term implications of deploying our marks and industry), an increased sensitivity solutions in the society? to hypes, growing social pressure (through peer reviews), and a lack of constructive doubt Understanding Broader Implications when adopting new ideas and algorithms. The questions just given should inform the Take, for example, the recent hype around design of methods and algorithms and help us deep learning, which has started to dominate understand their impact on our users in a broad sessions in multimedia conferences and is and long-term context. Problems related to increasingly perceived as a universal approach dealing with and getting the most from big to solving all problems. (multimedia) data are becoming more complex In addition, we tend to select approaches that and thus require complex solutions. A simplis- give us solutions that perform optimally in terms tic approach runs the risk of addressing only of some popular evaluation criteria and metrics, some aspects of the problem, leaving other, pos- without much discussion about whether the

50 August 2017 Published by the IEEE Computer Society 2469-7087/17/$33.00 © 2017 IEEE 4 1070-986X/17/$33.00 c 2017 IEEE Published by the IEEE Computer Society New Editorial Board Member EIC’s Message Tao Mei is a senior researcher PhD from the University of Science and Technol- with Microsoft Research Asia, ogy of China. He is a Fellow of IAPR, a Distin- Beijing, China. His research guished Scientist of ACM, and a senior member interests include multimedia of IEEE. He will be taking over for Rong Yan as content analysis and com- editor of the Startups department. Contact him Multimedia Research: What Is the puter vision. Mei received his at [email protected]. Right Approach? solutions reflect the requested user require- ing a popular approach and following common ments. Here, the statistical improvement—for evaluation criteria, we cannot really say that we example, in terms of the Mean Average Precision did our best to fit our solution to the problem Alan Hanjalic ur multimedia research community has sibly critical aspects, unanswered. Now that (MAP)—over the best performing baseline is typ- we address. Instead, it seems that we keep fit- Associate EIC O become increasingly open to and proac- multimedia technology has reached the level of ically the key. If we manage to achieve sufficient ting the problem to our proposed solution. In Delft University of tive in addressing the needs and concerns of large-scale, real-world deployment, our users improvement on one portion of our data that other words, it seems that we keep interpreting Technology people living in a world dominated by big (mul- will increasingly be confronted with these pushes the average performance of the entire and defining the terms, including “relevance,” timedia) data. Initiatives coming from recent unanswered aspects. dataset above the state of the art, we can easily “interestingness,” or “usefulness” as we go, multimedia conferences—such as the call for As indicated by Moshe Vardi in his Editor’s publish our approach, even if we (and our depending on the results of our algorithms or “novel topics” at ACM Multimedia 2017 (www. Letter, “Technology for the Most Effective Use reviewers) know that our performance on the how our dataset is created and annotated. We acmmm.org/2017/program/novel-topics) or of Mankind,” appearing in the January 2017 rest of the data points might show significant also tend to assume that the statistical improve- the focus on societal impact in the “Brave New issue of Communications of the ACM, IT is deficiencies relative to our competitors. ment of our method over the competitors Ideas” session at ACM Multimedia 2016— “changing the world, but not always for the directly translates into improvement from the clearly show a focus shift. We’ve moved from better.” He states that “deploying technology perspective of every user we wish to serve. As dis- user-agnostic problems (such as semantic image without understanding its societal context may Targeting Individual Users cussed, this is not necessarily the case. interpretation) to user-centric problems (such have adverse societal consequences.” He also Why is the focus on the statistical performance as investigating whether an image is relevant, gives the example of the “frictionless sharing” improvement problematic? The real-world interesting, or useful). technology that eventually gave rise to the problems in the multimedia field have to do hese observations are illustrative of the The quality of the solutions we offer for these “fake-news phenomenon.” Specifically con- with an individual real-world user who is not T need to increase awareness of—and inten- problems largely depends on the approach we cerning multimedia technology—such as auto- interested in any hyped algorithm or its statisti- sify community discussions about—the suit- choose, including the method, algorithm, and matic algorithmic solutions for multimedia cal performance over many users and data ability of the criteria, metrics, and design dataset. The question I would like to raise, based content indexing, recommendation, and distri- points. An individual user is solely interested in principles underlying the approaches we pro- on my observations of the recent developments bution—the implications of algorithms failing the ability of the system to help her with her pose. To initiate this, I suggest the following list in our community, is are we as open and proac- in a real-world setting might also be significant. concrete problems, help her develop her knowl- of necessary actions to guide the selection and tive when it comes to discussing whether the Consider, for example, an algorithm that auto- edge and skills over a long time period, or make design of our research approaches: approaches we choose to address these problems matically assigns wrong or offensive labels to her life easier and more pleasant in general. I Start by understanding the problem you are the right ones? What are our guiding princi- images. would therefore argue that in our multimedia are trying to solve: Who are the users, what ples in this choice? Do we have a philosophy However, instead of taking these implica- field, a system that can help each target user at are their needs, and what is the individual underlying these principles, and, if so, what is tions more and more seriously, quite an op- least a little bit more than the state of the art is or social use context for which the solution this philosophy based on? How does it help us posite development can be observed in our more valuable than a system that helps some must be optimized? optimize the mapping between the problems community. We’re witnessing an enormous target users greatly but others not at all, even if we try to solve and the solutions we offer? What technology push (through evaluation bench- it means that the average performance increase Be open to all reasonable approaches to are the long-term implications of deploying our marks and industry), an increased sensitivity in terms of MAP is lower than in state-of-the-art solve the problem and give sufficient atten- solutions in the society? to hypes, growing social pressure (through approaches. Consequently, it is critical to focus tion to all related solutions. peer reviews), and a lack of constructive doubt on the failure cases and analyze them from the Understanding Broader Implications when adopting new ideas and algorithms. perspective of an individual user in order to Make sure that the dataset used does not The questions just given should inform the Take, for example, the recent hype around define productive directions for further improv- influence conclusions about the quality of design of methods and algorithms and help us deep learning, which has started to dominate ing our research approach or choosing a new an approach. understand their impact on our users in a broad sessions in multimedia conferences and is one. 2017 April–June and long-term context. Problems related to increasingly perceived as a universal approach Furthermore, coming back to the examples Focus on understanding to what extent dealing with and getting the most from big to solving all problems. of user-centric problems—such as searching for each considered approach addresses all rel- (multimedia) data are becoming more complex In addition, we tend to select approaches that relevant, interesting, or useful images—the evant aspects of the given problem. and thus require complex solutions. A simplis- give us solutions that perform optimally in terms question arises whether the images identified tic approach runs the risk of addressing only of some popular evaluation criteria and metrics, by our algorithms are indeed relevant, interest- Analyze possible negative implications of some aspects of the problem, leaving other, pos- without much discussion about whether the ing, or useful, and for whom. By simply adopt- different approaches and solutions for

www.computer.org/computingedge 51 4 1070-986X/17/$33.00 c 2017 IEEE Published by the IEEE Computer Society 5 users and society—especially implications of the failure cases.

Make sure that the arguments that best “sell” your approach are also valid from the perspective of each individual target user.

I am conﬁdent that following these guidelines will increase the impact of our multimedia research and help it change the world for the better. MM

Alan Hanjalic is an associate editor in chief of IEEE MultiMedia and a professor of computer science and the head of the Multimedia Computing Group at the Delft University of Technology, The Netherlands. Contact him at [email protected].

This article originally appeared in Call for Articles IEEE MultiMedia, vol. 24, no. 2, 2017.

IEEE Software seeks practical, readable articles that will appeal to experts and nonexperts alike. The magazine aims to deliver reliable, useful, leading-edge information to software developers, engineers, and managers to help them stay on top of rapid technology change. Topics include requirements, design, construction, tools, project management, process improvement, maintenance, testing, education and training, quality, standards, and more. Submissions must be original and no more than 4,700 words, including 250 words for each table and gure.

Author guidelines: www.computer.org/software/author Further details: [email protected]

www.computer.org/software Read your subscriptions through the myCS publications portal at http://mycs.computer.org.

52 Computing Edge August 2017 6 COMPUTING CAREERS

Cloud-Computing Careers

or this issue of ComputingEdge, we inter- Campbell: Successful students have worked on viewed Scott Campbell, senior director diff erent types of projects and solutions. Creat- F of technology and an instructor at Miami ing a portfolio of diff erent projects is important, University’s College of Engineering and Com- as is working with professors on tools and proj- puting, about cloud-computing careers. Camp- ects. Working with on-campus clubs to help set up bell was previously director of computing labs systems is also valuable. Internships continue to for the school’s Computer Science and Software be helpful in both building a resume and learning Engineering Department. He authored the article what types of work you like and dislike. “Teaching Cloud Computing” for Computer’s Sep- tember 2016 issue. ComputingEdge: What should applicants keep in mind when applying for cloud-computing jobs? ComputingEdge: Which cloud-computing careers will grow the most in the next several years? Campbell: Applicants should have a strong systems, security, and networking background as Campbell: I believe the entire fi eld of devops will well as scripting experience, which is key to mak- grow in importance as the diff erences between ing cloud computing scalable. Applicants should the roles of programming and administration also be aware that cloud computing is replacing continue to blur. To take full advantage of cloud all the roles in traditional datacenters with console- computing’s fl exibility, it will be necessary to management tools. include cloud management in program and application design. Cloud computing lets us ComputingEdge: How can new hires make the treat hardware as programmable objects, which strongest impression in a new position? necessitates a new way of thinking about solutions as we fi gure out how to “program” hard- Campbell: A manager whom I respect said that ware into our solutions. he hired for attitude first and skills second. Being positive and willing to tackle problems ComputingEdge: What would you tell college are key attributes that will make a good impres- students to give them an advantage over the sion. Later, when given a problem by your boss, competition? try to find one or two solutions and then ask

a colleague if they make sense. Communica- Campbell: My favorite saying as a teacher—and tions is also important. Reread all of your early parent—is “fi gure it out.” As a boss, I give assign- emails and memos after letting them sit for at ments to workers assuming they will spend time least 30 minutes. Then make sure they are clear, working on a solution. Employees must under- complete, and easy to follow. stand that they will receive assignments that aren’t well defi ned and that they will have to solve itera- ComputingEdge: Name one critical mistake that tively. So when given a task, spend time fi guring it young graduates should avoid when starting their out. After a bit of research and thought, circle back careers. with the person who gave you the task to see if you are on the right track and if you heard them cor- Campbell: Ignoring the workplace culture is a criti- rectly. Understand that your job is to fi gure out how cal mistake. Every group has both a culture and set best to add value to a project. of standard practices. Make sure to take time to learn the culture and then adapt to it rather than assuming it will adapt to you. Once you understand the culture omputingEdge’s Lori Cameron inter- and the reasons for the standard practices, you’ll be viewed Campbell for this article. Contact in a position to start making improvements. Cher at [email protected] if you would like to contribute to a future ComputingEdge ComputingEdge: Do you have any advice that article on computing careers. Contact Campbell at could benefi t those just starting out in their careers? [email protected].

TECHNOLOGY Oracle America, Inc. has openings for the following positions (all levels/types) in San Mateo County, including Redwood Shores, CA and San Bruno, CA; Alameda County, including Pleasanton, CA; San Francisco, CA; Santa Clara County, including Santa Clara and San Jose, CA; and other locations in the San Francisco Bay Area. Some positions may allow for telecommuting.

Hardware Developers (HWD717): Evaluate reliability of materials, properties and techniques used in production; plan, design and develop electronic parts, components, integrated circuitry, mechanical systems, equipment and packaging, optical systems and/or DSP systems.

Product Managers (PM717): Participate in all software and/or hardware product development life cycle activities. Move software products through the software product development cycle from design and development to implementation, testing, and/or marketing.

Software Developers (SWD717): Design, develop, troubleshoot and/or test/QA software.

Applications Developers (APD717): Analyze, design, develop, troubleshoot and debug software programs for commercial or end user applications. Write code, complete programming and perform testing and debugging of applications.

Programmer Analysts (PA717): Analyze user requirements to develop, implement, and/or support Oracle’s global infrastructure.

Technical Analysts-Support (TAS717): Deliver solutions to the Oracle customer base while serving as an advocate for customer needs. Offer strategic technical support to assure the highest level of customer satisfaction.

Consultants (TCONS717): Analyze requirements and deliver functional and technical solutions. Implement products and technologies to meet post-sale customer needs. Travel to various unanticipated sites throughout the U.S. required.

Sales Consultants (TSC717): Provide presales technical/functional support to prospective customers. Design, validate and present Oracle’s software solutions to include product concepts and future direction. Travel to various unanticipated sites throughout the U.S. required.

Software Developers (TSWD717): Design, develop, troubleshoot and/or test/QA software. Travel to various unanticipated sites throughout the U.S. required.

Applications Developers (TAPD717): Analyze, design, develop, troubleshoot and debug software programs for commercial or end user applications. Write code, complete programming and perform testing and debugging of applications. Travel to various unanticipated sites throughout the U.S. required.

Submit resume to [email protected]. Must include job#. Oracle supports workforce diversity.

54 Computing Edge August 2017 It’s work that matters. It's what we do at Symantec. Symantec is the world leader in providing solutions to help individuals and enterprises assure the security, availability, and integrity of their information. In essence, we protect the free flow of information in a connected world. As the fourth largest independent software company in the world, Symantec has operations in more than 40 countries with 475 out of Fortune's global 500 companies using our solutions. People look to us to safeguard the integrity of their information, ensuring it is secure and available. Achieving this ambitious goal is only possible through the combined efforts of the innovators and visionaries that Symantec continuously attracts. Symantec draws the very best people with a variety of backgrounds, experiences and perspectives and provides them with a work environment where uniqueness is valued and empowered. The creative people we attract help define the spirit of innovation at Symantec. Symantec is proud to be an equal opportunity employer. We currently have openings for the following positions (various levels/types):

Springfield, OR Software Engineers (SWEOR717) Responsible for analyzing, designing, debugging and/or modifying software; or evaluating, developing, modifying, and coding software programs to support programming needs.

Engineering Managers (EMOR717) Direct and supervise team of engineering (QA and/or development teams). Develop standards for products and/or oversee development and execution of software and/or analysis of test results. Some travel required to various, unanticipated sites throughout the United States.

Operations Research Analysts (1648.2288) Responsible for evaluating, developing and implementing operations processes, procedures, programs and strategies to increase technical and operational efficiencies both within the group and other functions.

Herndon, VA MSS Security Engineers (MSSVA717) Drive resolutn of issues that are not getting reslvd thru norm incidnt & problm mgmt procss.

Business Operations Specialist (BOAVA717) Analyze competitive market strategies thru analysis of rltd prdct, mrkt, or share trends. Some travel may be req’d to var, unanticipated sites in US.

Cambridge, MA Software Engineers (SWEMA717) Resp for analyzing, dsigng, debuggng &/or modifying sftwr; or evaltng, devlpng, modifying, & coding sftwr programs to supprt progrmmng needs.

Software QA Engineers (SQAMA717) Resp for dvlpng, applying & maintaing quality standards for company prdcts. Dvlp & execute sftwr test plans. Analyze & write test standards & procedures.

Draper, UT Engineering Managers (EMUT717) Direct and supervise team of engineering (QA and/or development teams). Develop standards for products and/or oversee development and execution of software and/or analysis of test results.

Columbia, MD Software Engineers (SWEMD717) Resp for analyzing, dsigng, debuggng &/or modifying sftwr; or evaltng, devlpng, modifying, & coding sftwr programs to supprt progrmmng needs.

Submit resume to [email protected] . Must reference position & code listed above. EOE. For additional information about Symantec and other positions visit our website at http://www.symantec.com. PREPARE TO CONNECT

The IEEE Computer Society is launching INTERFACE, a new communication tool to help members engage, collaborate and stay current on CS activities. Use INTERFACE to learn about member accomplishments and find out how your peers are changing the world with technology.

We’re putting our professional section and student branch chapters in the spotlight, sharing their recent activities and giving leaders a window into how chapters around the globe meet member expectations. Plus, INTERFACE will keep you informed on CS activities so you never miss a meeting, career development opportunity or important industry update.

Launching this spring. Watch your email for its debut.

How Abundance Changes Software Engineering &gt; Multimedia Research

How Abundance Changes Software Engineering > Multimedia Research