Xcell Journal Special Issue: Xilinx Customers Shape a Brilliant Future

ISSUE 92, THIRD QUARTER 2015

V SPECIAL ISSUE V

5G Wireless Brings Xilinx Customers Ubiquitous Connectivity The Coming Revolution Shape a Brilliant in Vehicle Technology Machine Learning in the Cloud: Future Deep Neural Networks on FPGAs Power Fingerprinting Cybersecurity Using Zynq SoCs

World’s First Programmable City Arises, Built on Xilinx FPGAs 18 www.xilinx.com/xcell Lifecycle Technology

Design it or Buy it?

Shorten your development cycle with Avnet’s SoC Modules

Quick time-to-market demands are forcing you to rethink how you design, build and deploy your products. Sometimes it’s faster, less costly and lower risk to incorporate an off-the-shelf solution instead of designing from the beginning. Avnet’s system-on module and motherboard solutions for the Xilinx Zynq®-7000 All Programmable SoC can reduce development times by more than four months, allowing you to focus your efforts on adding differentiating features and unique capabilities.

Find out which Zynq SOM is right for you http://zedboard.org/content/design-it-or-buy-it

facebook.com/avnet twitter.com/avnet youtube.com/avnet Integrated Hardware and Software Prototyping Solution

HAPS and ProtoCompiler accelerate software development, HW/SW integration and system validation from individual IP blocks to processor subsystems to complete SoCs.

Integrated ProtoCompiler design automation software speeds prototype bring-up by 3X Enhanced HapsTrak I/O connector technology and high-speed time-domain multiplexing deliver the highest system performance Automated debug captures seconds of trace data for superior debug visibility Scalable architecture supports up to 288 million ASIC gates to match your design size

To learn more visit: www.synopsys.com/HAPS LETTER FROM THE PUBLISHER

Kudos to Customers—and to a Xcell journal New Quarterly: X cell Software Journal PUBLISHER Mike Santarini [email protected] 408-626-5981 elcome to this special issue of Xcell Journal celebrating the ways in which Xilinx cus - EDITOR Jacqueline Damian tomers are enabling a new era of innovation in six key emerging markets: Wvision/video, ADAS/autonomous vehicles, Industrial IoT, 5G, SDN/NFV and cloud ART DIRECTOR Scott Blair computing. Each of these segments is bringing truly radical new products to our society. And as the technologies advance over the next few years, the six sectors will converge into a net - DESIGN/PRODUCTION Teie, Gelwicks & Associates 1-800-493-5551 work of networks that will bring about substantive changes in how we live our lives daily. Vision systems are quickly becoming ubiquitous, having long since evolved beyond their initial ADVERTISING SALES Judy Gelwicks niches in security, digital cameras and mobile devices. Likewise undergoing rapid and remarkable 1-800-493-5551 growth are advanced driver assistance systems (ADAS), which are getting smarter and expanding [email protected] to enable vehicle-to-vehicle communications (V2V) for autonomous driving and vehicle-to-infra -

INTERNATIONAL Melissa Zhang, Asia Pacific structure (V2I) communications that will sync vehicles with smart transportation infrastructure to [email protected] coordinate traffic for an optimal flow through freeways and cities. These smart vision systems, ADAS and infrastructure technologies form the fundamental Christelle Moraga, Europe/ Middle East/Africa building blocks for emerging Industrial Internet of Things (IIoT) markets like smart factories, [email protected] smart grids and smart cities—all of which will require an enormous amount of wired and wire - less network horsepower to function. Cloud computing, 5G wireless and the twin technologies Tomoko Suto, Japan of software-defined networking (SDN) and network function virtualization (NFV) will supply [email protected] much of this horsepower.

REPRINT ORDERS 1-800-493-5551 Converged, these emerging technologies will be much greater than the sum of their individual parts. Their merger will ultimately enable smart cities and smart grids, more productive and more profitable smart factories, and safer travel with autonomous driving. Xilinx ® customers have begun creating remarkable systems in all these market segments with our 28-nanometer All Programmable FPGAs, SoCs and 3D ICs. Still on deck are even more ingenious technologies destined to be built around our 20nm UltraScale™ and 16nm FinFET UltraScale+™ technologies as Xilinx rolls out more of these devices over the course of the next two years. While Xilinx continues to innovate by increasing the sophistication and system functionality of our devices, we are also constantly developing ways to enable more design teams to bring new innovations to existing markets and to pioneer emerging markets. To this end, in the last eight months Xilinx took a bold step forward by releasing three new development environments in our SDx™ line (see cover story, Xcell Journal issue 91 ). The new SDSoC™, SDAccel™ and SDNet™ offerings enable software engineers, system architects and mathematicians (non-HDL, hardware design experts) to program the logic—not just the embedded www.xilinx.com/xcell/ processors—in Xilinx All Programmable FPGAs and SoCs. The result is to dramatically speed up software performance and create highly optimized designs with overall system performance per Xilinx, Inc. 2100 Logic Drive watt that can’t be replicated with any other semiconductor device. San Jose, CA 95124-3400 In fact, I’m proud to announce that the company is expanding the charter of my small and Phone: 408-559-7778 FAX: 408-879-4780 mighty team here at Xilinx to launch a sister publication to Xcell Journal . The new quarterly maga - www.xilinx.com/xcell/ zine, called Xcell Software Journal , will roll out later this summer, focusing on high-level design entry methods for software engineers, systems engineers and anyone else who is interested in using our © 2015 Xilinx, Inc. All rights reserved. XILINX, SDx development environments and high-level tools from Xilinx Alliance Program members. the Xilinx Logo, and other designated brands included herein are trademarks of Xilinx, Inc. All other trade - I hope you will enjoy reading this special issue of Xcell Journal celebrating our customers’ marks are the property of their respective owners. efforts in these exciting new markets. We continue to welcome articles about your own experi -

The articles, information, and other materials included ences with Xilinx devices, and now you will have two venues for publication: Xcell Journal and our in this issue are provided solely for the convenience of new quarterly, Xcell Software Journal. our readers. Xilinx makes no warranties, express, implied, statutory, or otherwise, and accepts no liability with respect to any such articles, information, or other materials or their use, and any use thereof is solely at the risk of the user. Any person or entity using such information in any way releases and waives any claim it might have against Xilinx for any loss, damage, or expense caused thereby. Mike Santarini Publisher Need to Find Bugs in Your FPGA Design Faster?

You can with Synplify Premier…

Debug where you design in the RTL and integrate hardware ﬁxes quickly with incremental synthesis Simulator-like visibility enables viewing signals from an operating FPGA at the target operating speed

To learn more about how Synopsys FPGA design tools accelerate debug, visit: www.synopsys.com/fpga CONTENTS

VIEWPOINTS

Letter from the Publisher Kudos to Customers— and to a New Quarterly: Xcell Software Journal… 4

Special Issue Xilinx Customers Shape a Brilliant Future 8 THIRD QUARTER 2015, ISSUE 92

XCELLENCE BY DESIGN APPLICATION FEATURES

Xcellence in Smart Cities World’s First Programmable City Arises, Built on Xilinx FPGAs… 18 18

Xcellence in 5G Wireless Communications 5G Wireless Brings Ubiquitous Connectivity… 26

Xcellence in Industrial IoT Innovative Platform-Based Design for the Industrial Internet of Things… 32

Xcellence in ADAS/Autonomous Vehicles The Coming Revolution in Vehicle Technology and its BIG Implications… 38

Xcellence in Data Center Cloud Computing Machine Learning in the Cloud: Deep Neural Networks on FPGAs… 46 Xcellence in SDN/NFV 38 All Programmable SDN Switch Speeds Network Function Virtualization… 52

Xcellence in Software-Defined Networking Xilinx FPGAs Serve Performance SDN… 58

Xcellence in Cybersecurity Implementing Power-Fingerprinting Cybersecurity Using Zynq SoCs… 64

XTRA READING Xclamations! Share your wit and wisdom by supplying a caption for our wild and wacky artwork… 70 26

Excellence in Magazine & Journal Writing Excellence in Magazine & Journal Design 2010, 2011 2010, 2011, 2012 COVER STORY Xilinx Customers Shape a Brilliant Future

by Mike Santarini Publisher, Xcell Journal Xilinx, Inc. [email protected]

8 Xcell Journal Third Quarter 2015 COVER STORY

Xilinx customers are leading the way in the development of today’s major emerging market trends. Xilinx is enabling this development with All Programmable technologies that deliver software intelligence and hardware optimization.

Ever since Thomas Edison flipped the switch to power the first electric light, the pace of electronic indus- Etry innovation has never let up. We now enjoy so many remarkable electronic innovations that shape our daily lives that it’s easy to overlook the moment when a true milestone in electronics is being reached. Today we are fast approaching one of those milestones. Six important emerging markets—video/vision, ADAS/ autonomous vehicles, Industrial Internet of Things, 5G wireless, SDN/NFV and cloud computing—will soon merge into an omni-interconnected network of networks that will have a far-reaching impact on the world we live in. This convergence of intelligent systems will enrich our lives with smart products that are manufactured in smart factories and driven to us safely in smart vehicles on the streets of smart cities—all interconnected by smart wired and wireless networks deploying services from the cloud. Xilinx Inc.’s varied and brilliant customer base is leveraging Xilinx® All Programmable devices and software-defined solutions to make these new markets and their convergence a reality. Let’s examine each of these emerging markets and take a look at how they are coming together to enrich our world. Then we’ll take a closer look at how customers are leveraging Xilinx devices and software-defined solutions to create smarter, connected and differentiat- ed systems that in these emerging markets to shape a brilliant future for us all (Figure 1).

IT STARTS WITH VISION Vision systems are everywhere in today’s society. You can find cameras with video capabilities in an ever-growing number of electronic systems, from the cheapest mobile phones to the most advanced surgical robots to military and commercial drones and un- manned spacecraft exploring the universe. In concert,

Third Quarter 2015 Xcell Journal 9 COVER STORY

the supporting communications and units—a combination of cameras and alerting guards, homeowners or police storage infrastructure is quickly shift- thermal, night-vision and radar sensors. of suspicious behavior. ing gears from a focus on moving voice These fusion sensors can, in all weather The mainframe system can also gath- and data to an obsession with fast video conditions, autonomously perform fa- er metadata to be stored, analyzed and transfer. cial and object recognition, identify and cross-referenced at a later date by inte- Just three decades ago, vision/vid- track erratic or suspicious activities and grated security centers. Companies can eo systems were very crude by today’s even identify and track individuals—all use the data gleaned from their surveil- standards. For example, the most so- in near-real time. Each unit in these sur- lance technology for purposes beyond phisticated surveillance and security veillance systems will autonomously security. For example, retailers can systems of the time primarily consist- capture visual or even thermal images, use the metadata to analyze customer ed of a video camera (with poor reso- enhancing them through image-correc- browsing and buying habits to better lution) connected by a coaxial cable tion algorithm computations, and even serve their clientele. They can also li- to a monitor, which may or may not perform localized processing that can cense the metadata they have gathered have been diligently overseen by a se- instantaneously analyze everything in to affiliates and product vendors to im- curity guard or attendant. The camera its field of view. prove product marketing and sales. may or may not have been linked to What’s more, these individual units As discussed in depth in the cover a recording device that had a limited are often networked—by wire or story of Xcell Journal issue 83, this number of hours to record what imag- wirelessly—into a mainframe system, smart vision/video technology is be- es the camera captured. allowing all the points in the surveil- coming pervasive and is being lever- By comparison, today’s most ad- lance system to work in concert to aged in a growing number of applica- vanced surveillance systems are highly continuously track individuals in the tions. One of them, in the automotive intelligent. They are composed of sophis- system’s field of vision while simulta- industry, is advanced driver assistance ticated processing-driven, fusion-sensor neous recording their movements and systems (ADAS), a field that in turn

Figure 1 – Customers are leveraging Xilinx All Programmable solutions to create innovations for the emerging markets of ADAS, Industrial IoT, video/vision, 5G wireless, SDN/VFV networks and cloud computing.

10 Xcell Journal Third Quarter 2015 COVER STORY

Figure 2 – The sophistication of advanced driver assistance systems is rapidly evolving thanks in large part to customers’ use of Xilinx’s Zynq-7000 All Programmable SoC devices to build fusion-sensor ADAS platforms.

is advancing via processing to enable detected an object behind the vehicle. on to take the next bold step and extend autonomous vehicles. Advanced vi- The automotive industry has since en- lessons learned in ADAS technology to sion technology is being further lever- hanced this technology greatly by fusing enable vehicle-to-vehicle (V2V) com- aged in smart factories, smart medical the radar sensor with rear-view cameras munications, vehicle-to-infrastructure equipment, transportation infrastruc- and improving the algorithms to widen (V2I) communications and semi-auton- ture and even in smart cities—all the sensor’s field of view. Now, these omous and ultimately autonomous ve- emerging sectors of the Industrial In- rear-view sensor systems can more hicles, in which drivers will be able to ternet of Things (IIoT) market. accurately track objects in the sensor merely copilot their vehicles. With these array’s field of view and identify poten- technologies in place, there will pre- ADAS’ DRIVE TO tially dangerous conditions. In the very sumably be fewer accidents. Moreover, AUTONOMOUS VEHICLES highest-end vehicles, the sensor systems vehicles can be platooned on highways If you own or have ridden in an automo- are fused and connected to the vehicle’s and traffic can run more efficiently, bile built in the last decade, chances are central control unit to automatically which will cut down on fuel consump- you have already experienced the value brake if the driver is distracted. tion. That, in turn, holds the potential to of ADAS technology. Indeed, perhaps From the humble but effective be- mitigate fossil fuel pollution. some of you wouldn’t be here to read ginnings of the rear-view camera, au- OEMs today are actively building and this article if ADAS hadn’t advanced tomakers now offer ADAS systems even beginning to publicize their prog- so rapidly. The aim of ADAS is to make with full 360-degree views around and ress in autonomous vehicles. Daimler drivers more aware of their surround- even inside vehicles. Figure 2 displays subsidiary Freightliner, for example, ings and thus better, safer drivers. the many types of ADAS systems on has received licensing in the state of The very first of these ADAS tech- an automobile today, and shows how Nevada to operate its self-driving Inspi- nologies was rear-view warning. The advanced processing and specialized ration Super Truck. Meanwhile, Mer- earliest versions used a radar sensor algorithms have enabled a small num- cedes-Benz, Google, Audi and Tesla are connected to an automobile’s central ber of relatively inexpensive sensors among the many companies that are electronic control unit (ECU). When a to perform multiple tasks. actively striving to bring autonomous driver placed the vehicle in reverse, the ADAS systems have proven so suc- vehicles to the mass market. It’s truly a system sounded a warning if the sensor cessful and so reliable that the race is race. And the stakes are high.

Third Quarter 2015 Xcell Journal 11 COVER STORY

The cyber-physical systems of Factory 4.0 will be impressive, bringing varying degrees of artificial intelligence to the already smart systems and enabling the factory equipment to be self-correcting and self-healing, with autonomous operation. A robot in a factory line will be able to detect if it is not running optimally.

The challenges for introducing fully and enabling some truly major, substan- maintenance and preorder parts that autonomous vehicles involve ensuring tive advances in society. they’ll need to replace. In turn, they can the vehicles are aware of their locations In Germany, the manufacturing sec- schedule factory downtime to perform and their surroundings. They must be tor of Industrial IoT is seen as such a multiple repairs at once to increase fac- able, in real time, to act accordingly as critical market that the government tory efficiency and productivity, and ul- road conditions change second by sec- is actively sponsoring IIoT develop- timately to maximize profitability. ond to ensure the safety of those in and ment. In a German government effort But the cyber-physical systems of around the vehicle. How best to do this called Industry 4.0, companies are Factory 4.0 will be far more impressive, given that not all vehicles on the road combining processing, sensor fusion bringing varying degrees of artificial in- will have autonomous-driving capabili- and connectivity to create machine intelligence to the already smart systems ties is a question industry and govern- telligence for cyber-physical systems and enabling the factory equipment to ments are debating. The answers will (CPS) for factories, hospitals and civ- be self-correcting and self-healing with undoubtedly fall to safety standards for ic infrastructure. The result will be the autonomous operation. For example, a smart communications between vehi- enabling of the fourth industrial revo- robot in a factory line will be able to de- cles and more forward-looking commu- lution (Figure 3). German companies tect if it is not running optimally. It will nications between vehicles and civic in- alone expect to spend $44 billion per run self-diagnostics to determine if a frastructure. Advances in the emerging year on the CPS retooling, and coun- part is wearing out, and will even try to realm of the Industrial IoT will help to tries like China, Taiwan and India—all reboot or adjust its motor performance create this infrastructure. known for manufacturing—will need to delay system failure. The information to follow suit to stay competitive. can be networked to the factory’s main- IIOT’S EVOLUTION TO THE CPS designs employ smart architec- frame system to order new parts while FOURTH INDUSTRIAL REVOLUTION tures equipped with fusion sensors sim- other robots work faster to ensure over- The term Internet of Things has received ilar to those used in ADAS. The smart, all factory efficiency remains constant. much hype and sensationalism over the fusion-sensor-based control units in The Industrial IoT market also in- last 20 years—so much so that to many, today’s most advanced factories can cludes smart grids and smart transporta- “IoT” conjures up images of a smart re- quickly spot defects in products as they tion that use the same any-to-any connec- frigerator that notifies you when your whirl along assembly lines and remove tivity concepts of a smart factory but on a milk supply is getting low and the wear- the faulty items. Factories use smart grander scale, extending automation and able device that receives the “low-milk” control systems to create virtual bar- connectedness to the power grid and to notification from your fridge while also riers that spot unsafe conditions for planes, trains, automobiles and shipping. fielding texts, tracking your heart rate workers. Companies have networked Megacorporation General Electric, for ex- and telling time. These are all nice-to- these sensors with the machines in the ample, is adding intelligent and connect- have, convenience technologies. factory to shut down machinery instant- ed systems across the many industries it But to a growing number of people, ly if a worker comes too close to dan- serves, including power grid, transporta- IoT means a great deal more. In the last gerous parts of the equipment. tion, oil and gas, mining and water. In rail couple of years, the industry has divid- Smart sensor systems of today also transportation, for instance, GE is outfit- ed IoT into two segments: consumer monitor the wear of factory motors ting its locomotives with smart technol- IoT for convenience technologies (such and parts. Sensors are networked with ogies to prevent accidents and monitor as nifty wearables and smart refrigera- factory control centers and enterprise systems for wear for more accurate, pre- tors), and Industrial IoT (IIoT), a bur- systems to help companies perform ventative and predictive maintenance. geoning market opportunity addressing and optimally schedule equipment At the same time, GE is also diligently

12 Xcell Journal Third Quarter 2015 COVER STORY

Figure 3 – Industry 4.0 is the evolution from embedded systems to cyber-physical systems that, through advanced processing, enable smart manufacturing, infrastructure and cities. The result will likely be the world’s fourth industrial revolution.

building smart rail infrastructure equip- munications to create a truly connect- INTERCONNECTING EVERYTHING ment that it has networked with its loco- ed, intelligent city. To do so, the Bristol TO EVERYTHING ELSE motives. This allows railway operators Is Open project is heavily leveraging the In response to the need for better, to efficiently run their lines and schedule newest, open, yet secure network topolo- more economical network topologies maintenance accordingly to keep freight gies, enabling companies wishing to cre- that can efficiently and affordably ad- and passengers moving efficiently, again ate solutions for smart cities to connect dress the explosion of data-based ser- maximizing operator profitability. their networks to Bristol Is Open’s master vices required for online commerce On an even grander scale, variations network. Hamburg, Chicago and Tokyo and entertainment as well as the many of these smart-infrastructure technol- are among the many other municipalities emerging IIoT applications, the com- ogies are now being integrated in the worldwide that are actively engaging in munications industry is rallying be- IIoT market segment called smart cities, smart-city development. hind two related network topologies: which is projected to be a $400 billion in- The emerging trends toward soft- software-defined networks and net- dustry globally by 2020. As you will read ware-defined networking (SDN) and work function virtualization. in a contributed article in this issue, the network function virtualization (NFV) in Traditional wired networks have city of Bristol, England, is currently un- wired communications, along with the been based on fairly rigid and pro- dertaking a project that offers a peek into advent of 5G wireless technologies, are prietary hardware with limited pro- the cities of tomorrow. The project is in- seen as key to enabling further growth of grammability and versatility. SDN at- tegrating disparate networks for city san- smart-city and other Industrial IoT mar- tempts to add greater flexibility into itation and maintenance, traffic and grid ket sectors in this mass electronic-sys- network administration by decoupling management, and emergency services tems convergence as network traffic the top-level control plane functions, along with business and personal com- grows exponentially in the coming years. which decide where data will be sent,

Third Quarter 2015 Xcell Journal 13 COVER STORY

from the lower-level data plane func- industry expects that by 2020, wireless SECURITY EVERYWHERE tions such as routers and switches— networks will be connecting more than As systems from all of these emerging the devices that actually forward data 50 billion devices worldwide. Among its smart markets converge and become to the selected destination. A soft- many advantages over 4G, 5G promises massively interconnected and their ware-programmable abstraction layer to increase end-user data rates by 10x to functionality becomes intertwined, between the control and data planes 100x while decreasing download laten- there will be more entry points for allows operators to provision new ap- cy fivefold. Further, these bandwidth nefarious individuals to do a greater plications in software, prioritize and increases will enable more people and amount of harm affecting a greater optimize where data is delivered in the businesses to use cloud-based services amount of infrastructure and greater control plane and deliver that data on and storage. More companies will be number of people. The many compa- existing proprietary hardware (or, with able to create virtual stores reaching nies actively participating in bringing NFV added, via vendor-neutral hard- new customers worldwide, while con- these converging smart technologies ware) that operators can scale with sumers will have the ability to store and to market realize the seriousness of changing service requirements. access data anytime, anywhere. ensuring that all access points in their NFV approaches enable companies In turn, data centers supporting products are secure. A smart nuclear to further optimize data plane function- cloud-based business and storage reactor that can be accessed by a back- The wireless industry expects that by 2020, wireless networks will be connecting more than 50 billion devices worldwide. Among its many advantages over 4G, 5G promises to increase end-user data rates by 10x to 100x while decreasing download latency fivefold.

ality. By virtualizing in software what demands will need to expand mas- door hack of a $100 consumer IoT de- would typically be the job of very ex- sively to accommodate the daunt- vice is a major concern. Thus, security pensive specialized hardware (routers ing amount of traffic facilitated by at all point points in the converging and switches), NFV makes it possible 5G wireless networks and SDN/NFV network will become a top priority, to run the software-derived virtualized wireline topologies. Today’s data even for systems that seemingly didn’t functions on less expensive, more gen- centers are struggling to keep up require security in the past. eral-purpose hardware (personal serv- with demand, while their power con- ers and commercial data centers). NFV sumption is increasing exponentially. XILINX PRIMED TO ENABLE enables network hardware resources Data centers now consume upwards CUSTOMER INNOVATION to expand economically and, with SDN of 3 percent of the world’s electric Over the course of the last 30 years, added, scale on demand and as needed power, while producing 200 million Xilinx’s customers have become the as traffic loads increase and decrease metric tons of CO2. That enormous leaders and key innovators in all of around the world. power consumption costs data cen- these markets. Where Xilinx has played On the wireless communications ters more than $60 billion a year in a growing role in each generation of front, 5G promises to reach new data electricity. With data center traffic the vision/video, ADAS, industrial, and rate heights that will not only enable expected to reach 7.7 zettabytes an- wired and wireless communications faster data downloads and streaming nually by 2017, it’s no wonder that segments, today its customers are plac- video for handset users, but will also data center operators are looking ing Xilinx All Programmable FPGAs, provide bandwidth increases that will for new hardware architectures to SoCs and 3D ICs at the core of the facilitate the convergence of IIoT and increase performance while keeping smarter technologies they are develop- smart-city applications. The wireless power consumption in check. ing in these emerging segments.

14 Xcell Journal Third Quarter 2015 Third Quarter 2015 Xcell Journal 15 COVER STORY

Xilinx for smarter vision/video ing these innovations to consumers have ing secure and safe standards-compliant With a rich history in space exploration, shortened, thanks in large part to the smart platforms with sensor fusion, smart mil-aero and security systems, Xilinx has wide use of Xilinx All Programmable de- motion/motor control and smarter and long served the market with sophisticat- vices. Xilinx devices made their debut in faster enterprise connectivity. These All ed vision and video platforms as well as automotive infotainment systems but are Programmable platforms are the underly- the intellectual property (IP) and meth- now making a definitive mark in ADAS. ing technology for smart wind farms com- odologies to help customers build smart Today, Xilinx’s Zynq-7000 All Pro- posed of many smart wind turbines, each video/vision systems. grammable SoC is fast becoming the de of which can adapt to changing weather Customers are using Xilinx All Pro- facto platform provider for advanced conditions for maximum efficiency. The grammable FPGAs and SoCs in their ADAS systems. Audi, Mercedes-Benz, turbines are connected to control and en- vision platforms for real-time analytics BMW, Ford, Chrysler, Honda, Mazda, terprise systems that monitor wear and to create ADAS systems with high-ve- Nissan, Toyota, Acura and Volkswagen schedule preventative maintenance so as locity object detection/recognition; are among the OEMS using Zynq SoCs to avoid entire-system malfunctions. clinically precise imaging systems that or other Xilinx All Programmable devic- With the greater capacity, function- help surgeons guide robotic instru- es in their ADAS systems. The Zynq SoC ality and processing clout of Ultra- ments with pinpoint accuracy; and serves as a multicamera, multifeature Scale™ and UltraScale+ devices, Xil- UAVs and surveillance systems that driver assist platform, a high-resolution inx’s IIoT customers will be able to have instantaneous friend-vs.-foe rec- video and graphics platform, a vehicle advance these smart platforms even ognition and tracking. networking and connectivity platform further, endowing them with greater With the soon-to-arrive 16-nano- and an image-processing and recogni- intelligence for next-generation cy- meter Zynq® UltraScale+™ MPSoC tion platform. Customers implement ber-physical systems. With the Zynq boasting a total of seven onboard pro- algorithms for their design’s most com- MPSoC’s seven processors, for exam- cessing cores (quad-core ARM® Cor- plex and compute-intensive functions ple, customers will be able to integrate tex®-A53, dual-core Cortex-R5 and a in the logic portion of the Zynq SoC and more sensor and motor/motion con- Mali GPU core), Xilinx customers will use the onboard ARM processing system trol functions into a single device and be able to create even more intelli- for serial processing. achieve real-time response not possible gent and highly integrated video sys- With its seven processors, Xilinx’s with any other ASSP-plus-FPGA con- tems, speeding up ADAS’ push toward new Zynq Ultrascale+ MPSoC is destined figuration. The Zynq MPSoC’s on-chip autonomous vehicles and Industrial to provide even more fuel for innova- processing and logic will enable im- IoT’s drive to Industry 4.0 factories tion as OEMs drive toward semi-auton- proved self-monitoring and diagnostics and smart-city infrastructure. omous and fully autonomous vehicles. functionality. Equipment will employ With 64-bit application processors, re- self-healing algorithms or partial recon- From ADAS to autonomous vehicles al-time processors, a graphics proces- figuration to optimize performance as In the early 2000s, Xilinx added au- sor, on-chip memory and FPGA logic all machine conditions change or demand tomotive-grade variants to its FPGA on the same device, OEMs can create ebbs and flows. What’s more, the Zynq product portfolio. Ever since then, ever-more-sophisticated fusion systems Ultrascale+ MPSoC can work in harmo- automotive customers have given Xil- including V2V communications. What’s ny with Zynq SoC-based systems. inx devices a growing role in their ef- more, IIoT smart infrastructure and In smart-city applications, compa- forts to enrich the driving experience smart cities can leverage these same nies can use Zynq SoC-based smart-sen- through electronics. Zynq MPSoC platforms for V2X. The in- sor systems at the edge of the smart The automotive industry has gone nate programmability ensures the V2V city’s surveillance network to enhance through a remarkable renaissance of and V2I networks will scale as the stan- camera resolution and perform object quality, safety and reliability thanks to dards evolve and as more autonomous detection and real-time threat analysis. electronics. For many decades, automo- vehicles enter the roadways. Then, they can turn to the Zynq Ultra- tive electronics largely consisted of wire Scale+ MPSoC to synchronize the data harnesses connecting lights and radios to Enabling cyber-physical received from each Zynq SoC-based a battery and an alternator. Then, in the systems for IIoT smart sensor and communicate it ac- early 2000s, OEMs began using electronic Customers in the industrial market have cordingly with traffic control or author- control units to replace highly unreliable greatly advanced factory efficiency and ities as threats, odd behavior, accidents mechanical actuators. Every year since safety over the last two decades using Xil- or congestion are detected. then, OEMs have added more advanced inx devices. Today, with Xilinx’s All Pro- Likewise in the factory, in addition electronics to their vehicle lines. What’s grammable FPGAs and SoCs, customers to being at the heart of cyber-physical more, the development cycles for bring- in all the major segments of IIoT are build- systems, the Zynq Ultrascale+ MPSoC

Third Quarter 2015 Xcell Journal 15 COVER STORY

For data centers at the core of cloud computing, Xilinx’s devices en- In SDN/NFV, Xilinx All Programmable able companies to create equipment with maximum programmability and technologies are enabling customers to very high performance per watt that they can rapidly optimize for changing throughput, latency and power build equipment with intrusion detection, requirements from a wide range of applications such as machine learning, load balancing and traffic management. video transcoding, image and speech recognition, big-data analysis, Cloud- Xilinx supports efficient management and RAN and data center interconnect. Xilinx for smart security routing of data flows, a wide range of With so many exciting technologies under development and certain to reach communication protocols and programmable new levels of sophistication, autonomy and intelligence while all being interconnected, security measures will need data plane acceleration on demand. to keep up. With many decades playing in the mil/aero and security sectors, Xilinx can function as the macro controller of devices, Xilinx is enabling custom- provides physical security by means a factory network of Zynq SoC-based ers today to quickly bring to the mar- of anti-tamper technology to protect motor control, motion control and ket 5G and SDN/NFV infrastructure IP and sensitive data implemented fusion factory-line quality and safety equipment with the highest degree on its devices from physical attacks. systems. Companies can leverage the of programmability. Xilinx’s All Pro- Xilinx also provides application secu- seven processors to coordinate real- grammable FPGAs, SoCs and 3D ICs rity via fault-tolerant design, an imple- time response and analysis received are the most flexible platforms for the mentation methodology that ensures from the Zynq SoC control system. At evolving software and hardware re- the design can correct faults from the same time, they can perform meta- quirements of 5G and SDN/NFV. Fur- propagating. Xilinx devices and IP en- data analysis and communicate it with ther, they are the ideal programmable able customers to implement several the enterprise through proprietary net- solution for the performance-per-watt types of fault-tolerance techniques works (in full compliance with safety demands of data center systems at the including real-time system monitor- and reliability standards) and through heart of the cloud computing business, ing, modular redundancy, watchdog emerging high-speed 5G wireless and poised to expand rapidly with 5G and alarms, segregation by safety level SDN/NFV wired networks. SDN/NFV networking. or classification, and isolation of test In SDN/NFV, Xilinx All Programma- logic for safe removal. Xilinx for 5G, SDN/NFV ble technologies are enabling custom- and cloud computing ers to build equipment with intrusion MORE BRILLIANT MINDS, Xilinx’s devices have played a signifi- detection, load balancing and traffic MORE INNOVATIONS cant role in every buildout of the wire- management. Xilinx supports efficient In a move that will enable all of these less and wired networking infrastruc- management and routing of data flows, impending innovations in all of these ture since the 1980s. With every cycle of a wide range of communication proto- many markets to come to fruition more Moore’s Law, Xilinx devices have grown cols and programmable data plane ac- rapidly, Xilinx recently introduced its in capacity and functionality to the celeration on demand. SDx™ development environments to point where today’s All Programmable In 5G, customers are leveraging ease the programming job. The new devices enable design teams to innovate Xilinx All Programmable devices to products will bring the performance new networking systems with the high- create distributed small cells, mas- and programmability advantages of est level of system programmability and sive-MIMO systems with hundreds of Xilinx devices to a far wider user base differentiation ever seen. antennas and platforms that perform than ever before. By providing design With Xilinx’s 7 series, 20nm Ultra- centralized baseband processing via entry via high-level languages, the SDx Scale and upcoming 16nm UltraScale+ Cloud-RAN. environments enable software engi-

16 Xcell Journal Third Quarter 2015 Third Quarter 2015 Xcell Journal 17 COVER STORY

neers and system architects to pro- Xilinx FPGAs in C, C++ and OpenCL™ As we are fast approaching the mile- gram Xilinx devices with languages to accelerate the performance of virtu- stone where video/vision, ADAS/auton- they are accustomed to using (see cov- alized network functions (VNFs). omous vehicles, IIoT, 5G wireless, SDN/ er story, Xcell Journal issue 91). Soft- To enable further innovation in vid- NFV and cloud computing converge, we ware engineers outnumber hardware eo/vision, ADAS/autonomous vehicles are certain to see a number of innovations engineers worldwide 10 to 1. and IIoT applications that call for em- that will drastically change the society we To enable further innovation in SDN, bedded processing, Xilinx’s SDSoC™ live in—hopefully for the better. Today, Xilinx’s new SDNet™ software-defined development environment allows soft- we are at the early stages of all these in- environment lets systems engineers ware and system engineers to create novations, and Xilinx is well equipped to build programmable data plane solu- entire systems in C++. They can op- help customers bring their brilliant prod- tions with a high-level language to meet timize system performance by having ucts to market. In the following pages in a network’s unique performance and the environment’s compiler implement this special issue of Xcell Journal, you will latency requirements. To fuel further slower functions in the Zynq SoC’s or get a small sampling of the many exciting innovation in NFV and other network MPSoC’s logic blocks. In this way, ar- innovations Xilinx customers are creating architectures and topologies, develop- chitects and software engineers can in these emerging markets and a peek at ers can use Xilinx’s SDAccel™ environ- create systems with optimum perfor- how they are leveraging Xilinx’s All Pro- ment, which enables system and soft- mance and functionality that simply grammable solutions today to make them ware engineers to program the logic in isn’t achievable in two-chip platforms. a reality for us all…very soon.

Debugging Xilinx's ZynqTM -7000 family with ARM® CoreSightTM

► RTOS support, including Linux kernel and process debugging

► SMP/AMP multicore Cortex®- A9 MPCoreTMs debugging

► Up to 4 GByte realtime trace including PTM/ITM

► Profiling, performance and statistical analysis of ZynqTM's multicore Cortex®-A9 MPCoreTM

Third Quarter 2015 Xcell Journal 17 XCELLENCE IN SMART CITIES

World’s First Programmable City Arises, Built on Xilinx FPGAs

by Bijan R. Rofoee Senior Network Engineer Bristol Is Open [email protected]

Mayur Channegowda Chief Scientist, SDN Zeetta Networks www.zeetta.com

Shuping Peng Research Fellow University of Bristol Chief Scientist, Virtualization Zeetta Networks

George Zervas Professor of High-Performance Networks University of Bristol

Dimitra Simeonidou CTO, Bristol Is Open Professor of High-Performance Networks University of Bristol

18 Xcell Journal Third Quarter 2015 XCELLENCE IN SMART CITIES

people to cities will grow the number of Bristol, England, has become a testbed urban residents by 60 million every year for smart-city technologies. The Bristol during that decade. [2] The result is that more than 70 percent of the world’s pop- Is Open project is a living experiment in ulation will be living in cities by 2050. Considering also that cities occupy just the evolution of the Internet of Things. 2 percent of the world’s landmass while consuming about three-quarters of its resources, the ongoing urbanization pres- y 2050, the human popu- ment in the smart-city sector includes ents economic and societal challenges lation will have reached around $150 million for research into and a strain on the urban infrastructure. 9 billion people, with 75 smart cities funded by Research Coun- Growing cities will have to deal with a percent of the world’s cils U.K.; $79 million over five years variety of challenges to maintain eco- inhabitants living in cit- earmarked for the new Future Cities nomic advancement, environmental sus- ies. With already around Catapult center being established tainability and social resiliency. B80 percent of the United Kingdom’s by the Technology Strategy Board in The solution is to make cities smart- population living in urban areas, the London; $52 million invested in future er. Although there is no absolute defini- U.K. needs to ensure that cities are fit city demonstrators earlier this year; tion for smart cities, there are a number for purpose in the digital age. Smart cit- and $63 million recently allocated to of key aspects widely recognized [3] for ies can help deliver efficiency, sustain- Internet of Things (IoT) research and a smart city’s operations. They include: ability, a cleaner environment, a higher demonstrator projects. • Citizen-centric service delivery, quality of life and a vibrant economy. Bristol Is Open is leading the way which involves placing the citizen’s To this end, Bristol Is Open (BIO) is to building a city-scale research and needs at the forefront. a joint venture between the University innovation testbed. The aim is to drive • Transparency of outcomes/perfor- of Bristol and Bristol City, with collabo- digital innovation for the smart cities of mance to enable citizens to com- rators from industry, universities, local the future: the open and programmable pare and critique performance, communities, and local and national communities that will be the norm in establishment by establishment governments. Bristol Is Open (www. the latter part of the 21st century. and borough by borough. bristolisopen.com) is propelling this The BIO testbed is equipped with • An intelligent physical infrastruc- municipality of a half million people in leading-edge programmable networking ture, enabling service providers southwest England to a unique status as technologies, enabled by a citywide op- to manage service delivery, data the world’s first programmable city. erating system called NetOS, that allow gathering and data analyzing Bristol will become an open testing smart-city applications to interact with effectively. ground for the burgeoning new market city infrastructure—to program, virtual- • A modern digital, secure and open of the Industrial Internet of Things— ize and tailor network functions for op- software infrastructure, to allow that is, the components of the smart-city timum performance. Xilinx devices as citizens to access the information infrastructure. The Bristol Is Open proj- high-performance generic platforms are they need, when they need it. ect leverages Xilinx® All Programmable utilized at many points in the city from FPGA devices in many areas of develop- the wired, wireless and IoT networking Technological enablers for smart ment and deployment. infrastructure to emulation facilities. cities are inspired by the Internet of Let’s take a tour of this new type of Things, a market that, according to THE VISION OF THE SMART CITY urban community, starting with the Gartner, [4] will grow to 26 billion units A smart city utilizes information and overall vision for programmable cit- installed as of 2020. That total rep- communications networks along with ies. Then we will take a deeper look resents an almost thirtyfold increase Internet technologies to address ur- at how the Bristol project is utilizing from 0.9 billion in 2009, with the rev- ban challenges, with the objective of Xilinx devices to build urban “white enue from technologies and services dramatically improving livability and boxes” and to deliver various net- exceeding $300 billion. Smart cities de- resource sustainability. It is predicted working functions. ploy IoT technologies on a wide scale, [1] that the smart-cities industry will enabling data gathering from sensors value more than $400 billion global- FUTURE SMART CITIES and things present in the ecosystem, ly by 2020, with the U.K. expected to More than 100 cities of 1 million people pushing them for analysis and feeding gain at least a 10 percent share, or $40 will be built in the next 10 years world- back commands to actuators, which billion. The U.K. government invest- wide [2], while the continuous influx of will control city functions.

Third Quarter 2015 Xcell Journal 19 XCELLENCE IN SMART CITIES

From sensing and analysis, infor- tions. These technologies exploit open BRISTOL IS OPEN: mation passes back to actuators in software and hardware platforms, VISION AND ARCHITECTURE the city infrastructure to control op- which users can program to tailor Launched in 2013, Bristol Is Open is a erations dynamically. This arrange- network functions for different use program funded by the local, national ment is an enabler for driverless cars case requirements. Improved con- and European governments and also by using smart transport facilities; great- trol, monitoring and resource alloca- the private sector. BIO is already deliver power efficiency thanks to smart tion in the network are the evident ering R&D initiatives that contribute to lighting; the management of network benefits of deploying programmable the advancement of smart cities and the resources for different times (daily networks. More important, program- Internet of Things. and seasonal changes); the movement mable technologies facilitate the BIO aims to serve as a living lab—an of resources depending on occasions integration of networks with IT fa- R&D testbed targeting city-driven digi- such as sports events, which require cilities, which will result in greater tal innovation. It provides a managed high-quality broadcast and coverage; application awareness. multitenancy platform for the develop- and efficient handling of emergency Software-defined networking (SDN) ment and testing of new solutions for situations (city evacuation). is one of the main enablers for program- information and communication infra- mable networks. The SDN foundation structure, and thus forms the core ICT PROGRAMMABLE CITY is based on decoupling infrastructure enabling platform for the Future Cities VS. SMART CITY control from the data plane, which agenda. At the infrastructure level, BIO Smart cities aim to improve and en- greatly simplifies network management comprises five distinctive SDN-enabled hance public and private service offer- and application development while also infrastructures, as shown in Figure 1: ings to citizens in a more efficient and allowing deployment of generic hard- • Active nodes as optoelectronic-net- cost-effective way by exploiting net- ware in the network for delivering network white boxes using FPGA work, IT and, increasingly, cloud tech- working functions. programmable platforms and het- nologies. To achieve this goal, smart SDN-based scalable and facilitat- erogeneous optical and Layer 2/3 cities rely extensively on data collected ed network management also greatly networking infrastructure from citizens, the environment, vehicles empowers network virtualization. Net- and basically all the “things” present in work virtualization essentially enables • Heterogeneous wireless infrastructure the city. The more data that becomes multiple users to operate over shared comprising Wi-Fi, LTE, LTE-A and 60- available, the more accurately city op- physical resources, isolated from one GHz millimeter-wave technologies erations can be analyzed, which in turn another, reducing the need for install- • IoT sensor mesh infrastructure will lead to the design and availability of ing supplementary physical hardware. smart-city services. Network function virtualization (NFV), • Network emulator comprising a serv- For the network infrastructure, city- a more recent innovation in virtualiza- er farm and an FPGA-SoC-network wide data retrieval and processing mean tion technologies, offers software im- processor farm massive amounts of sensor data that plementation of network functions in • Blue Crystal high-performance com- needs to be collected, aggregated and commodity hardware. Network func- puting (HPC) facility transferred to computational facilities tions such as firewall, deep packet in- (data centers) for storage and possibly spection, load balancing and so on are On the metro network, the infra- processing. The wide diversity of sce- deployed as pluggable software con- structure offers access to dynamic opti- narios and applications presents major tainers in generic machines, expedit- cal switching supporting multi-terabit/ challenges regarding networking and ing network service deployments with second data streams, multirate Layer computing infrastructure requirements great cost-efficiency. 2 switching (1 to 100 GbE) and Layer in smart cities. Legacy information and In addition to software-driven net- 3 routing. The metro is also equipped communications technology (ICT) ur- working, hardware and infrastructure with programmable hardware plat- ban infrastructure can be a major bot- programmability will progress beyond forms and high-performance servers tleneck for smart-city operations, as it fixed-function hardware data planes. to allow open access to the infrastruc- does not offer the capacity, flexibility Adding high-level programmability and ture and a capability to create and and scalability desirable for the emerg- more sophisticated functionality to experiment with new hardware and ing, future-proof, resource-demanding the data plane, accessed via standard software solutions. This wired part of and scalable smart-city applications. software APIs, will make it possible to the infrastructure also connects to the Programmable networking technolo- manage networking resources more in- Blue Crystal HPC facilities at Bristol in gies offer unique capabilities for raising telligently and efficiently, increasing the order to support experimentation with the performance of smart-city opera- rate of innovation. advanced cloud infrastructures.

20 Xcell Journal Third Quarter 2015 XCELLENCE IN SMART CITIES

The access network infrastructure ture communication technologies and also enables continuous investment includes overlapping and seamless cloud networking. into smart infrastructure at the lowest wireless connectivity solutions (mac- layers of the ICT installations by driv- ro and small-cell radio technologies) SOFTWARE-DEFINED ing the reduction of costs for physical using a combination of cellular and NETWORKING FOR CITY components and pushing more of the Wi-Fi technologies enhanced with INFRASTRUCTURES operational aspects into the software. millimeter-wave backhaul and direct The communications sector has seen As SDN is now reaching beyond ICT connections to the optical network. a flowering of innovative solutions in infrastructures into the IoT platforms, it The facility also supports experimen- recent years based on the concept of creates the opportunity to realize a full tation platforms for new 5G-and-be- SDN, bringing advances in IT to the circle of adaptability of computing and yond access technologies such as mil- traditional hardware-driven telecom- communication infrastructures, where limeter-wave-based access solutions munications world. This decoupling sensory and real-world information with beam tracking, as well as new of control and data through SDN en- drives the operation of the network. technology enablers such as massive ables innovative ways of controlling Network infrastructures in turn are uti- MIMO for ultrahigh-density networks a network, while relying on a basic lized to provide the sensor information in the 2-GHz band. data-forwarding operation, common to applications and services in a mean- In addition, BIO provides priority ac- across all networking elements. The ap- ingful and timely manner. At BIO, it is cess to the infrastructure (for example, proach allows the integration of novel our vision for that programmability and lampposts) for the additional installation architecture concepts, such as infor- adaptability across the various layers of of sensor nodes in the area, supported mation-centric networking (ICN), into the overall system to ultimately imple- by suitable data aggregators, computing such a software-driven network. SDN ment the notion of what we call a Living and storage resources. Optionally, these resources can directly interface into the wired and wireless network. BIO has also installed a low-energy wireless-sensor mesh network. This network will support IoT-based research, with initial sensors supporting environmental monitoring (temperature, air quality, pollution levels, lighting, noise and humidity) and smart streetlights. BIO will also provide access, through suitable secure interfaces, to IoT assets already installed elsewhere in the city, including parking sensors, traffic lights, traffic flow sensors, surveillance (safety) cameras and public-vehicle sensors. Small sensors, including the smartphones and GPS devices of willing participants, will supply information about many aspects of city life, including energy, air quality and traffic flows. All the data generated will be rendered anonymous and made public through an “open data” portal. The entire platform uses SDN control principles and, as such, is fully programmable by experimenters and end users. Internationally, the BIO experimental network will be the first of its kind and will generate new and exciting Figure 1 – The Bristol Is Open fiber network places active core nodes at four locations in opportunities to pioneer the develop- the city. HPC facilities and emulation are accessible through the network core. Wireless technologies (802.11ac, 802.11ad, LTE, LTE-A) are spread out through the center. ment of hardware and software for fu-

Third Quarter 2015 Xcell Journal 21 XCELLENCE IN SMART CITIES

Network, where the Internet and things SDN principles, will provide the need- to establish dedicated infrastructures to truly merge into a coherently managed ed programmability and adaptability support specific applications. Therefore, and operated computing and communi- for smart cities. NetOS will be an over- one of the key challenges for the city cation environment. arching and distributed operating sys- infrastructure operators is to offer cus- Demonstrating SDN-based platforms tem spanning from terminals (even the tomized, application-specific network on a citywide scale is crucial. Future more advanced ones, e.g., mobile robots, solutions over a common ICT infrastruc- Internet and 5G technologies are pres- drones) through the network elements to ture. Virtualization, when integrated with ent in the BIO testbed, specifically an the cloud/IT resources. This citywide OS an SDN-enabled control platform, is a SDN-enabled optical-backbone infra- will cope with the heterogeneity of un- key technical enabler for addressing this structure using current and contempo- derlying resources based on a distributed challenge. Virtualization is able to create rary (i.e., Wi-Fi, LTE, millimeter-wave) software architecture. NetOS will act as a multiple coexisting but isolated virtual in- radio access technologies. The stimu- logical entity that is implemented in a hi- frastructures running in parallel, serving lating media and entrepreneurial com- erarchical manner with distributed soft- its tenant’s application requirements. munity is present throughout the BIO ware, making it possible to map varied By thorough analysis of each tenant’s testbed (the engine shed in Figure 1 is services on the infrastructure. requirements in terms of social policy, home to a startup incubator and the wa- security and resources, it’s possible to tershed is home to the media communi- VIRTUALIZATION FOR construct a virtual infrastructure with a ty in Bristol). Members of these commu- CITY INFRASTRUCTURE certain network topology, indicating the nities also serve as an excellent set of A large number of highly diverse city way that virtual nodes are interconnect- early-user groups for the use case work. applications need to be supported on ed with virtual links. Performance pa- Their involvement in BIO allows us to top of the city infrastructures. For ex- rameters (for example, latency) and recapture the insights and requirements ample, some applications will demand source requirements (such as network posed by the municipal communities. high capacity and very low latency. Oth- bandwidth, compute CPU/memory) are The wired, wireless and RF mesh net- ers will consume very little bandwidth specified in the virtual nodes and links. works are technology-agnostic, built on but will need to support a very large Generally, virtual resources (nodes and open-network principles using SDN tech- number of endpoints. Still others will links) are obtained by partitioning or nologies that enable network function have strict requirements on resiliency aggregating physical resources. There- virtualization. A city operating system or security, privacy and so on. fore, a programmable hardware infra- called NetOS (Figure 2), also based on It is neither feasible nor cost-effective structure is essential to support the

OPEN DATA PLATFORM SOFTWARE DEFINED NETWORK CITY OPERATING SYSTEM

DATA PRE-PROCESSING SERVICES srv1 srvN

PLATFORM VIRTUALIZATION/SLICING

INFORMATION MODELS OpenStack, etc.

DATA COLLECTION ENGINE sync async

CITY NETWORK(S) SDN controller

gw/drv gw/drv gw/drv gw/drv gw/drv gw/drv gw/drv gw/drvgw/drv

MULTIPLE SENSORS/PLATFORMS & HARDWARE

Figure 2 – NetoOS is an SDN-based platform, built in a multilayer structure, which can communicate with networking, IT and IoT technologies. This platform natively supports data collection, virtualization, information modeling and interfacing with third-party applications.

22 Xcell Journal Third Quarter 2015 XCELLENCE IN SMART CITIES

Figure 3 – Bristol Is Open’s network white box is built around Xilinx FPGAs.

composition of virtual infrastructures load customized operating systems highly desirable platforms for designing with high granularity and scalability. and enable on-demand redefining of and prototyping network technologies In the city environment, the devices network functions without the re- that must demonstrate high degrees of deployed in the urban infrastructure are strictions of vendor-locked devices. flexibility and programmability. heterogeneous, including wireless/mo- Network processors were the initial We are using Xilinx FPGAs that have bile, wired, optical networks, data centers/ route to hardware programmability evolved into system-on-chip (SoC) de- cloud and functional appliances. In order of the underlying network, leveraging vices in multiple points within the BIO to enable seamless service provisioning, the ease of defining functions through infrastructure: in active nodes (see Fig- it’s mandatory to support converged vir- software APIs. Network processors are ure 2) as optoelectronic white boxes, tual infrastructures enhanced with virtual well-known hardware platforms that emulation facilities, wireless LTE-A network functions across the multitech- provide generic programmable fea- experimental equipment and IoT plat- nology, multidomain city infrastructure, tures similar to general-purpose CPUs forms. BIO uses programmable and so that each tenant can get its own slice of (with extended hardware resources), customizable network white boxes the city infrastructure. However, currently and can be programmed to perform that consist of programmable electrical these technology domains are controlled various networking functions. The (FPGA) and optical (switching, process- and managed in silos. The NetOS with main advantage of processor-based ar- ing, etc.) parts. These boxes—which SDN capabilities at BIO provides a logi- chitectures is rapid implementation of enable high-capacity data processing cally centralized control platform that can networking functions using high-level and transport, function programmability break through the management silos and languages such as C, which is highly and virtualization—are deeply control- bridge the different technology segments. desirable for rapid prototyping. Net- lable through SDN interfaces. Figure 3 The operating system is able to abstract work processors, however, are not op- demonstrates the FPGA-based platform, the heterogeneous city devices, hide their timized for parallel operations, which which can host multiple functions in a complex technical details and expose the are essential for building high-perfor- programmable way, and is interfaced to infrastructure in a uniform way. mance data plane technologies sup- a programmable photonic part. [5] porting high-data-rate transport. FPGAs offer several advantages, in- THE VISION FOR THE WHITE BOX Field-programmable gate arrays cluding hardware repurposing through Open network devices, or network (FPGAs) are high-performance and ge- function reprogrammability, easier up- white boxes, use unbranded, generic, neric processing platforms utilizing pro- gradability and shorter design-to-deploy modular and programmable hardware grammability from transistor-level to IP- cycles than those of application-specific platforms. This type of equipment can based function level. This makes them standard products (ASSPs).

Third Quarter 2015 Xcell Journal 23 XCELLENCE IN SMART CITIES

The photonic part of the network with the BIO city network and other re- white boxes uses an optical backplane mote interconnected laboratories: Unlike any other existing facilities that on which a number of photonic function offer computer host-based emulation en- 1. Node and link emulation: This plat- blocks are plugged into optical functions vironments, BIO uniquely includes pro- form can emulate network elements such as amplification, multicasting, wave- grammable hardware (FPGAs, network such as routers and switches from the length/spectrum selection, signal add/ processors) as well as dynamic and flex- wired and wireless domains, along drop, etc. Critically, the input and output ible connectivity to multitechnology test- with the interconnecting links with links are decoupled from any of the func- beds and a rich, dedicated connectivity various physical attributes. tions that the node can offer, unlocking infrastructure. The use of programmable flexibility, efficiency and scalability, and 2. Protocol emulation: Whether cen- hardware and external interconnectivity minimizing disruptive deployment cycles tralized or distributed, network nodes will allow users to accurately emulate the with on-service hitless repurposing. rely on the protocols to communi- functionality and performance of network cate. The emulation facility with pre- and computing technologies in scale and ZYNQ SOC-BASED cise modeling of the network technol- use them to synthesize representative EMULATION PLATFORM ogies allows the user/researcher to complex systems. Exploiting the FPGA’s To broaden the capabilities of BIO facil- try out communication protocols and parallel-processing capabilities and high- ities in experimenting with larger and study their behavior on scale. speed I/Os, BIO is equipped to emulate more-realistic scenarios, we have de- current or experimental network tech- 3. Traffic emulation: Depending on nologies and topologies, be they wired or ployed a network emulator facility with- the emulation scenario (wireless net- wireless, precisely and at scale. in BIO. This platform enables network works, data center networks, etc.), The network emulator uses a vast emulation as well as resource virtualiza- traffic patterns with arbitrary inter- amount of advanced networking and tion and virtual-infrastructure composi- vals and operating from a few mega- IT technologies. An FPGA farm, server tion techniques for advanced network, bits to multiple terabits per second farm and L2/L3 programmable network- cloud and computational research. The can be generated and applied to the ing equipment are the main building emulation platform also utilizes local target emulated or physical network. and remote laboratory-based facilities blocks of the facility, enabling the users and distributed research infrastructures 4. Topology emulations: Any topolog- to build, experiment with and use vari- (networks and computing). Figure 4 ical formation of the desired nodes ous networking technologies in the data demonstrates the multilayer, multiplat- and links is possible using the BIO plane and control plane, such as virtual- form emulation facilities at the core of emulation facility. This gives the user ization, SDN and NFV, resource/work- the Bristol Is Open infrastructure. a chance to fully examine various load allocation tools and algorithms, etc. The emulation facility offers a num- aspects of the desired technology on The emulator is connected to the BIO ber of functions instrumental for en- the realistic network topologies be- city network through 10-, 40- and 100- hanced network studies in conjunction fore deployment and installation. Gbps ports. The emulated networks can

Figure 4 – The emulation facility in Bristol Is Open includes programmable hardware in the form of FPGAs and network processors.

24 Xcell Journal Third Quarter 2015 XCELLENCE IN SMART CITIES

use standard data plane protocols such additionally facilitates running multiple REFERENCES as Ethernet, OTN and Infiniband, or cus- operating systems on the same SoC chip. tom and proprietary protocols, to inter- In this way, BIO can let multiple opera- 1. https://www.gov.uk/government/news/uk- connect with other network domains. tors host their VNFs on the same device, set-to-lead-the-way-for-smart-cities ® The emulator uses Xilinx’s ARM - and have shared and/or dedicated access 2. UN State of World Cities report, 2012/13, based Zynq®-7000 All Programmable SoC to the parallel hardware computing re- http://www.unhabitat.org/pmss/ listItem- platform, a single-chip implementation sources to boost performance. Details.aspx?publicationID=3387 of processing and FPGA technologies. 3 https://www.gov.uk/government/up- Algorithm acceleration is one of the tar- EXPERIMENTATION AS A SERVICE loads/system/uploads/attachment_data/ get use cases for the Zynq SoC, where The way cities work is changing. Using file/246019/bis-13-1209-smart-cities-back- computationally intensive tasks for re- digital technologies, BIO is creating an ground-paper-digital.pdf source allocation, path calculation, load open, programmable city that gives cit- balancing and the like are offloaded to izens more ways to participate in and 4. http://www.gartner.com/newsroom/ FPGA-based parallel processing. Hard- contribute to the way their city works. We id/2636073 ware-assisted network function virtual- call it “City Experimentation as a Service.” 5. Bijan Rahimzadeh Rofoee, George Zer- ization is another example of how we use Being open guides our procurement, our vas, Yan Yan, Norberto Amaya and Dimi- Zynq SoC-based platforms in BIO for run- data management and the hardware and tra Simeonidou, “All Programmable and ning performance-critical virtual network the software we use. Being open means Synthetic Optical Network: Architecture functions (VNFs) such as deep packet the stakeholders in BIO proactively share and Implementation,” Journal of Optical inspection, service control and security. what we learn with other cities, technolo- Communications and Networking 5, Xen-based virtualization of ARM cores gy companies, universities and citizens. 1096-1110 (2013)

FPGA-Based Prototyping for Any Design Size? Any Design Stage? Among Multiple Locations? That’s Genius! Realize the Genius of Your Design with S2C’s Prodigy Prototyping Platform

Download our white paper at: http://www.s2cinc.com/resource-library/white-papers

Third Quarter 2015 Xcell Journal 25 XCELLENCE IN 5G 5G Wireless Brings Ubiquitous Connectivity

by David Squires Vice President of Business Development BEEcube, A National Instruments Company [email protected]

As the 5G communications market begins to take shape, wireless equipment manufacturers turn to emulation systems built around Xilinx FPGAs.

26 Xcell Journal Third Quarter 2015 XCELLENCE IN 5G

to commence in 2020; 5G radio access hardware. Any prototype platform will be built using the next evolution must be capable of scaling to tens of of existing wireless radio access tech- terabits per second, accepting hun- nologies, such as LTE and Wi-Fi, com- dreds of optical fibers and supporting bined with entirely new technologies. tens of gigasamples of RF analog data. While the industry has defined the The DSP processing power re- end goals of 5G, how to actually achieve quired to implement high-order mod- those goals is the multibillion-dollar ulation schemes across many anten- question. Many companies worldwide nas and many sectors, as in massive are in the process of developing 5G in- MIMO, is immense. Tens of thousands frastructure equipment as well as the of multiply-accumulate (MAC) units many remarkable devices that will com- will be required. As wireless operators continue their municate through it. As the complexity of modern commu- relentless march to be the first to The detailed technical approaches nications systems increases, it becomes Aprovide consumers with new services for 5G are still uncertain; however, sev- impossible for all but the very largest and devices, additional bandwidth eral things are clear. Future wireless OEMs to maintain all of their own intel- and service plans that yield higher systems will use existing bandwidth lectual property. Having a rich set of IP, profits, infrastructure companies are more efficiently by exploiting spa- including massive MIMO, CPRI, multiple also racing to field the 5G equipment tial diversity through massive MIMO, waveforms as well as an LTE-Advanced that will form the foundation of the beam forming and related techniques. protocol stack, can dramatically acceler- next generation of wireless commu- New allocations of spectrum will be ate development (see sidebar). nications. To enable this 5G wireless dedicated to cellular, adding to the The world’s carriers are all trying infrastructure, BEEcube (recently overall channel capacity. Higher user to push as much processing as pos- acquired by National Instruments) throughput will be achieved, mainly sible into the cloud. This effort lever- leveraged Xilinx® FPGAs and Zynq®- through carrier aggregation and new ages the scale of data centers and in 7000 All Programmable SoCs to pro- frequency bands. The density of urban so doing, drives down the cost of provide 5G equipment manufacturers cell sites will increase, simultaneous- cessing each call. Efficient connec- with a new emulation system as well ly reducing power requirements and tion to the cloud requires 10GE, 40GE as a mobile-handset emulator. The allowing much higher spectral reuse or PCIe® interfaces. BEE7 and nanoBEE are enabling within a given area. The core network The programming model must sup- design teams to be innovative and will make increased use of the cloud port the major existing design flows productive so they can bring 5G for both data and control purposes. of C, C-to-gates, VHDL, Verilog and technologies to market ahead of the Since 5G standards have not yet been high-level modeling environments (Lab- competition. set, those companies that can demon- VIEW and MATLAB®/Simulink® are the Before describing BEEcube’s new strate working “over-the-air” systems us- two most popular). FPGA-based products in detail, let’s ing FPGA-based platforms with massive For clocks, the hardware must be take a look at the wireless communica- I/O and computational capabilities will capable of extracting embedded clocks tions industry’s vision of the 5G market have an advantage in getting their ideas from CPRI or synchronous Ethernet, and the technical challenges facing it. and specifications adopted by interna- and cleaning up the clocks and main- tional standards bodies. These platforms taining clock jitter of less than 300 fs THE 5G VISION enable rapid prototyping, making it easy across racks of gear at ADC sampling A key component of the wireless fu- to test out algorithms with real data in frequencies of up to 6 GHz in order ture will be the widespread deploy- the field and run for days or weeks. to preserve the integrity of informa- ment of 5G wireless networks. The tion-dense broadband wireless signals. primary goals of 5G are to support a THE IDEAL WIRELESS To address these challenges, BEEcube thousandfold gain in capacity, connec- INFRASTRUCTURE PROTOTYPING has created a powerful new emulation tions for at least 100 billion devices PLATFORM platform called BEE7 that leverages and 10-Gbps data rates delivered to in- No single platform can meet all the the best-in-class features of Xilinx’s dividual users. Additionally, these new requirements for prototyping 5G; Virtex®-7 FPGAs. networks will be capable of providing however, it’s already possible to dis- mass low-latency connectivity among cern the key requirements. BEE7 PLATFORM ARCHITECTURE people, machines and devices. De- A 1,000x increase in data through- The BEE7 platform is a state-of-the- ployment of 5G networks is expected put will stress any 5G communications art architecture that BEEcube de-

Third Quarter 2015 Xcell Journal 27 XCELLENCE IN 5G

Figure 1 – The BEE7 blade in an ATCA chassis is used for prototyping and field trials of the most demanding 5G wireless applications, including C-RAN, massive MIMO and millimeter wave.

signed from the ground up to meet basestation cabinets for field trials. and this is the performance rating we the above requirements of next-gen- Four of the 690T FPGAs are connected used on the BEE7. This provides 800 eration communications systems. as shown in Figure 2. Four FMC sites Gbps of connectivity per FPGA, allo- Let’s take a detailed look at the BEE7 connect each FPGA to a high-perfor- cated as shown in Figure 2. and see how one platform solves the mance analog card, supporting sam- Let’s look at the specific aspects of 5G prototyping challenges. ple rates up to 5.6 Gsps. A total of 64 the BEE7 prototyping environment and The single biggest challenge you face Gbytes of DDR3 memory is available some of the trade-offs and design deci- when creating an advanced wireless pro- to either capture data or act as a buf- sions we made along the way. totyping architecture is connectivity. The fer for broadcast data. This memory is amount of data that must be moved quick- extremely useful in the early stages of POINT-TO-POINT CONNECTIVITY ly and efficiently is enormous. The heart prototyping. Design teams can use Na- One of the goals of the BEE7 architecture of the BEE7 prototyping system is the Xil- tional Instruments’ LabVIEW or The is to provide the lowest possible data- inx XC7VX690T. This device combines 80 Mathworks’ MATLAB to create simula- flow latency and guaranteed streaming serial transceivers with 3,600 DSP slices, tion vectors and then download them throughput. These objectives would be making the 690T a world-class engine for to system memory for playback, or per- virtually impossible to achieve using a advanced wireless applications (both to form rich analysis on the captured data. shared-bus architecture, since different prototype and for early field trials). The serial transceivers in the 690T clients on the bus can tie it up at any giv- Figure 1 shows the BEE7 blade. devices are rated for 13.1 Gbps. Many en moment, increasing the latency and Note the ATCA form factor, commonly of the standards used in telecom, disrupting a true streaming environment used in the telecom world. This allows such as 10 Gigabit Ethernet and CPRI for other clients. Hence, the BEE7 uses a the BEE7 to be deployed in existing (rate 8), are centered around 10 Gbps point-to-point connectivity model instead.

28 Xcell Journal Third Quarter 2015 XCELLENCE IN 5G

High-speed serdes are the backbone short-haul optical provides the most Challenges commonly encountered of data movement within the BEE7 en- cost-effective alternative. The BEE7 is when designing with serdes include how vironment. PCB trace widths, dielec- available with short-haul optical mod- to deal with the delays; calibration; and tric thickness and via placements and ules built in. Figure 2 shows each FPGA clocking. BEEcube’s BPS software per- sizes are all tuned to provide 100-ohm having 12 lanes of serdes connecting forms automatic calibration upon boo- transmission lines from point to point to an intermodule optical transceiver tup and most of the low-level details of and thus ensure optimal performance (iMOT). These ports are exposed on the serdes are abstracted away. The net and signal integrity. In many cases, the front of the BEE7 blade and could result is that designing with serdes with- high-performance traces are buried on be used to connect directly to nearby in BEEcube’s environment is relatively inner board layers, resulting in less EMI remote radio heads (RRHs) using the straightforward, with the delay charac- radiation and an easier path to CE certi- Common Public Radio Interface (CPRI). teristics of each multigigabit transceiver fication or FCC approvals. Longer distances require special (MGT) behaving like a FIFO. Connectivity from the BEE7 blade to long-haul optical transceivers, which other equipment (including other BEE7 can transmit up to 40 kilometers with- CLOCKING ISSUES blades) can be divided into three cat- out using repeaters. These transceivers In distributed systems it becomes ex- egories: less than 3 meters, more than are easily plugged into the SFP+ and tremely difficult to traverse long dis- 300 meters and in between. QSFP connectors in the Rear Transition tances with separate clock and data. For links of less than 3 meters, elec- Module (RTM) and would be used for Standards such as CPRI are the norm in trical connections over copper are pos- RRHs that are located farther than 300 the wireless world for passing data from sible and are definitely the lowest-cost meters from the BEE7. remote radio heads to the baseband alternative. This is possible in the BEE7 The total connectivity of a BEE7 ATCA processing unit. Recovered embedded environment using the SFP+ or QSFP blade is 640 Gbps from the RTM and 480 clocks (as in CPRI) usually have bad connectors and short-patch cables, and Gbps from the front-side iMOT connec- phase-noise characteristics. The BEE7’s is encouraged for blade-to-blade com- tors. If analog I/O is not required, an ad- special PLL-based circuitry reduces this munication within one equipment rack. ditional 320 Gbps is available by using phase noise to less than 300 femtosec- For longer distances, up to 300 meters, appropriate FMC cards. onds. These clocks can be multiplied to

1 1 GbE GbE 8 8 FMC FMC 2 FPGA 16 FPGA 2 Backplane Backplane 16 A C 16 RTM RTM 12 12 iMOT iMOT 8 16 8 16 1 1 GbE GbE 8 FMC 8 FMC 2 FPGA FPGA 2 Backplane Backplane B 16 D RTM 16 16 RTM iMOT 12 12 iMOT

Figure 2 – This diagram of the BEE7 interconnect architecture shows the number of 10-Gbps channels. Total serial transceiver connectivity per FPGA is 800 Gbps.

Third Quarter 2015 Xcell Journal 29 XCELLENCE IN 5G

generate sampling clocks in the giga- Currently, modules with sample rates jitter less than 300 fs when measured hertz range, while retaining phase noise up to 5.6 Gsps are available, allowing 2 phase noise offsets from 100 Hz and 10 of less than 300 fs. GHz of spectrum to be directly synthe- MHz with a 307.2-MHz reference clock. Flexible clocks can be distributed to sized or digitized and passed from or to These DACs and ADCs also require spe- the analog FMC cards (most critical for the FPGA motherboard for modulation/ cial training sequences, which set the sampling clocks) and to the FPGAs. demodulation and any other processing phase of the data strobes to maximize required. The analog FMC cards sup- data integrity when pushing data to or RF CONSIDERATIONS port the first and second Nyquist zones, pulling data from the high-speed devic- Direct RF sampling and synthesis up to so one can examine the entire spectrum es. BEEcube’s platforms perform all the 6 GHz have long been a goal of soft- below 2 GHz or blocks of 2-GHz spec- training sequences when the boards first ware-defined radio, but only recently trum at higher frequencies. boot up. As a result, the developer never have they become practical thanks High-speed DACs and ADCs are no- needs to address these low-level details, to the advent of high-speed DACs toriously difficult to integrate effectively allowing for “out-of-the-box” operation. and ADCs. BEEcube has developed into real systems. They are interleaved a modular architecture, whereby for the highest performance and require DESIGN FLOW AND IP high-performance analog interfaces extremely stable clocks, with clock C/C++, MATLAB, VHDL, Verilog, Lab- are supported via FMC cards con- jitter requirements below 500 fs. The VIEW and Simulink all have a role to nected to the motherboard. BEE7 platform provides a typical clock play in the development of next-gen-

IP: The Accelerated Path to 5G

The algorithms being explored on forms is to increase the spectral ef- connected together in the company’s the journey to 5G wireless standard- ficiency and to improve the power LabVIEW Communication System ization are complex, sophisticated characteristics. OFDMA, as used Design Suite. LabVIEW also pro- and represent an enormous invest- in LTE-A, has high peak-to-average vides all the waveform sources and ment for anyone wanting to devel- power, necessitating expensive cir- analysis tools necessary to stimulate op them from scratch. Companies cuitry to keep power amplifiers op- and analyze a design. can accelerate their development erating linearly and thereby reduc- LabVIEW and the various IP li- efforts by partnering with someone ing out-of-band interference and braries can save months of devel- who has an inventory of the neces- intermodulation distortion. opment time. In addition, the IP is sary intellectual property (IP). Millimeter wave requires differ- known to work. LabVIEW is easy What sort of IP can speed these ent channel-model estimates be- to use and interacts with the Xilinx efforts? At the most basic level, IP cause of the different propagation tool chain in a seamless fashion, al- such as 10GE, CPRI and DDR3 is characteristics at these frequen- lowing rapid exploration and exper- essential for any high-performance cies. IP must also target the very imentation. When combined with wireless system. Moving up the wide bandwidths (up to 5 GHz) and the myriad hardware platforms of- chain, any 5G system must support high peak data rates that come with fered by NI, this is almost certain- the legacy LTE-A network, so a ba- high bandwidth. ly the quickest way to implement sic LTE-A stack is also a necessity. Having available IP is not suffi- a working prototype for any 5G Then comes IP targeting the differ- cient in and of itself. You must be able communications design. It should ent research areas of 5G: air inter- to easily connect IP together. Nation- be noted that BEEcube, now a Na- face waveforms, massive MIMO, al Instruments has an excellent selec- tional Instruments Company, will millimeter wave and C-RAN. tion of available IP that runs on a com- be providing LabVIEW support for New air interface waveforms bination of FPGAs and processors, its hardware in the near future. include GFDM, UFDM, FBMC and including a library that is focused on others. The intent of these wave- 5G prototyping. The IP can be easily – David Squires

30 Xcell Journal Third Quarter 2015 XCELLENCE IN 5G

The nanoBEE uses the same power amps, diplexors, input filters and other signal chain elements to provide a 3GPP-compatible UE emulator (output power of +23 dBm, input sensitivity of -94 dBm) that operates on a majority of LTE-A bands as well as the unlicensed bands at 2.4 and 5 GHz. The nanoBEE, which is shown in Figure 3, was completed from concept to product launch in less than 18 months.

FIVE YEARS AWAY The race is on to solve many 5G technical challenges. We’re still five years away from commercial deployment, but many companies need to prototype Figure 3 – The nanoBEE is a terminal emulation system designed for speeding these emerging algorithms and appli- the development of next-generation wireless products. cations now as standards begin to firm up. Xilinx FPGAs and Zynq SoC devices, coupled with commercially avail- eration 5G designs. BEEcube plat- processing and interconnect, is likely to able 5G prototyping platforms such as forms have always been design tool be run from a battery for mobile testing those from BEEcube, can save signifi- agnostic, allowing designers to use and will have a highly integrated MAC cant development time vs. the develop- whatever design flow they prefer. and upper layers of protocol handling ment of custom prototyping platforms. With all bases covered from a tool built in. BEEcube was able to leverage These tools allow system architects flow perspective, the focus quickly the Zynq XC7Z100 SoC device to create and designers to get on with the job of turns to intellectual property. an elegant UE emulator. finding the best architectures and al- BEEcube provides many of the The physical layer of a 5G UE must gorithms, rather than architecting the low-level interfaces necessary to be flexible and would be challenging platform on which to prototype. They build high-performance communica- for any typical processor architec- also allow the carriers to accelerate tions designs. BEEcube offers 10- and ture, but for the 2,020 DSP slices in their early trials and gain experience 1-Gigabit Ethernet cores, while Xilinx the Zynq 7100 device, implementing with new systems, algorithms and net- supplies CPRI and PCIe, along with a the PHY is straightforward. UE con- work architectures. synchronous version of Xilinx’s Au- nectivity demands at 10 Gbps are also As we look to 2020 for widespread rora core for internal communication straightforward in the Zynq 7100 SoC. 5G deployment, it is likely that most between FPGAs. In addition, the in- What made the Zynq family ideal OEMs will sell production equipment terface to onboard DDR3 memory is for a UE emulator were the two A9 based on Xilinx FPGAs and All Pro- provided, as are standard FIFOs and ARM® cores, which can implement grammable SoCs. The hardware com- Block RAM interfaces. the MAC and higher-level protocol lay- plexity of 5G’s physical layer is just too High-level IP blocks are a great ers. A large percentage of existing cell challenging to guarantee that ASIC im- means to accelerate the design process. phones use ARM processors, so com- plementations will be free of hardware The sidebar discusses them in detail. panies are able to reuse much of their bugs and flexible enough to address existing code base for upper-layer pro- ever-evolving standards. Keeping the NANOBEE, THE SOLUTION FOR cessing. The tight interface between hardware “soft” will be the wise path USER EQUIPMENT the ARM core and programmable fab- chosen by the smartest OEMs. The BEE7 satisfies the need for massive ric keeps latency low and improves A great way to learn more about connectivity and DSP horsepower for performance. Keeping the Zynq SoC BEEcube’s solutions (and those of infrastructure solutions. What about a and the other nanoBEE hardware un- parent company NI) is to attend Na- tool for emulating the handset (or to der 5 watts means you can power the tional Instruments’ NI Week in Austin, use the industry term, user equipment, product with a battery pack, definitely Texas, from Aug. 3 to 6, http://www. or UE)? A handset requires modest DSP an asset for testing a UE emulator. ni.com/niweek.

Third Quarter 2015 Xcell Journal 31 XCELLENCE IN INDUSTRIAL IOT

Innovative Platform-Based Design for the Industrial Internet of Things Flexible IIoT systems connecting everything to everything will adapt and evolve through a platform approach.

by Andy Chang Senior Manager, Academic Research National Instruments Corp. [email protected]

32 Xcell Journal Third Quarter 2015 XCELLENCE IN INDUSTRIAL IOT

ver the past decade, be characterized as a vast number of subsystems that may be in a different our society has be- connected industrial systems that are software language or framework, and come increasingly de- communicating with one another and different hardware protocol. For the pendent on the latest coordinating their data analytics and ac- last four decades, NI has provided pow- technologies in elec- tions to improve industrial performance erful, flexible technology solutions that tronics and commu- and benefit society as a whole. help engineers and scientists accelerate Onications, from mobile devices to in- Industrial systems interfacing the productivity, innovation and discovery. telligent vehicles to home automation. digital world to the physical world NI invests greatly in providing an inte- These physical objects or “things” are through sensors and actuators that grated hardware and software platform embedded with electronics, software, solve complex control problems are to help its broad spectrum of custom- sensors and connectivity to create the commonly known as cyber-physical ers—from healthcare and automotive Internet of Things (IoT). Introduced systems. These systems are being com- to consumer electronics and particle by technology pioneer Kevin Ashton bined with Big Analog Data solutions physics—overcome complexity. in 1999, the concept of the IoT is that to gain deeper insight through data and Specifically, the NI LabVIEW recon- advancement in connectivity among hu- analytics. Imagine industrial systems figurable I/O (RIO) architecture, as mans, machines and infrastructure will that can adjust to their own environ- shown in Figure 2, takes advantage of increase intelligence, business insight, ments or even their own health. Instead the openness of both LabVIEW soft- efficiency and innovation. of running to failure, machines sched- ware and commercial off-the-shelf The IoT has the potential to impact ule their own maintenance or, better (COTS) hardware to provide a common our lives profoundly. NI customers play yet, adjust their control algorithms dy- architecture for designing and building a critical role in inventing, deploying namically to compensate for a worn IoT systems. Recently, LabVIEW RIO and refining the consumer and industri- part, and then communicate that data incorporated the Xilinx Zynq®-7000 All al products and systems at the center of to other machines and the people who Programmable SoC platform. The result the IoT, as well as the wired and wire- rely on those machines. will be to continue to drive openness less infrastructure connecting those As such, the landscape of the IoT, as and scalability through the introduction products and systems together. shown in Figure 1, can be further seg- of Linux real-time operating systems Spanning well over a decade, the mented into three parts: the intelligent (RTOS) and by creating platforms that NI and Xilinx technology partnership edge (sensor/actuator), the system of span across academia and industry has provided engineers and scientists systems and end-to-end analytics that with the same chip set. By combining with tools to create world-changing in- support all the connectivity and data LabVIEW RIO architecture with tech- novations. NI has delivered latest gen- analytics while meeting requirements nologies such as NI DIAdem and NI In- erations of Xilinx® devices in succes- of latency, synchronization and reliabil- sightCM for data management and data sive generations of its most advanced ity. More often than not, different ven- aggregation, customers can design, products, ranging from NI FlexRIO dors produce these intelligent products, build and test IoT devices throughout modules to CompactRIO controllers, which have various embedded proces- their product design cycle and perform as well as NI System on Module (SOM) sors, protocols and software. The inte- preventative maintenance with a com- and myRIO devices. NI takes great gration of these products throughout mon platform and architecture. pride in its role helping innovators to their design cycles to the final deploy- design, build and test these intelligent ment is a key challenge. It will take a INTELLIGENT EDGE SENSORS devices with integrated software and platform-based approach to achieve a FOR HEALTHCARE hardware platforms. fully connected world. The Internet of Things is already im- pacting our lives greatly. We have be- THE CHALLENGES OF IOT PLATFORM-BASED DESIGN come increasingly dependent on per- According to Gartner Inc., an estimat- The platform-based design concept sonal devices such as smartphones ed 4.9 billion connected devices will be stems from a formal modeling tech- and tablets and home devices such as used in 2015, rising to 25 billion in 2020. nique, clearly defined abstraction levels the Nest thermostat and Philips’ Hue These connected systems range from and the separation of concerns to pro- light bulbs. Meanwhile, the healthcare smart factory machines and advanced mote an effective design process. All Internet of Things market segment is driver assistance systems (ADAS) in of these factors are critical in design- poised to hit $117 billion by 2020 us- automobiles, to energy grids in smart ing and building IoT systems. The idea ing smart and connected sensors that cities and wellness wearables that help is to provide engineers with the right allow patients to stream data to the people live longer, healthier lives. The level of abstraction while also provid- medical infrastructure for diagnosis Industrial Internet of Things (IIoT) can ing connectivity to other elements and and prognosis. Devices such as fitness

Third Quarter 2015 Xcell Journal 33 XCELLENCE IN INDUSTRIAL IOT

wearables and smart watches have just low patients to perform additional ma- include both digital and analog signals begun to emerge in the marketplace, neuvers such as negotiating stairs and from sensors, RF antennas and more— and researchers are actively developing walking on slopes. all at consumer volumes and for the low- technologies for in-home rehabilitation est price possible. For tomorrow’s test and even intelligent prostheses. MACHINE-TO-MACHINE (M2M) challenges, traditional ATE falls short. In this market, Cyberlegs is a European COMMUNICATIONS Test engineers will need smart ATE for FP-7 project led by Professor Paolo Dario Gartner estimates there will soon be the IoT’s smart devices. ST-Ericsson is a of the BioRobotics Institute at the Scuola more connected devices than there case in point. Superiore Sant’Anna di Pisa in Italy. The are humans on the planet. By 2022, ST-Ericsson is an industry leader project aims to develop an artificial cog- each household could contain more in semiconductor development for nitive system for lower-limb functional than 500 connected devices, creating smartphones and tablets. It has de- replacement for trans-femoral amputees. 35 zettabytes of data that the commu- velopment and test centers world- The goal is a multidegree-of-freedom sys- nications infrastructure must be able wide and multiple characterization tem with the ability to replace the lower to handle. With new intelligent devices labs that test and validate RF com- limb and otherwise assist the patient. being introduced to the marketplace ponents and platforms used in the Dr. Nicola Vitiello, who is responsi- and new communications standards company’s products. These platforms ble for developing and integrating the and protocols proliferating, compa- usually contain multiple radios such Cyberlegs system, used CompactRIO nies need to ensure they have a scal- as GPS, Bluetooth, 3G and 4G, among extensively to create initial prototypes able framework to design, prototype others. For one test set, a platform and validate subsystems and control and test these M2M communications needs to make about 800,000 mea- algorithms to predict accurate walking to stay ahead of their competition. surements. The complex nature of gaits for different patients (see Figure Traditional automated test equipment the chips that ST-Ericsson develops 3). Using the Zynq SoC’s scalability in (ATE) was optimized to test technolo- requires validation labs that are pli- an NI SOM drastically decreased the gy that harnessed the power of Moore’s able enough for a variety of RF stan- footprint and power consumption re- Law, and it does this very well. But over dards, but also have high enough quired. Dr. Vitiello took advantage of the past few decades, a subtle shift to in- performance for very stringent tests. the platform’s adaptability and was tegrate more analog technology into ICs Even interfacing with these chips re- able to push the intelligence closer has resulted in a test challenge that requires multiple standard and custom to the sensor and actuators in order quires much more than Moore. Innova- digital protocols. Traditional boxed to upgrade the prosthesis with a fully tion for the IoT has tasked test engineers instruments such as RF analyzers, active knee. This development will al- with verifying mixed-signal systems that generators and digital pattern gener-

PWM

Intelligent Edge Systems System of Systems End to End Analytics The Edge RFID

Smart Glasses Processes & Rules Management RGB Camera NI System on Module Switches

NI CompactRIO Network & PLM

Wired Torque Sensor or Wireless Smart Tools PC Plant Node RFID NI Single-Board LabVIEW Server RIO Motor Drives

PWM Manipulator RFID NI PXI NI CompactRIO Motor Drives

Figure 1 – The three fundamental pillars of smart systems and the Internet of Things are intelligent edge systems,the system of systems and end-to-end analytics. Devices are becoming increasingly intelligent and defined through software.

34 Xcell Journal Third Quarter 2015 XCELLENCE IN INDUSTRIAL IOT

I/O Modules

Signal Screw ADC Conditioning Terminals

Signal DAC BNC Conditioning + PCI Bus + Signal DI Conditioning D-Sub

Signal D0 Custom Conditioning

Real-Time High-Speed Reconﬁgurable Digitizers Attenuation Connector Sensors Processor Bus FPGA and Isolation and Filters Block and Actuators

Figure 2 – NI LabVIEW’s Reconfigurable I/O (RIO) architecture is based on four components: a processor, a reconfigurable FPGA, modular I/O hardware and graphical design software.

ators are bulky, expensive and simply Future” implies the extensive use of a these smart tools. Use of the NI SOM sped not flexible enough. modular platform with a high abstrac- up the development process from design ST-Ericsson’s test engineers have tion level based on COTS modules, as to prototype to deployment. Before de- replaced their traditional boxed instru- shown in Figure. 4. Smarter tools are key veloping on the NI SOM, Airbus created a ments with the NI PXI platform and components for improving efficiency in prototype built around a Zynq SoC-based chose to use NI’s FlexRIO—which con- the Factory of the Future. These smart CompactRIO controller (NI cRIO-9068) tains a Xilinx Virtex®-5 FPGA—to com- devices communicate with a main infra- that allowed them to integrate IP from municate with different digital standards structure or locally with operators, but existing Airbus libraries and open-source such as serial peripheral interface (SPI) only when needed to provide situational algorithms to validate their concepts and inter-integrated circuit (I2C). When awareness and make real-time decisions quickly. The flexibility of using graphical a digital adapter module was unavail- based on local and distributed intelli- and textual programming, along with reus- able, the team quickly developed its own gence in the network. ing third-party development ported on top without having to worry about the back In the case of a manufacturing facility, of the Xilinx Zynq SoC, and the NI Linux end to the PC and communication with smart tools can help simplify the produc- RTOS offered the perfect level of abstrac- the FPGA. Overall, the PXI-based system tion process and improve efficiency by tion for developing these tools. Airbus was 10 times faster and three times less removing physical data logs and manuals. engineers can now reuse the code they expensive than the previous solution, the Operators must focus on their operation- developed on the NI SOM as a deployed company reported. The PXI platform also al tasks, during which they need to keep solution rather than having to restart the provided the flexibility needed to adapt their hands free for using the appropriate entire design process. to multiple digital and RF standards. tools. Most of Airbus’ previous initiatives Airbus evaluated several SOMs and involved paperless projects that focused embedded single-board computers FACTORY OF THE FUTURE on paper suppression or on replacing pa- (SBCs), and found there was no com- Airbus, a leader in aircraft manufactur- per with tablets; they still consumed pas- parison with NI’s platform-based de- ing, is launching a research-and-technol- sive, “dead” data. sign approach and hardware-software ogy project aimed at pushing emerging Smart tools provide an alternative, data integration. Airbus engineers estimate technologies to improve the competive- in context, which is generated and con- that their time to deliver with the NI ness of Airbus’ manufacturing process- sumed continuously—in other words, live SOM is a tenth of what it would be uses, still dominated by manual opera- data. Airbus tested the Zynq SoC-based NI ing alternative approaches due to the tions today. The Airbus “Factory of the SOM as the foundation platform for all of productivity gains of NI’s approach to

Third Quarter 2015 Xcell Journal 35 XCELLENCE IN INDUSTRIAL IOT

system design, particularly with NI Li- nux Real-Time and the LabVIEW FPGA Module. With the software already provided by the NI SOM, Airbus can focus more on a system’s key features, such as image processing on FPGAs.

SMART RENEWABLE ENERGY Another key Industrial IoT application is in renewable energy, where demand has rapidly increased as fossil fuel plants become decommissioned. Grid operators are finding that traditional measurement systems do not offer adequate coverage to handle these new challenges or manage the new risks they face. National Grid U.K., the transmission system operator for nearly 20 million people in the United Kingdom, is deploying an advanced, upgradable grid measurement system to provide better operational data for the condition of the U.K. grid. Like many energy providers, Na- tional Grid U.K. faces the challenges that come with a rapidly changing Figure 3 – Italy’s Cyberlegs project has developed an artificial cognitive system for lower-limb replacement and rehabilitation. grid; thus, the company is focused on developing a flexible solution that can be

Figure 4 – The Factory of the Future will require distributed, networked processing and I/O capabilities that add intelligence to the tools and devices used in manufacturing.

36 Xcell Journal Third Quarter 2015 XCELLENCE IN INDUSTRIAL IOT

Figure 5 – A smart grid architecture with an open, extensible approach to intelligent devices allows grid engineers to quickly respond to rapidly evolving measurement and control needs.

upgraded with new software as the mea- ment, National Grid U.K. engineers can ble hardware architecture such as the Xil- surement needs of the grid and amount customize the information available for inx Zynq SoC, deployed across many ap- of data available evolve (see Figure 5). grid operation and easily make upgrades plications, removes substantial hardware Gathering reliable, real-time data from all as needs change. This approach improves complexity and makes each new problem grid areas is critical to identifying prob- grid monitoring and reliability while re- primarily a software challenge. The same lems early and preventing power disrup- ducing the amount of equipment needed. principle must be applied to software tions. To keep the grid running consis- Additionally, with the advanced process- tools to form a powerful hardware-soft- tently, operators must be able to gather ing power of CompactRIO, National Grid ware platform that creates a unified data from a wide range of measurements U.K. can easily maintain its network of solution. An effective platform-based and quickly gain insight from that data connected systems and push intelligence approach does not focus on hardware or to monitor the overall health of the grid. down the grid to turn massive amounts of software but instead on the innovation Software-designed systems provide cus- raw data into useful information, keeping within the application itself. tomized measurement solutions that can the lights on for millions of businesses and The ongoing design of the IIoT be upgraded in the future as new grid homes throughout the United Kingdom. represents a massive business and modernization challenges arise. technology opportunity for everyone. To address these challenges, Nation- A SMARTER CONNECTED WORLD Organizations worldwide are working al Grid U.K. adopted a platform built on The idea of a smarter world that involves hard to define the IIoT and actively the Zynq SoC-based CompactRIO system systems with sensors and local process- gather use cases to better understand that can provide more measurements ing connected to share information is how best to foster more innovation. and adapt with the evolving grid for gen- taking hold in every industry. These IIoT Engineers and scientists are already erations to come. This interconnected systems will connect with users and with implementing systems on the leading network includes 136 systems, with 110 one another on a global scale to help us- edge of the IIoT, but they face many permanently installed in substations ers make more informed decisions. De- unknowns and much work ahead. throughout England and Wales, and 26 veloping and deploying these systems Engineers and scientists must start portable units that provide on-the-go will involve a massive investment for concentrating on a platform-based spot coverage as needed. An identical decades to come. The only way to meet approach and become part of the IIoT software application runs on both ver- the needs of today and tomorrow is by generation by getting involved with sions, minimizing the impact on system deploying a network of systems flexible these bodies to define the future and integration, training and support. With an enough to evolve and adapt through a ensure that businesses focus on inno- open, flexible, software-designed instru- platform-based approach. A single flexi- vation and not simply integration.

Third Quarter 2015 Xcell Journal 37 XCELLENCE IN ADAS/AUTONOMOUS VEHICLES The Coming Revolution in Vehicle Technology and its BIG Implications

38 Xcell Journal Third Quarter 2015 XCELLENCE IN ADAS

The three major trends in the automotive industry— electrification, connectivity and autonomy—have one thing in common: software. We are on the threshold of a radical change in vehicle technology. No, it’s not Wautomation, although that will come very soon. Instead, change is being driven by the underlying technology for automa- by Thomas Gage tion that is already here and advancing CEO and Managing Director rapidly; that is, crash avoidance technol- Marconi Pacific ogy delivered by advanced driver assis- [email protected] tance systems, or ADAS for short. Jonathan Morris ADAS makes safety and marketing Senior Associate sense. Whether it is Daimler, Toyota, Ford, Marconi Pacific Nissan, GM, another vehicle OEM or even [email protected] Google, none are going to put vehicles on the road that can steer, brake or accelerate autonomously without having confidence that the technology will work. ADAS promises to first reduce accidents and assist drivers as a “copilot” before eventually taking over for them on some and eventually their entire journey as an “autopilot.” As for how quickly the impacts of this technology will be felt, the adoption curves for any new technology look very similar to one another. For example, the first commercial mobile-phone network went live in the United States in 1983 in the Baltimore-Washington metropoli- tan area. At the time, phones cost about $3,000 and subscribers were scarce. Even several years later, coverage was unavail- able in most of the country outside of dense urban areas. Today there are more mobile-phone subscriptions than there are people in the United States, and more than 300,000 mobile-phone towers connect the entire country. Low-end smartphones cost about $150. Vehicle technology is moving forward at a similar pace. And, because transportation is so fundamental to how we live, the disruptive effects are likely to be as- toundingly large.

Third Quarter 2015 Xcell Journal 39 XCELLENCE IN ADAS/AUTONOMOUS VEHICLES

The ADAS systems of today and autonomous driving systems of tomorrow will rely on software to make sense of a slew of data from sensors, cameras, the Internet, infrastructure and other vehicles.

THREE VEHICLES, ONE tive industry: a sleek Tesla Model S SOFTWARE: REFINING TODAY, REVOLUTION rolling quietly past, a late-model sedan REVOLUTIONIZING TOMORROW The development of automation and with an Uber “U” in the back window Since 2004, the costs of electronics in ADAS is not the first trend to upend picking up a passenger and a heavily an average vehicle have doubled from the auto industry status quo. Inter- modified Lexus SUV with a spinning 20 to 40 percent. Today’s luxury vehi- national competition and liberalized lidar on the roof, driving itself down cles commonly contain 100 micropro- trade forever altered the automotive the street while a Google employee cessors and run 100 million lines of soft- OEM landscape, eroding the U.S. sales (or an employee from an auto OEM ware code, controlling everything from market share of the Big Three auto- in one of their vehicles in other parts engine timing to infotainment systems. makers from 72 percent to 45 percent of the world) collects data. These We are now at an inflection point where in the last 20 years. And while vehicle daily sights represent three technol- software, sensors and processors are technology has advanced enormously, ogy-driven trends that are simultane- delivering entirely new areas of vehicle the basics of driving have not changed ously arriving to significantly disrupt functionality, and not simply transition- much in the last 40 years. the automotive status quo: electrifi- ing conventional functions from me- Now, every day in California’s cation, connectivity and autonomy. chanical to electronic control. Both the South Bay, you can commonly see Each trend is moving at a different ADAS systems of today and the autono- three vehicles representing three pace, but all three have one thing in mous driving systems of tomorrow will world-changing trends in the automo- common: It’s all about the software! rely completely on software to make

New Technology Systems

Inputs Processors Actuators Automated Sensors Vehicle • Radar • Electronic Control • Brake Actuators Systems • Camera Unit (ECU) • Steering Actuators • Trac Jam Assist • Ultrasonic • Throttle Actuators • ACC + Lane • Lidar Keeping Other • Highway • Laser mapping Automated Driving • V2V System Software Operating System (OS) Driver – Vehicle Interface (DVI)

Figure 1 – The basic ADAS architecture starts with a set of sensors that provide data on driving conditions to an ECU.

40 Xcell Journal Third Quarter 2015 XCELLENCE IN ADAS/AUTONOMOUS VEHICLES

sense of a slew of data from sensors, Driving Environment Complexity cameras, the Internet, infrastructure and other vehicles. The increasing complexity of vehicles has already shifted the automotive Fast value chain. The trends of electrifica- Interstate tion, connectivity and automation will Crusing State only accelerate this shift in value to- Freeway Highway ward those companies that create elec- Merging tronics and software, and away from On/Off OEMs that fail to innovate. This shift will have two effects. First, software will become a critical mar-

Speed Residential Streets & ket differentiator, pressuring OEMs to Intersections shorten product cycles and provide Highway City Streets support and updates for legacy systems. Trafﬁc Jam & To meet consumer demands for current Intersections technology, OEMs are now forced to Parking significantly modify or introduce new

Slow models after only three or four years, while previous product cycles averaged Threat Complexity five to eight years. This leaves OEMs Low High with many challenges including rapid innovation, complex QA testing, higher development costs, less time to amor- Figure 2 – ADAS software algorithms must account for road types, speed and threat complexity. tize R&D and the need for new sales and vehicle-ownership models. Second, the shift to software allows new entrants to innovate in an industry Technology Introduction Paths with notoriously high barriers to entry. After decades of the same players High dominating the industry, Google, Apple, Tesla and Uber are all poised to remake the automotive landscape through software, a thought that would have seemed highly unlikely even five years ago. In a typical ADAS-equipped vehicle (Figure 1), applications such as forward Technology collision avoidance (FCA) are enabled Functionality by a set of sensors that provide data on the external driving environment to an electronic control unit (ECU). This unit then uses software to determine whether a threat is present and operates brake actuators (or potentially, other counter- Low measures) to mitigate the threat. The sensors available today for Complexity of Environment driver assistance applications are the hardware foundation for autonomous Low High vehicles. But tomorrow’s sensors will necessarily be smaller, faster and Figure 3 – Simpler systems like “traffic jam assist” will roll out first, followed by systems able to cheaper. For example, Continental AG’s operate the vehicle. sensors and processors can transmit

Third Quarter 2015 Xcell Journal 41 XCELLENCE IN ADAS/AUTONOMOUS VEHICLES

The software algorithms that will let vehicles drive themselves more efficiently and safely than human drivers in complex environments remain the biggest challenge. and recalculate algorithms needed to plex urban settings, and offer addition- this technology are available, how fast understand the driving environment al functionality such as the ability to will consumers adopt it? every 10 to 60 milliseconds, while the merge, change lanes or negotiate an To understand the adoption of human brain can pass a message from intersection. A subset of these OEMs, ADAS-enabled and autonomous vehi- a sensory neuron to a motor neuron in such as Volvo and Ford, are introduc- cles, it is instructive to look at adoption only a few milliseconds. ing moderate-functioning systems for rates of other technologies. As a general But the real gap between the ADAS defined geographic areas (typically geo- trend, modern technologies such as the systems of today and the fully autono- fenced), such as a particular stretch of cell phone, Internet and PC have been mous systems of tomorrow is seen in an interstate between two cities, to take adopted at a much faster rate than old- software. Regardless of how fast inputs advantage of laser scan mapping data. er technologies such as the VCR or TV. can be processed, the software algo- Over time, system functionality will in- Cars have conventionally been one of rithms that will allow vehicles to drive crease and the number and complexity the slower technologies to be adopted. themselves more efficiently and safely of geographic areas available to the sys- This is largely due to their high relative than human drivers in complex driving tem will expand (green line in Figure 3). cost as compared with consumer elec- environments remain the biggest chal- Finally, Google’s approach has been to tronics, and to the need for highways to lenge. Complexity is defined by both develop a highly functioning, fully au- be constructed. In contrast, the smart- the number of threats, characterized by tonomous vehicle from the outset (in phone is considered to be the fastest- the types of threats that a driver can en- geo-fenced areas and for low-speed city adopted technology in history, on track counter on different road types (for ex- or campus driving), then test and refine to reach saturation in a decade. Mobile ample, pedestrians, vehicles traveling at its capabilities in increasingly complex phones (largely what we today call “fea- a right angle to your vehicle, bicyclists) environments (in orange on Figure 3). ture phones”) took 20 years to reach sat- and the speed at which the vehicle is uration and conventional landlines took driving (see Figure 2). CONSUMER ADOPTION a century (largely because of the need As they race to improve their soft- AND DIFFUSION to build out the landline networks). ware, vehicle OEMs and their suppliers While OEMs are choosing different ADAS-equipped and autonomous are introducing their technology to the strategies to bring ADAS and vehicle vehicles likely will be adopted at rates market in three distinct ways. OEMs autonomy to market, ADAS-equipped slightly slower than other modern tech- such as BMW, Daimler and Nissan have vehicles of increasing capability have nology due to vehicle costs, but they already begun to sell moderate-func- already been introduced nearly every will still be adopted much faster than tionality ADAS systems designed to year since 2010 and continue to roll out conventional automobiles were. As operate in simple driving environments annually. In 2013, fully 29 percent of pas- with the uptake of other new technol- such as interstates. Without needing senger vehicle models offered optional ogies, we expect a wave of first movers to account for traffic signals, turns or forward-collision warning, and of those, and early adopters to drive early sales multidirectional traffic, these vehicles 12 percent had autonomous braking. of ADAS-equipped vehicles, followed automatically steer, brake and accel- This year’s Mercedes entry-level premi- by gradual adoption by the majority erate in lower-speed situations using um CLA sedans come standard with a of consumers once the safety benefits systems like “traffic jam assist” (a tra- forward-collision prevention system, have been proven (see Figure 4). Impor- jectory represented by the blue line in and Volvo has made its City Safety brak- tantly, the current additional cost of a Figure 3). Eventually, systems will op- ing system standard on its XC60 since typical ADAS suite of equipment is only erate at higher speeds or in more-com- 2010. Now that the early generations of about $3,000 (declining at about 7 to 9

42 Xcell Journal Third Quarter 2015 XCELLENCE IN ADAS/AUTONOMOUS VEHICLES

percent per year), or about 10 percent of the cost of the average vehicle sold in Annual additional % of new vehicle sales the United States of $33,560. For luxury w/ tech vehicles, the ADAS equipment cost represents only 2 to 3 percent of the vehicle 16% – Entry Mass Market sale price on average. Premium Luxury Marconi Pacific consumer research 14% – into ADAS and autonomy indicates Early that consumers will be initially drawn 12% – adopters to the safety and convenience of this technology. Safety will be a large moti- 10% – First vator for families as they begin to hear movers that ADAS-equipped vehicles avoided 8% – crashes that might have injured or killed the vehicle’s occupants. But the Early big driver (pun intended!) will be time 6% – Majority recapture. Being able to cruise along a Late freeway (and soon enough, other road 4% – Majority types) while paying limited attention to the road will be a significant accel- 2% – erator of demand. Laggards Marconi Pacific has built a diffusion model to better understand the pace 0% – – – – – – – – – – – – – – – – – – – – – – – – – of introduction of the technology and the uptake by consumers. The model is Figure 4 – Additional sales for ADAS and autonomous vehicle (AV) technology will pick up as scenario based, with numerous inputs. consumers see the safety and convenience benefits. A few key factors are annual vehicle sales, ADAS technology introduction dates and fleet turnover forecasts. The Cumulative % of new vehicle sales w/ tech results are striking. In one run of the Entry model, by 2035 more than 50 percent of 120% – Mass Market vehicles and 85 percent of new-vehicle Premium sales across all segments had one gen- Luxury eration or another of ADAS-equipped 100% – or autonomous vehicles (see Figure 5). Of course, different levels of ADAS and of autonomy will have different 80% – impacts on society, including different levels of total annual crash reduction, different impacts on traffic congestion 60% – and different impacts on shared-vehicle, Uber-like services.

40% – AUTO ECOSYSTEM IMPLICATIONS The automotive sector and adjacent industries form a large ecosystem 20% – with pervasive reach across the global economy; in the United States, transportation represents just under 10 per-

0% – – – – – – – – – – – – – – – – – – – – – – cent of GDP. As innovation in the form –– of electrification, connectivity and au- Figure 5 – Cumulative sales for ADAS/AV technology could approach 85 percent of total car tomation disrupts the status quo, the sales by 2035, according to one model. effects will be felt not just by OEMs,

Third Quarter 2015 Xcell Journal 43 XCELLENCE IN ADAS/AUTONOMOUS VEHICLES

Xilinx All Programmable Devices: De Facto Standard Platforms for ADAS and Beyond

by Mike Santarini sensor performing a number of tasks. To- phisticated ADAS systems and are quick- Publisher, Xcell Journal day’s high-end automobiles can include ly replacing much less versatile ASSPs. ® Xilinx, Inc. ADAS products with not only highly The combination of the Zynq SoC’s ARM [email protected] advanced rear-camera systems, but fu- processors and FPGA logic on the same sion-sensor systems that simultaneously device has enabled OEMs to build highly sophisticated, All Programmable ADAS ilinx Inc. has a rich his- perform multiple tasks such as blind-spot platforms that can scale with automotive tory in the automotive and lane-departure warning, pedestri- product lines and be upgraded with new market, but over the an and sign detection, automated cruise enhancements to suit demanding and last four years and with control, forward-collision warning and ever-evolving customer requirements. the commercial launch even drowsy-driver detection and warn- Automotive OEMs leverage the Zynq of the Zynq®-7000 All ing. The latter system monitors the driv- SoC in many platform configurations. The XProgrammable SoC in 2011, the com- er’s eyes to detect eye patterns that may device serves as a multisensor, multifea- pany has quickly become the platform indicate he or she is falling asleep at the ture driver assist platform, a high-resolu- provider of choice in the burgeoning wheel and needs a sonic alert or even a tion video and graphics platform, a vehi- market for advanced driver assistance puff of smelling salts to snap to. cle networking and connectivity platform, systems (ADAS). Companies such as What’s more, over the last five years, and an image-processing and recognition Mercedes-Benz, BMW, Nissan, VW, Hon- an increasing number of features once platform. In these applications, customers da, Ford, Chrysler, Toyota, Mazda, Acu- offered only in premium vehicle lines are implement algorithms for their design’s ra, Subaru and Audi are among the many quickly becoming standard in even econ- most complex and compute-intensive OEMs that have placed Xilinx® FPGAs omy lines. In short, OEMs are leveraging functions in the logic portion of the Zynq and Zynq SoCs at the heart of their most ADAS as competitive selling points for SoC and implement serial processing advanced ADAS systems. And with the their vehicles. functions in the onboard ARM process- new Zynq UltraScale+™ SoCs, Xilinx is Today, OEMs are moving above and being system. They leverage the Zynq SoC’s sure to play a leadership role in the next yond the ADAS warning features and are high-speed I/O to link to sensors and cre- phases of automotive electronic innova- starting to network ADAS into the con- ate highly reliable connections to automo- tion: autonomous driving, vehicle-to-ve- trols of the vehicle to actively and momen- tive networks. Customers also leverage hicle communication and vehicle-to-in- tarily take charge. Adaptive cruise control, IP from Xilinx and from members of the frastructure communication. intelligent speed control, lane-keep assist, Xilinx Alliance Program, as well as Xil- The basic goal of an ADAS technology collision avoidance and even automated inx’s Vivado® Design Suite and the new is to make drivers more aware of their en- parking are available in many models. And SDSoC™ development environments, to vironment so they can drive safely while these remarkable technologies represent quickly develop ADAS platforms. enjoying the driving experience. Around the first steps in the automotive industry’s Xilinx’s new Zynq Ultrascale+ SoC is 10 years ago, luxury OEMs began to of- race to offer consumers fully autonomous, destined to enable these same OEMs to fer the first ADAS systems in the form of self-driving vehicles in which the driver is bring autonomous vehicles to the mass backup radar/lidar sensor-based products essentially the copilot. What’s more, these market. With 64-bit application proces- that would simply beep if they detected technologies will also be leveraged heav- sors, real-time processors, on-chip memo- an object behind the vehicle. Over time, ily to facilitate vehicle-to-vehicle and very and FPGA logic on the same device, the these systems advanced to multisensor hicle-to-infrastructure (V2X) communica- UltraScale+ version of the Zynq SoC will versions that fused radar with cameras tions designed to enable governments to allow OEMs to create ever-more-sophisti- and even lidar, to not only give drivers a build smart infrastructure—streets, traffic cated fusion systems with a programma- view of what’s behind them but also detect signals and so on—to streamline traffic ble platform that can’t be matched by any if something is coming up from the side. flow in real time, making transportation other silicon architecture. In subsequent years, the ADAS sen- safer, more efficient and economical, and For more information on Xilinx’s role sor arrays morphed from doing one task better for the environment. in the rapidly advancing ADAS market, at the back of a vehicle into networked Xilinx’s All Programmable devices and visit http://www.xilinx.com/applications/ sensor arrays that see 360 degrees around especially the multi-award-winning Zynq and even inside an automobile, with each SoC are at the heart of today’s most so- automotive.html.

44 Xcell Journal Third Quarter 2015 Third Quarter 2015 Xcell Journal 45 XCELLENCE IN ADAS/AUTONOMOUS VEHICLES

but also by numerous other sectors and security companies have significant The three technology-driven trends and businesses that have previously opportunities to enable and secure this that are simultaneously arriving to been structured around conventional new functionality. Telematics content and significantly disrupt the automotive personal vehicles (see Figure 6). platform providers, as well as telecom net- status quo—electrification, connec- Automakers have many opportu- work operators, have opportunities in ar- tivity and autonomy—are here today. nities as the race to deliver advanced eas such as mapping, car sharing, parking Companies that move quickly to take functionality accelerates. These in- apps, infotainment, vehicle-to-X commu- advantage of the opportunities are clude more luxury vehicles and fea- nication and vehicle-to-Web integration. likely to succeed. Laggards—well, his- tures, more telematics/infotainment Traditional vehicle hardware suppli- tory has shown what usually happens and new “driving” experiences. But ers are likely to be price-squeezed as to them. there are also risks regarding com- value moves to software and infotain- petitive timing, technology capability ment. Auto insurance companies will REFERENCES (hardware and software), complex need to develop new business models 1. Wards Auto, “U.S. Total Vehicle Sales sourcing, technical selling capability as crashes diminish in both frequency Market Share by Company, 1961-2014” of dealers and brand differentiation. and severity, with corresponding reduc- 2. Gapper, John, “Software Is Steering the Automotive OEM, component and af- tions in premiums. Property develop- Auto Industry,” Financial Times, Feb. termarket suppliers also are likely to ers, garages, transportation engineer- 18, 2015 have increased product liability risks ing and construction firms, and transit 3. Kuchinskas, Susan, “Crash Course: Train- as their technologies assume direct agencies (to name a few industries) ing the Brain of a Driverless Car,” Scien- responsibility for more of the driving. must all consider how transportation tific American, April 11, 2013 Auto parts and component suppliers will change as vehicles become safer, 4. IIHS Status Report, Vol. 48, No. 3, April and adjacent industries have their own perhaps owned less by individual fam- 25, 2013 opportunities and risks. Chip makers ilies and ultimately are fully automated.

Ecosystem Implications of Driver-Assisted and Autonomous Vehicles

Action Industry Time Sector Opportunity Risk Frame

• Vehicle OEMs High High Now • Traditional OEM suppliers Medium Medium Now • Tech OEM suppliers High Low Now • Motor Insurance Carriers Low High Now • Telecom Carriers Medium Low Now • Telecom platform providers High Medium Now • Security solutions High Low Now • Transportation agencies Medium High Soon • Auto – Repair/body shop/gas Low High Later • Auto dealerships Medium High Now • Big-data analytics High Low Now

Figure 6 – ADAS and autonomy will have a major impact on many ancillary industries besides just automotive.

Third Quarter 2015 Xcell Journal 45 XCELLENCE IN DATA CENTER CLOUD COMPUTING Machine Learning in the Cloud: Deep Neural Networks on FPGAs by Nagesh Gupta Founder and CEO Auviz Systems [email protected]

46 Xcell Journal Third Quarter 2015 XCELLENCE IN DATA CENTER CLOUD COMPUTING

Because of their attractive performance/power metrics, Xilinx FPGAs are a top choice for designers building convolutional neural networks. New software tools ease the implementation job. Artificial intelligence is undergoing a revolution thanks to the rapid advances in machine learning. AWithin the field of machine learning, a class of algorithms called “deep learning” is generating a lot of interest because of its excellent performance in large data sets. In deep learning, the machine can learn a task from a large amount of data in either a supervised or an unsupervised manner. Large-scale supervised learning has been very successful in tasks such as image recognition and speech recognition. Deep-learning techniques use a large amount of known data to find a set of weights and bias values to match the expected results. The process is called training, and it can result in large models. This fact has motivated engineers to move toward specialized hardware such as GPUs for training and classification purposes. As the amount of data increases even further, machine learning will move to the cloud, where large machine-learning models would be implemented on CPUs. While GPUs are a better alternative in terms of performance for deep-learning algorithms, the prohibitive power requirements have limited their use to high-performance computing clusters. Therefore, there is a dire need for a processing platform that can accelerate algorithms without a substantial increase in power consumption. In this context, FPGAs seem to be an ideal choice, with their inherent capability to facilitate the launching of a large number of concurrent processes at a low power profile. Let’s take a closer look at how to implement a convolutional neural network (CNN) on a Xil- inx® FPGA. CNN is a class of deep neural networks that has been very successful for large- scale image-recognition tasks and other, similar machine-learning problems. In the current scenario, a feasibility study of implementing CNN on

Third Quarter 2015 Xcell Journal 47 XCELLENCE IN DATA CENTER CLOUD COMPUTING

Image recognition, speech recognition and natural-language processing are a few popular applications of the CNNs.

FPGAs can serve as an indicator as to layers (Figure 1). Each convolution convolution operation from pixel loca- whether FPGAs are fit for large-scale layer convolves the set of input feature tion (x,y) at the input feature map n. machine-learning problems. maps with a set of weight filters, re- The activation function used is a sulting in a set of output feature maps. rectified linear unit, which performs WHAT IS A CONVOLUTIONAL The dense layers are fully connected the function Max(x,0). The activation NEURAL NETWORK? layers, where every output is a func- function introduces nonlinearity in the Convolutional neural networks are a tion of all the inputs. transfer function of the network. Max form of deep neural networks (DNNs) pooling is the subsampling technique that engineers have recently begun us- CONVOLUTION LAYER used in AlexNet. Using this technique, ing for various recognition tasks. Image A convolution layer in AlexNet does only the maximum values in the local recognition, speech recognition and three major jobs, as shown in Figure neighborhood of a pixel are selected to natural-language processing are a few 2: 3D convolutions, activation function propagate to the next layer. popular applications of the CNNs. using a rectified linear unit (ReLu) and In 2012, Alex Krishevsky and oth- subsampling (max pooling). Three-di- DEFINING DENSE LAYERS ers [1] from the University of Toronto mensional convolutions are represent- A dense layer in AlexNet corresponds to proposed a deep architecture based on ed by the following equation: a fully connected layer, wherein every in- CNNs that won that year’s Imagenet Large Scale Visual Recognition Chal- lenge. Their model achieved a substantial improvement in recognition compared with its competitors or with models from previous years. Since then, AlexNet has become the bench- where Y(m,x,y) is the output of convo- put node is connected to each of the out- mark for comparison in all image-rec- lution at location (x,y) for output fea- put nodes. The first dense layer in Alex- ognition tasks. ture map m, S is the local neighborhood Net has 9,216 input nodes. This vector is AlexNet consists of five convolu- around (x,y), W is the set of convolution multiplied with a weight matrix to create tion layers followed by three dense filters and X(n,x,y) is the input to the output in 4,096 output nodes. In the next N Y (m, x, y) = ∑ ∑ W (m, n, ∆x, ∆y)X(n, x – ∆x, y – ∆y) n=1 (∆x,∆y) S

dense dense dense

Input Image 1000 (RGB) Max pooling 4096 4096 Max Max pooling Stride pooling of 4

Figure 1 – AlexNet, an image-recognition benchmark, consists of five convolution layers (blue boxes) and three dense layers (yellow).

48 Xcell Journal Third Quarter 2015 XCELLENCE IN DATA CENTER CLOUD COMPUTING

Figure 2 – A convolution layer in AlexNet does 3D convolutions, activation and subsampling.

layer, this 4,096-node vector is multiplied sign environments such as Xilinx’s SD- Here, an 11 x 11 weight matrix is con- with another weight matrix to create Accel™ to launch kernels on an FPGA. volved in parallel with an 11 x 11 input 4,096 outputs. Finally, these 4,096 out- The simplest approach is to imple- feature map to create one output value. puts are used to create probabilities for ment the convolutions and the vec- This process involves 121 parallel mul- 1,000 classes using softmax regression. tor-matrix operation in a sequential tiply-accumulate operations. Depending manner. Given the number of computa- on the FPGA resources available, we IMPLEMENTING CNN ON AN FPGA tions involved, sequential computations could convolve 512 or even 768 values With the advent of newer advanced de- will create significant latency. in parallel. sign environments, it has become easi- The main reason for the very high To further increase the throughput, er for software developers to port their latency of a sequential implementation we can pipeline the implementation. designs to Xilinx FPGAs. The software is the sheer number of computations Pipelining enables higher throughput developer can exploit the inherent ar- involved in a CNN. Figure 3 shows the for operations that take more than chitectural advantages of an FPGA by number of computations and the data one cycle to complete, such as float- calling functions from C/C++ code. Li- transfers for each layer in AlexNet to ing-point multiply and add. With pipe- braries from Auviz Systems, such as Au- illustrate the complexity. lining, the latency increases for the first vizDNN, provide optimized functions Therefore, it is essential to com- output very slightly, but we can obtain for the user to create custom CNNs pute in parallel. There are many ways an output every cycle. for a variety of applications. These to parallelize the implementation. One A complete implementation of CNNs functions can be called from within de- such example is illustrated in Figure 6. on the FPGA using AuvizDNN just looks

1.E+09 1.E+08 1.E+07 1.E+06 1.E+05 1.E+04 1.E+03 1.E+02 Computations 1.E+01 Data transfers 1.E+00

Output Dense 1 Dense 2

Convolution 1Convolution 2Convolution 3Convolution 4Convolution 5

Figure 3 – Chart measures computation complexity and the number of data transfers involved in AlexNet.

Third Quarter 2015 Xcell Journal 49 XCELLENCE IN DATA CENTER CLOUD COMPUTING

text of data centers is performance per watt. Data centers require high performance but at a power profile within the limits of data center server requirements. FPGAs, such as Xilinx’s Kintex® Ul- traScale™ devices, can provide higher than 14 images per second per watt, making it a great choice for data center applications. Figure 6 will give you an idea of the achievable performance with different classes of FPGAs. Figure 4 — Here is the sequence of function calls to implement a CNN. IT ALL STARTS WITH C/C++ Convolutional neural networks are like a sequence of function calls from a first layer in AlexNet can be created. becoming increasingly popular and C/C++ program. After setting up the ob- AuvizDNN provides configurable are being deployed at a large scale for jects and data containers, function calls functions with which you can create any tasks such as image recognition, natural are made to create each of the convo- type and configuration of CNN. AlexNet speech processing and many others. As lution layers, followed by the dense is used just as an illustration. The CNN CNNs migrate from high-performance layers and finally the softmax layer, as implementation is loaded into the FPGA computing applications (HPC) to the shown in Figure 4. as a complete bitstream and called from data center, efficient methods are re- AuvizDNN, a library of functions to the C/C++ program, allowing develop- quired to implement them. implement CNNs on FPGAs from Au- ers to use AuvizDNN without running FPGAs can be used to implement viz Systems, provides all the required the implementation software. CNNs very efficiently. FPGAs also pro- objects, classes and functions to imple- FPGAs have a large number of lookup vide a very good performance/watt met- ment CNNs easily. The users just need tables (LUTs), DSP blocks and on-chip ric, and so are suitable for the data center. to supply the required parameters to memory, which make them a good choice AuvizDNN provides a library of func- create different layers. For example, the to implement very deep CNNs. More im- tions to implement CNNs on FPGAs. code snippet in Figure 5 shows how the portant than raw performance in the con- AuvizDNN masks the complexity of us-

/*********** calling convolution layer functions***********/

OclConv.loadkernel(convolutionLayer_xclbin);

// Layer 1 poolDesc.createPoolingDesc(pool_mode,window_size,pool_stride);

tensorØ.createTensor(datatype,b,3,224,224);

tensor1.createTensor(datatype,b,96,55,55);

conv_filter1.createConvFilter(datatype,96,3,11,11): tensorØ.moveTensorData(contex, input_data, HOST2DEVICE); conv_filter1.moveConvFilter(context, con1_data, HOST2DEVICE); bias1.createBiasVector(datatype, 96); bias1.moveBiasVector(context, bias1_data, HOST2DEVICE); poolDesc.movePoolingDesc(context, HOST2DEVICE); clFinish(); OclConv.convolutionForward(tensorØ,con_filter1,tensor1,bias1,NULL,RELU, 4);

Figure 5 – Code snippet for creating Layer 1 of AlexNet using AuvizDNN.

50 Xcell Journal Third Quarter 2015 XCELLENCE IN DATA CENTER CLOUD COMPUTING

ing an FPGA, and provides simple func- ferent than writing C/C++ with calls to REFERENCE tions that users can call from their C/ functions within the AuvizDNN library. 1. A. Krizhevsky, I. Sutskever, G. E. Hinton, C++ program to get the acceleration on For further information, visit www. “ImageNet Classification with Deep Convolu- the FPGA. With AuvizDNN, getting the auvizsystems.com or e-mail sales@au- tional Neural Networks,” Advances in Neural acceleration of FPGAs is not any dif- vizsystems.com. Information Processing Systems, 2012

Xilinx Kintex Ultrascale XCKU115 class FPGAs

300 768 Multiply & Adds

250 Xilinx Virtex 7 XC7VX690T class FPGAs

512 Multiply & Adds 200

150 256 Multiply & Adds Images/sec 100

0 167 200 250 300 167 200 250 300 167 200 250 300 Frequency of operation, MHz

Figure 6 – Performance of AlexNets will vary depending on the class of FPGA chosen.

TigerFMC

200 Gbps bandwidth The highest VITA 57 compliant optical bandwidth for FPGA carrier

www.techway.eu

Third Quarter 2015 Xcell Journal 51 XCELLENCE IN SDN/NFV All Programmable SDN Switch Speeds Network Function Virtualization

by David Levi Chief Executive Officer Ethernity Networks Ltd. [email protected] A programmable COTS NIC based on a Xilinx FPGA accelerates the performance of NFV software applications by 50x.

52 Xcell Journal Third Quarter 2015 XCELLENCE IN SDN/NFV

In the last five years, operators, need to be reprogrammable to optimize academics and vendor upstarts have overall system performance. This allows been calling for a move to ubiquitous vendors and operators to leverage NFV hardware, network neutrality, open and SDN in a “work smarter, not hard- systems and software compatibility er” manner to meet growing network by maximizing hardware and software demands of the operators’ end custom- programmability. NFV and SDN are at ers—consumers. A truly hardware- and the vanguard of this trend, carrying software-programmable infrastructure the banner for this growing and sure- is the only way to truly realize the vision to-succeed revolution. of NFV and SDN. With NFV, companies run a variety SDN is a modern approach to net- of network functions in software on working that eliminates the complex commodity, general-purpose hardware and static nature of legacy distribut- The shift toward network function vir- platforms, as opposed to running each ed network architectures through the tualization (NFV) and software-defined specialized network task on custom- use of a standards-based software ab- Tnetworks (SDN) represents the most ized and expensive proprietary hard- straction between the network control transformative architectural network ware. Maximizing programmability on plane and underlying data-forward- trend in nearly 20 years. With their these open, ubiquitous platforms en- ing plane in both physical and virtu- promise of open systems and network ables companies to run in data centers, al devices. Over the last five years, neutrality, NFV and SDN stand poised or even in smaller networked nodes, the industry has developed a stan- to make a far-reaching impact in shap- many tasks heretofore performed by dards-based data plane abstraction ing the communications networks and specialized hardware devices. NFV fur- called OpenFlow that provides a novel businesses of tomorrow. ther aims to reduce the time it takes to and practical approach to dynamically We at Ethernity Networks are lever- establish a new network service by al- provisioning the network fabric from a aging Xilinx® devices to bring truly open lowing operators to simply upload new centralized software-based controller. and highly programmable SDN and NFV network software of a given service to An open SDN platform with cen- solutions to the market sooner. Let’s the commodity hardware resources as tralized software provisioning delivers take a look at how Ethernity crafted its needed. This allows operators to scale dramatic improvements in the network solution, after first exploring the prom- networks easily and choose best-in- agility via programmability and auto- ise and requirements of NFV and SDN. class functionality for their businesses mation, while substantially reducing instead of being forced to purchase and the cost of the network operations. An UBIQUITOUS HARDWARE AND operate new proprietary hardware that industry-standard data plane abstrac- THE NFV/SDN REVOLUTION affords limited software flexibility. tion protocol like OpenFlow gives The network infrastructure business of NFV can be effective because many providers the freedom to use any type the last few decades has largely been a nodes in a network share common func- and brand of data plane device, since continuance of the mainframe business tionality requirements. Those nodes all the underlying network hardware model in which a handful of large cor- with common requirements include is addressable through a common ab- porations offer proprietary infrastruc- switching and routing, classification of straction protocol. Importantly, Open- ture equipment that runs proprietary millions of flows, access control lists Flow facilitates the use of “bare-metal software, all purposefully built not to (ACL), stateful flow awareness, deep switches” and eliminates traditional communicate with systems from com- packet inspection (DPI), tunneling vendor lock-in, giving operators the petitors. In most cases, infrastructure gateway, traffic analysis, performance same freedom of choice in networking vendors create custom hardware for monitoring, fragmentation, security, vir- as can be found today in other areas of each node of their customer’s network, tual routing and switching. NFV has its IT infrastructure, such as servers. and they build each node to be mini- challenges, however. With exponential Because SDN is in its infancy, stan- mally programmable and upgradable, growth expected in Internet and data dards are still in flux. This means that ensuring that customers wanting to ex- center traffic in the coming years, net- equipment vendors and operators pand or upgrade their networks would work infrastructure equipment must be need to hedge their bets and design need to buy next-generation equipment able to handle vast increases in traffic. and operate SDN equipment with from the same vendor or make the futile Software programmability alone won’t maximum flexibility, leveraging the choice of buying an entirely new net- be enough to enable generic hardware hardware and software programma- work from another company but run- to scale easily with growing bandwidth bility of FPGAs. FPGA-based SDN ning into the same consequences. demands. The ubiquitous hardware will equipment entering the market today

Third Quarter 2015 Xcell Journal 53 XCELLENCE IN SDN/NFV

is quite affordable even for mass de- acceleration to increase network per- (VM) based on tag information such as ployment. It offers the highest degree formance. The specification describes IP, MAC or VLAN. The packet is then of hardware and software flexibili- how a processor component can off- DMA’ed directly to the right VM locat- ty and maximizes compliance with load certain functions to a network ed at the server for processing. Each OpenFlow. interface card (NIC) to support cer- virtual networking function (VNF) tain acceleration functions, including runs on a different VM, and certain THE NEED FOR PERFORMANCE TCP segmentation, Internet Protocol networking functions require the use ACCELERATION (IP) fragmentation, DPI, filtering of of multiple or even tens of VMs. Perhaps the most critical requirement millions of entries, encryption, perfor- OpenFlow controls the hardware for both NFV and SDN above and be- mance monitoring/counting, protocol acceleration functions, such that these yond openness is high performance. interworking and OAM, among other hardware acceleration functions locat- While NFV hardware will seemingly be acceleration capabilities. ed at the NIC can be viewed as an ex- less expensive than proprietary systems, The main engine driving this accel- tension of the SDN switch. NFV architectures will need to sustain eration is the NIC, which is equipped NFV performance can be addressed competitively high data volumes, meet with physical Ethernet interfaces to for many functions by deploying multi- complex processing requirements for enable server connectivity to the net- ple VNFs on multiple VMs and/or cores. next-generation networking and be in- work. As described in Figure 1, when This raises two main performance chal- creasingly energy efficient. a packet arrives the NIC, over 10GE, lenges for NFV. The first challenge is In fact, the NFV Infrastructure 40GE or 100GE ports, it is placed in at the “vSwitch,” which is typically a Group Specification includes a spe- a virtual port (VP) or queue that rep- piece of software that processes net- cial section describing the need for resents a specific virtual machine work traffic between the Ethernet NIC and the virtual machines. The second performance challenge is balancing incoming 40/100GE data among multiple Server/CPU VMs. When adding IP fragmentation, TCP segmentation, encryption or oth- VirtualVirtual MachineMachine VirtualVirtual MachineMachine er dedicated hardware functions, NFV software requires assistance to meet VNF1 VNFn the performance needs and reduce the power. Ideally, it should be compact to reduce the amount of real estate required to house the network equipment. To address NFV challenges and the variety of networking functions, NIC cards for NFV and SDN must be ex- Control Data tremely high performance but also as flexible as possible. OpenFlow In an attempt to be first to market with NFV hardware, several chip vendors have proposed platforms for NIC cards, each with some degree of programmability. Intel today is the main VPVP VPn NIC component provider, equipped with its DPDK package for packet-process- VNF ing acceleration. EZchip offers its NPS Acceleration Data in/out multithreaded CPU running Linux and Physical NIC programmed in C. Marvell offers two all-inclusive data plane software suites for its Xelerated processor for both metro Ethernet and the Unified Fiber

Figure 1 – When a packet arrives, the NIC enters a virtual port (VP) Access Application, which consist of that represents a specific virtual machine. The packet is then sent through DMA an application package running on the to the right virtual machine at the server for processing. NPU and a control-plane API running

54 Xcell Journal Third Quarter 2015 XCELLENCE IN SDN/NFV

development environments to enable Network Server OpenCL™ and C++ design entry and program acceleration, further open- Classiﬁcation ing up NFV equipment design to a Search greater number of users. To accelerate NFV performance, NFV Inbound Packet Onbound Packet solution providers increase the number of VMs with a goal of distributing the VNFs on multiple VMs. When operating Destination Timing Source with multiple VMs, new challenges arise selection selection related to balancing the traffic load between the virtual machines while sup- Hierarchical porting IP fragments. In addition, challenges also exist in supporting switching CPU load Traf c Manager between VMs and between VMs and the Monitoring NIC. A pure software-based vSwitch el- ement simply doesn’t provide adequate Header Manipulation performance to address these challenges. The integrity of the VMs must also be maintained so that the VMs store specific bursts adequately and do not deliver packets out of order. Figure 2 – This high-level block diagram shows the virtual machine’s load balancing and switch. Focusing on solving the performance issues for NFV, Ethernity’s ENET FPGA on a host CPU. Cavium has opted for a have published papers describing firmware is equipped with a virtual more generic software development kit the dramatic performance increases switch/router implementation that en- for its Octeon family. Broadcom, Intel they’ve gained with a hybrid architec- ables a system to accelerate vSwitch and Marvel L2/L3 switches are mainly ture. In the case of Microsoft, a white functionality to switch data based on used for search and vSwitch offload. paper titled “The Catapult Project” L2, L3 and L4 tags, while maintaining a Meanwhile, Netronome’s new Flow- cites a 95 percent performance in- dedicated virtual port for each VM. If a NIC is equipped with software that runs crease at a cost of only a 10 percent specific VM is not available, the ENET on that company’s specialized network power increase. Intel cited the potency can store up to 100 milliseconds of processor hardware. of this combination of FPGA-plus-CPU traffic; then, upon availability, it will re- While all of these offerings are in data center NICs as the primary rea- lease the data through DMA to the VM. claiming to be open NFV approaches, son for its $16.7 billion acquisition of Equipped with delay measurement ca- they really aren’t. All of the approaches the No. 2 player in FPGAs, Altera Corp. pabilities through an implementation of involve rigid and, arguably, too restric- The same hybrid CPU-FPGA ap- a standard CFM packet generator and a tive hardware implementations that are proach is applicable for NFV, which packet analyzer, our ENET can measure only programmable in software and runs virtual networking functions on the availability and health of a VM and rely once again on rigid, proprietary virtual machines. In this approach, the instruct the ENET’s stateful load balanc- hardware implementations in SoCs or FPGA serves as a complete programma- er regarding the availability of each VM standard processors. ble NIC that can be extended to accel- for load distribution. The packet reor- erate the virtual-network functions that der engine can maintain the order of the ALL PROGRAMMABLE ETHERNITY run on the server’s CPUs/VMs. frame if, for example, a packet moves NIC FOR NFV PERFORMANCE But a NIC card based entirely on out of order, which can result in the use ACCELERATION FPGAs is the ideal COTS hardware of multiple VMs for one function. To add programmability while also architecture for NFV. Multiple FPGA Figure 2 depicts a block diagram of the improving performance dramatical- firmware vendors can provide firm- the VM load-balancing ENET solution. ly, many companies are examining a ware for NFV performance accelera- In Figure 2, the classification block hybrid approach that pairs an off-the- tion running on the FPGA NIC. And performs hierarchical classification for shelf CPU with an FPGA. Over the last FPGA companies have recently de- L2, L3 and L4 fields to maintain a route two years, a number of data center veloped C compiler technologies such for connection and flow supporting the operators—most notably, Microsoft— as Xilinx’s SDAccel™ and SDSoC™ long-lived TCP (telnet, FTP and more)

Third Quarter 2015 Xcell Journal 55 XCELLENCE IN SDN/NFV

that doesn’t immediately close. The selection block’s load balancer as- VM load-monitoring block maintains load balancer must ensure that mul- signs a destination address from the information on the availability of tiple data packets carried across that available VM according to the Weight- each CPU/VM, using metrics such as connection do not get load-balanced ed Round Robin (WRR) technique. Ethernet CFM Delay Measurements to other available service hosts. ENET The WRR is configured based on the Message (DMM) protocol generation includes an aging-mechanism feature to information derived from the VM toward a VM. By time-stamping each delete nonactive flows. load-monitoring block. sent packet and measuring the delta In the classification block, we con- The hierarchical traffic manager time between send and receive, the figured the balancing hash algorithm block implements hierarchical WRR block can determine the availability based on L2, L3 and L4 fields. The algo- between an available VM and maintains of each VM and, based on that, in- rithm includes fragmentation such that an output virtual port for each VM to struct the destination selection block the load balancer is able to perform include three scheduling hierarchies on the available VMs. balancing based on inner tunnel infor- based on priority, VM and physical port. The Source Selection block deter- mation (such as VXLAN or NVGRE), The CPU hierarchy represents a cer- mines what outbound traffic sent from while an IP fragment connection can tain VM, and the priority hierarchy may the host to the user will be classified and be handled by a specific connection/ assign weights between different ser- determines the source of that packet. CPU. For VM-to-VM connection, the vices/flows that are under the service of The Header Manipulation block in classifier and search engine will for- a specific VM. Operating with external ENET will perform network address ward the session to the destination DDR3, the ENET can support buffering translation (NAT) to replace the incom- VM instead of the vSwitch software. of 100 ms to overcome the momentary ing address with the right VM IP address Meanwhile, a classifier feature assigns load of a specific VM. to enable the NIC to forward the flow, a header manipulation rule for each The VM load monitoring uses the packet or service to the right VM. For incoming flow based on its outgoing ENET Programmable Packet Gener- outbound traffic, the NAT will perform route, with eyes on modifying the IP ator and Packet Analyzer for carrier the reverse action and will send out address or offloading protocols. Ethernet service monitoring, which the packet to the user with its original For each new flow, the destination complies with Y.1731 and 802.1ag. The IP address. The Header Manipulation

10GE 10GE 10GE XFI XFI XFI

12G/6.6G 12G/6.6G 12G/6.6G 12G/6.6G Packet SERDES SERDES SERDES SERDES 1. .4 Memory DDR3 DDR 3 Controller Packet Xilinx’s 1. .4 Memory Kintex 325 DDR3 DDR 3 Controller

0. . .2 DDR3 DDR 3

Controller Search Tables 12G/6.6G 12G/6.6G 12G/6.6G 12G/6.6G SERDES SERDES SERDES SERDES

PCIE – GEN3

PCIe Gen 3 connector

Figure 3 – A Xilinx Kintex FPGA is at the heart of Ethernity’s NFV network interface card.

56 Xcell Journal Third Quarter 2015 XCELLENCE IN SDN/NFV

block also performs tunnel encapsu- platforms by 50 times, dramatically re- The ENET SoC Carrier Ethernet lation. Here, the Header Manipulation ducing the end-to-end latency associated Switch is an MEF-compliant L2, L3 block will execute the action rules as- with NFV platforms. The new ACE-NIC and L4 switch/router that can switch signed by the classifier from the classifi- is equipped with four 10GE ports, along and route frames with five levels of cation, and will strip out tunnel headers with software and hardware designed packet headers, between 16,000 inter- or other headers to or from the CPU op- for an FPGA SoC based on Ethernity’s nal virtual ports distributed over 128 eration. In the reverse direction, it will ENET flow processor, supporting PCIe® physical channels. It includes support append the original tunnel toward the Gen3. The ACE-NIC is further equipped for FE, GbE and 10GbE Ethernet ports, outgoing user port. with onboard DDR3 connected to the and four levels of traffic-management As the number of subscribers to FPGA SoC, to support 100-ms buffering scheduling hierarchy. With its inher- an operator’s network increases, the and search for a million entries. ent architecture to support fragment size of the flow tables can quickly The Ethernity ENET Flow Processor frames, the ENET can perform IP frag- grow to exceed the cache capacity of SoC platform uses a patented, unique mentation and reordering of functions standard servers. This is especially flow-based processing engine to process with technology of zero copy, such that the case with current OpenFlow sys- any data unit in variable sizes, offer- segmentation-and-reassembly does not tems, which have table entries that ing multiprotocol interworking, traffic require dedicated store and forward. require 40 different fields, IPv6 ad- management, switching, routing, frag- Furthermore, ENET has an integrated dresses, Multiprotocol Label Switch- mentation, time-stamping and network programmable packet generator and ing (MPLS) and Provider Backbone processing. The platform supports up packet analyzer to ease CFM/OAM oper- Bridges (PBB). The ENET search en- to 80 Gbps on a Xilinx 28-nanometer ation. Finally, the ENET can operate in gine and parser can support classifi- Kintex-7XC7K325T FPGA, or higher 3GPP, LTE, mobile backhaul and broad- cation on multiple fields and serve throughput on larger FPGAs. band access. It supports interworking millions of flows, thus offloading the The ACE-NIC comes with basic fea- between multiple protocols, all with ze- classification and search function tures such as per-frame time-stamping ro-copy operation and without a need to from the software appliance. that’s accurate within nanoseconds, a reroute frames for header manipulation. And finally, with the ENET packet packet generator, a packet analyzer, Clearly, the communications indus- header manipulation engine, the ENET 100-ms buffering, frame filtering and try is at the beginning of a new era. We can offload any protocol handling and load balancing between VMs. To serve are sure to see many new innovations in provide raw data info to the VM togeth- multiple cloud appliances, it also has NFV and SDN. Any emerging solution er with TCP segmentation, or inter- the ability to assign a virtual port per for NFV performance acceleration or working between various protocols, in- virtual machine. an SDN switch must have the ability to cluding 3GPP protocols for virtual EPC Furthermore, the ACE-NIC comes accommodate support for new versions (vEPC) implementation, XVLAN, MPLS, with dedicated acceleration functions of SDN. With Intel’s acquisition of Altera PBB, NAT/PAT and so on. for NFV vEPC. They include frame head- and the increasing number of hardware In addition to the firmware, Etherni- er manipulation and offloading, 16K vir- architectures seeking greater degrees ty has also developed an NFV NIC that tual-ports switch implementation, proof programmability, we will certainly we call the ACE-NIC (Figure 3). To cre- grammable frame fragmentation, QoS, see a growing number of hybrid proces- ate the NIC, we integrated our ENET counters and billing information, which sor-plus-FPGA architectures along with SoC firmware (already deployed in hun- can be controlled by OpenFlow for the new, innovative ways to implement NFV dreds of thousands of systems in carri- vEPC. With its unique hardware and performance acceleration. er Ethernet networks) into a single Xil- software design, the ACE-NIC acceler- FPGA-based NFV NIC acceleration inx Kintex®-7 FPGA. We also integrated ates software performance by 50x. can provide the flexibility of NFV based into the same FPGA the functionality of on general-purpose processors while at five discrete components: NIC and SR- THE ALL PROGRAMMABLE the same time supplying the necessary IOV support; network processing (in- ETHERNITY SDN SWITCH throughput that the GPP cannot sustain, cluding classification, load balancing, Similarly, Ethernity integrated the while performing certain network func- packet modification, switching, rout- ENET SoC firmware in an FPGA to tion acceleration that GPP can’t support. ing and OAM); 100-ms buffering; frame create an All Programmable SDN By efficiently combining the SDN and fragmentation; and encryption. switch, with support for OpenFlow the NFV in the FPGA platform, we can The ACE-NIC is an OpenFlow-enabled version 1.4 and complete carrier Eth- achieve the design of All Programmable hardware acceleration NIC, operated in ernet switch functionality, accelerat- network devices fostering the innova- COTS servers. The ACE-NIC accelerates ing time-to-market for white-box SDN tion to a new ecosystem for IP vendors performance of vEPC and vCPE NFV switch deployment. in network applications.

Third Quarter 2015 Xcell Journal 57 XCELLENCE IN SOFTWARE-DEFINED NETWORKING Xilinx FPGAs Serve Performance SDN Corsa Technology designed and sold its first software-defined networking switch in less than six months, leveraging the flexibility and reprogrammability of FPGAs.

58 Xcell Journal Third Quarter 2015 XCELLENCE IN SOFTWARE-DEFINED NETWORKING

ome people might argue the same lean hardware can function that software-defined net- as various elements in your network. by Yatish Kumar working (SDN) has re- SDN done right allows you to move Chief Technology Officer ceived more attention than away from local, rigid, fixed, com- Corsa Technology it has deserved in terms of plicated, proprietary hardware and [email protected] concrete results. In the ear- software. Corsa’s performance SDN, Xilinx FPGAs Serve Sly days of SDN, deployments emerged within the simple design pattern, from the work done at leading insti- moves you toward the real promise of tutions in conjunction with hardware software-defined networking with a companies that quickly customized flexible, high-performance hardware their existing non-SDN firmware. While platform that has the ability to scale. Performance SDN these efforts validated the theory of SDN, there remains a big difference be- HARDWARE DESIGN tween SDN for a proof of concept and UNDER PRESSURE SDN done right for a globally orches- This flexible SDN networking concept trated production network. has a direct impact on how network At Corsa Technology, we developed hardware design has to change. Be- a vision of SDN in conjunction with cause SDN network architecture can network architects and operators. change so rapidly thanks to new inno- Over and over, they told us that SDN vations, time-to-market for SDN hard- done right means your network archi- ware solutions becomes even more tecture alters and adapts in real time critical than ever. to traffic patterns and user demands. Hardware platforms are a combi- This flexibility delivers huge increas- nation of system design, board-level es in performance at a fraction of the design, mechanical design and SoC traditional cost. selection or design. Typically, in a new With this idea as the guiding prin- and emerging market like SDN, the SoC ciple, Corsa defined SDN as a simple is not available as merchant silicon and design pattern. Many others share the the hardware solution needs to either base concept: Separate the software take the ASIC, NPU or FPGA path. For from the hardware, communicate via SDN, given its pace of network change, an open interface, give the software the decision wasn’t too difficult for us. all the control (brains) and make the It takes three years to design, build hardware (brawns) as high in perfor- and implement networking hardware mance as possible. But at Corsa we with custom ASICs: six months for went deeper and took a really hard hardware selection and architecture, look at what the new world order of a year of ASIC design, four months for networking needs in terms of perfor- board design and fabrication, and 12 mance hardware (Figure 1). months for software integration and We came away with a definition of testing. And that’s if you get it right on hardware that delivers on network ar- the first spin. chitects’ vision of SDN. We call it lean This time-to-first-prototype was an hardware—sized right for deployment, unacceptable choice for Corsa. very high in performance while being On the other hand, network pro- very flexible and scalable to meet even cessing units (NPUs) are program- the largest network’s traffic volumes. mable merchant silicon designed spe- Why buy a big, expensive hunk of met- cifically for networking applications. al when you only need about 10 per- Although they do offer flexibility and cent of the functionality? Instead, if the can be reprogrammed, they have lim- hardware is flexible and programmable ited bandwidth, which is a barrier for enough, you can bend and adapt it to at-scale switching functions. They meet the specific network need. Wheth- also come with complex, proprietary er at the WAN edge or the campus edge, programming models that make it

Third Quarter 2015 Xcell Journal 59 XCELLENCE IN SOFTWARE-DEFINED NETWORKING

difficult to make changes. Since SDN budgeted capacity for the full feature requirements, but not having detailed needs full flexibility and performance set. It is not necessary to fully design customer engagement until after the at scale, we likewise ruled out NPUs. the entire architecture up front and product arrives in the lab. To meet SDN time-to-market de- then move to RTL, as you would with mands with the right solution, Corsa se- an ASIC- or NPU-based approach. As SDN SWITCHING DESIGN lected FPGAs and developed a solution a result, we could have working code SDN represents a significant depar- in six months leveraging the flexibility developed in parallel with the system ture from conventional notions about of Xilinx® Virtex®-7 devices. and in the hands of lead customers how network equipment should be Designing with FPGAs, we could much more quickly. built. A key requirement for SDN is do the following activities in parallel Not every use case, or application, that reprogrammable hardware can be (see Figure 2): needs every feature. By leveraging the built and sold competitively against hardware-level programmability of traditional fixed-function hardware. • System architecture (four months) the FPGAs, we could create smaller By exploiting this notion, SDN pro- • RTL coding (six months) RTL implementations that matched vides disruptive change in the way networking rolls out. The tenets of • Software design (six months) the feature bundles and performance needed for a particular use case. conventional networking design can • PCB design and fabrication During design and even today, it’s now be significantly improved. (four months) possible to substitute 10G and 100G Here are the three major factors fuel- ing interest in SDN. Of singular importance was the fact MACs, shift resources from switching fabrics to classification engines, and that we could make RTL changes on an 1. Velocity of new networking add or remove hardware acceleration FPGA platform on the fly while the vari- protocols to solve new networking for particular protocols. This flexibil- ous design activities progressed and were problems optimized for performance and scale. ity leads to a physically smaller footprint in terms of gate count, when A new network protocol takes at least THE BENEFITS OF compared with ASICs or NPUs. It also three years to make its way through INCREMENTAL DESIGN allows us to react to unforeseen use the standards process. It takes another We developed our system architec- cases that inevitably come up once two to three years to be implemented ture in slices based on an array of lead customers have been engaged. in hardware before it finally gets de- FPGAs. This approach made it possi- Sequential design often leads to the ployed. With SDN, new protocols are ble to develop a single slice with min- chicken-and-egg definition problem implemented in software and deployed imum viable features, while leaving of fully specifying the ASIC or NPU almost immediately in installed sys-

APPLICATION LAYER

Business Applications

API API API SOFTWARE

CONTROL LAYER Network Services Network Services

INFRASTRUCTURE HARDWARE LAYER

Figure 1 – SDN separates the packet-forwarding data plane from the control plane.

60 Xcell Journal Third Quarter 2015 XCELLENCE IN SOFTWARE-DEFINED NETWORKING

tems. This reduces the five-year cycle equipment has a three- to five-year life are finally able to deliver on a name to less than a few months. cycle. Any protocols or features that are assigned to them in the early ’90s: 2. Meritocracy of new networking not present at the time of purchase will field-programmable gate arrays. ideas, based on open hardware often have to wait for three to five years FPGA technology is now so high in platforms before the equipment is refreshed. With performance, so flexible and so scalable SDN, it is most likely that new protocols that it’s ideally aligned with the list of Standards are often more political than can immediately be deployed on equip- SDN attributes that network architects technical. Various factions argue about ment that is in the field. Extending the need. There are some key areas to high- and modify proposals, and the final life cycle of equipment past five years is light where FPGA technology offers specification is a compromise repre- a reality, while at the same time provid- significant advantages for SDN, starting senting a superset of everyone’s posi- ing instant availability for the next new with the IP library, memory and I/Os. tions. Small players are often ignored thing to emerge. In terms of intellectual property (IP), or set aside in the process. With SDN, base networking functions have been anyone can develop a protocol and of- FPGA VS. ASIC implemented using standard cells in fer it for industry use. If operators see To be competitive, SDN switching FPGAs. Among them are large blocks, the benefit, the protocol will thrive; if needs performance, flexibility and including dozens of 10/100G Ethernet not, it will wither. This “survival of the scale, all wrapped up in an afford- MACs, PCIe® interfaces, Interlaken in- fittest” approach is a much more robust able package. Conventional wisdom terfaces, embedded ARM® cores and selection process for the best technical says that fixed-function ASICs are DDR3 interfaces. These cores offer the ideas to succeed. required in order to build such com- SDN switch designer a wealth of prede- 3. Infrastructure reuse via field petitive systems. This used to be true signed and pre-optimized blocks. upgrades for protocols that have prior to the 28-nanometer technology In networking devices, scale is criti- not yet been invented node. However, at 28nm and beyond, cal. One specific area that contributes to FPGAs have reached a disruptive scale is memory, and for packet switch- Each year billions of dollars get spent scale. They are no longer large PLD ing large amounts of small memory, on new networking equipment. This devices for glue logic. Instead, they structures are required. These memory

Typical Best-Case ASIC Design Cycle

6 months 12 months 3 months 3 months First Customer

System ASIC Design FAB LAB Design PCB

Software Design

Corsa FPGA Design Cycle

System Design

First Customer PCB LAB

Software

RTL

Figure 2 – Corsa’s FPGA-based design cycle is far shorter than the typical ASIC design cycle.

Third Quarter 2015 Xcell Journal 61 XCELLENCE IN SOFTWARE-DEFINED NETWORKING

structures provide the bandwidth and nents, power amplifiers and digital logic. optimal form, leaving 50 percent or less capacity to move a terabit or more of The die area devoted to I/Os can be exces- of the die area to be considered for the traffic into and out of the processing sive. FPGA technology has excellent I/O CLB vs. standard-cell debate. Given val- units. FPGA memory is optimized and blocks that are comparable to network- ue pricing in the relatively low-volume requires minimum die area so it is instru- ing ASICs for their die area consumption. (100k units is considered big) network- mental in enabling terabit routing scale. After adding up the above contribu- ing-ASIC business, any price differenc- In terms of I/Os, networking needs tions to die area, it is easy to see that es come out in the wash. tremendous numbers of serdes interfac- the base FPGA technology delivers at What this means for SDN is that all of es, each containing large analog compo- least half the complexity of an ASIC in a sudden we have a highly programmable platform that can be field programmed OpenFlow in order to support the kinds of systems Classiﬁcation that previously required million-dollar pipeline prior to the NREs and huge ASIC developments. This switch fabric is akin to inventing the printing press in an era when all books were written one at a time, with a feather and inkpot. OpenFlow Fabric CORSA’S PERFORMANCE SDN OpenFlow At Corsa, we recognized that there are two disruptive trends in the networking market. The first is the desire for High-performance switch programmable network elements and High-performance features like QoS, metering, the second is the emergence of FPGAs classiﬁcation happens here scheduling, queuing and as the replacement for fixed-function buffering happen after SFP+ OpenFlow chips. So we set about the task of designing the ideal SDN switch. The system architecture for such a device is shown in Figure 3. Figure 3 – The two main elements of a high-performance SDN switch are a capable packet-classification engine and a speedy switch fabric.

ﬂowmods

100G CFP/CFP4 100G Ingress mac OF 1.3+ Ingress Pipeline x 10G Virtual Output Q Data Path 100G SFP+ 10G QoS-Aware x1 PCIE DDR3 DDR3 Buffering DDR3 DDR3 DDR3 DDRDDR3

100G CFP/CFP4 100G mac x1 PCIEE x 10G Egress Pipeline SFP+ 10G

x1 PCIE DDR3 DDR3 DDR3 DDR3 packetIn/Out Ports DDR3 DDR3 Switch

CFP/CFP4 100G DDR3 Multi-table Pipeline x1 PCIE SFP+ x 10G

x1 PCIE DDR3 DDR3 DDR3 DDR3 DDR3 DDR3 Multicasting 100G CFP/CFP4 Group Tables x1 PCIE x 10G Multi-table Pipeline SFP+

x1 PCIE DDR3 DDR3 OpenFlow DDR3 DDR3 DDR3 DDR3 Data Plane

Figure 4 – Corsa’s high-bandwidth and high-capacity system architecture has an FPGA-based pipeline and switch fabric.

62 Xcell Journal Third Quarter 2015 XCELLENCE IN SOFTWARE-DEFINED NETWORKING

A high-performance SDN switch as IPv4 addresses, MAC address, tun- has two components. It has a very nel IDs, etc. for the OpenFlow pipeline. All Programmable capable packet-classification engine, This gave us a channel implementation FPGA and SoC modules which is a precursor to the switch of two FPGAs per pipeline, with six fabric. The classifier is defined in DDR3 DIMMs per pipeline. The pipe- 4 the OpenFlow specification as a se- line channels are tied together with a form x factor quence of match action tables, which custom switch fabric built with a fabric 5 inspect packet headers and make FPGA, and the control plane is tied into forwarding decisions based on the the packet-forwarding engines using a source and destination fields of the capable Xeon processor with PCIe 3.0 various protocols within the pack- connections (Figure 4). et. Once the forwarding decision This design provided us with a sea of has been made, the packet enters gates, tremendous storage bandwidth the second component, a high-speed and capacity, and very high-speed con- switch fabric capable of buffering trol plane connectivity. Using the flex- and switching a terabit of data. ibility of OpenFlow, Corsa has built rugged for harsh environments The bandwidth and capacity of the line-rate processing engines for Internet extended device life cycle buffers required for these data rates Protocol-scale routers, MPLS switches, have a profound impact on the physical 100-Gig firewalls and DPI load balanc- Available SoMs: architecture of the performance SDN ers, along with numerous other net- switch. These switches require 100 mil- working use cases, with absolutely no liseconds or more of packet buffering changes to the hardware architecture in order to maintain high throughput or performance compromises. It is with in the presence of traffic congestion at some satisfaction that we see the emer- high-volume aggregation points, such gence of network function virtualiza- as at the WAN or campus edge. For 640 tion (NFV) service chaining; network Platform Features Gbits of front-panel bandwidth, the fol- service headers and protocols are still • 4×5 cm compatible footprint lowing calculation applies: being sketched out. • up to 8 Gbit DDR3 SDRAM • 256 Mbit SPI Flash 640 Gbps * 0.1 second = 64 Gbits of SCALE, PERFORMANCE • Gigabit Ethernet packet buffer storage AND FLEXIBILITY • USB option For Corsa, this is where the use of Programmable networks are the way of FPGAs made a difference. The only the future. Network operators are seeing storage technology that achieves the the benefits in terms of service velocity, storage density needed for perfor- infrastructure reuse and their ability to mance SDN is DDR3 memory. In 28nm, manage complexity through DevOps. DDR3-1600 is the fastest memory. In Concurrent with the emerging need for Design Services order to write and then read every programmable network elements, FPGAs • Module customization packet at full line rate, we required are achieving a new level of performance • Carrier board customization 1.28 Tbits of memory bandwidth. A and scale. At Corsa, we recognized this • Custom project development single DDR3 DIMM module, after ac- intercept point and used FPGAs in our counting for access inefficiencies, is SDN hardware platform to achieve SDN capable of handling about 64 Gbits of scale, performance and flexibility. traffic. This implied that we needed As fixed-function vendors continue 10 DDR3 DIMM modules in order to down the multiyear path of waiting for provide the packet buffers for an Inter- the standards, spinning their ASICs and net-scale SDN switch. delaying their product availability, Cor- Since a single FPGA cannot host that sa is able to deploy these new protocols much RAM, we are immediately led to immediately in new systems we ship. a distributed architecture with roughly Better yet, we can upgrade systems that difference by design three DIMMs per FPGA. We then added shipped yesterday to protocols that will extra memory capacity and bandwidth be invented tomorrow thanks to the use www.trenz-electronic.de to hold packet-classification data such of Xilinx FPGAs.

Third Quarter 2015 Xcell Journal 63 XCELLENCE IN CYBERSECURITY Implementing Power-Fingerprinting Cybersecurity Using Zynq SoCs

by Carlos R. Aguayo Gonzalez Chief Technology Officer PFP Cybersecurity [email protected]

Michael Fawcett Chief Technology Officer iVeia [email protected]

Dan Isaacs Director, Connected Systems: Strategic Marketing and Business Planning Xilinx, Inc. [email protected]

64 Xcell Journal Third Quarter 2015 XCELLENCE IN CYBERSECURITY

PFP Cybersecurity and iVeia employ POWER FINGERPRINTING: A NOVEL AND EFFECITVE SECURITY APPROACH a novel ‘fingerprint’ approach in Xilinx’s PFP Cybersecurity set out to find a solution to this problem that would be non- Zynq SoC to secure Industrial Internet intrusive, could operate effectively with the existing installed base of equipment of Things systems. and would not necessitate any significant equipment or software upgrades. The company developed the PFP tech- he hyper-accelerated growth THE VULNERABILITY OF nology as a novel approach for integrity of “connected everything” RESOURCE-CONSTRAINED assessment. Just as a human fingerprint that is driving the Industri- HARDWARE PLATFORMS is a unique identifier for an individual, T al Internet of Things (IIoT) A significant number of systems con- the same idea works for a particular is not simply about connecting mul- trolling critical infrastructure have system or chip. PFP uses a physical side titudes of disparate devices. It’s also little to no cybersecurity provisions, channel (for example, power consump- about the data collected, analyzed and since standard industrial control tion) to obtain information about the acted upon across a broad range of ap- equipment uses embedded and re- internal execution status in a processor plications. Critical to the concept of the source-constrained platforms. Today, across the full execution stack and inde- IIoT is the security of the devices that this system-level vulnerability is be- pendent of the platform or application. collect, assimilate and transmit data to ing recognized as a serious threat to The PFP technology will identify the other locations. critical infrastructure. Where many “fingerprint” of what the system looks The supersonic growth of the “con- of these systems have legacy proces- like normally. When a fingerprint taken nected everything” idea introduces new sors, use unique hardware or cannot subsequently doesn’t match, it can be an vulnerabilities faster than companies support the performance degradation indicator that something is wrong. can implement security measures. One that typical cybersecurity measures Implementation is by means of an ex- of the most often overlooked vulnerabili- introduce, they are left open to intru- ternal monitor that’s physically separated ties—brought to worldwide attention by sion. As recently as November 2014, from the target processor and capable of the Stuxnet virus that took out Iran’s nu- investigators found that systems con- detecting, with extreme accuracy, when clear reactors in 2010—is resource-controlling U.S. power plants, electric a cyber-attack has compromised the tar- strained hardware platforms. grids, water treatment facilities and get. PFP is applicable to new and legacy PFP Cybersecurity is a technology com- oil/gas infrastructure were infected systems, is complementary to existing pany that has devised a unique approach with a virus. [1] cybersecurity solutions and does not re- to address the security problems present- Four years earlier, the Stuxnet quire the installation of any hardware or ed by resource-constrained hardware virus infected programmable log- software on the target. platforms and the growth of cybersecuri- ic controllers (PLCs) responsible PFP supports a variety of sensors to ty threats, even Stuxnet-like viruses. iVeia for operating nuclear centrifuges in capture side-channel signals and relies helped PFP Cybersecurity implement its Iran, resulting in the destruction of on compute-intensive signal-process- novel and highly effective algorithm-based the centrifuges. [2] A PLC is one of ing algorithms for feature extraction cybersecurity solution for IIoT applica- many examples of a platform that is and machine learning for classification. tions leveraging Xilinx®’s Zynq®-7000 All too rigid and thus highly susceptible Sensing side channels can be accom- Programmable SoC. The resulting designs to intrusion. A PLC is largely com- plished using a variety of approaches, are nearly an order of magnitude smaller posed of an embedded MPU that au- including AC or DC current, or elec- and lower in power than PFP’s PC-based tomates the control and monitoring tromagnetic (EM) sensors that pick up proof of concept. of industrial equipment. Companies the changes in the electric or magnetic Let’s take a closer look at the growing often network their PLC platforms fields around the target. PFP extracts security vulnerabilities of resource-con- but tend not to have the resources for unique discriminatory features from strained hardware platforms before ex- any kind of security monitoring or in- the captured signals, compares them ploring how our two companies used tegrity assessment. [3] Nor do they against a set of baseline references and a proprietary technology called Power update these platforms often enough looks for deviations. The baseline refer- Fingerprinting (PFP) on the Zynq SoC to prevent against zero-day attacks, ences are the “fingerprints” that unique- to develop and commercialize an IIoT or security holes in software that the ly identify the normal execution of the cybersecurity solution. vendor is unaware of. [4] target software and are extracted using

Third Quarter 2015 Xcell Journal 65 XCELLENCE IN CYBERSECURITY

machine-learning approaches. PFP uses ter and decimate them. This approach The PC-based proof-of-concept sys- the stored references to detect unautho- yields a much more manageable data tem yields excellent results, but for sev- rized execution deviations in real time. bandwidth for the follow-on process- eral reasons falls short as a commercially PFP Cybersecurity successfully ing and greatly simplifies the analog viable system capable of widespread de- developed a proof-of-concept mon- portion of the system design. ployment. The PC system essentially en- itoring system and demonstrated it The feature-extraction and classifica- compasses one monitor node, of which using a personal computer (PC), an tion algorithms process the outputs of the a few hundred may be required per re- off-the-shelf data acquisition device DDCs and compare them against the set al-world installation. The algorithm per- with a high-speed analog-to-digital of baseline references, all of which must formance requirements dictate that the converter (ADC) and a custom ana- operate in real time in order to ensure PC have a capable high-end processor. log front end that interfaces an EM they don’t miss an intrusion. A control Therefore, it will typically require fan sensor to the data acquisition device algorithm runs in parallel to determine cooling, a relatively large enclosure and (as illustrated in Figure 1). The PFP processing parameters, such as the ADC a considerable power source. algorithm engine executes on the PC sample rate and bands of interest. This To maximize the noise immunity in and begins with the collection of the process performs a number of operations the system, the analog-to-digital conver- raw ADC data from the data acquisi- on a large contiguous block of the raw sion of the sensor signal should occur tion device. The front-end processing ADC samples, including a fast Fourier close to the target. The appropriate phys- of the system can be similar in design transform (FFT). The approach provides ical space and availability of power near to many multichannel digital radio continuous 24/7 integrity monitoring on a target processor vary from installation receivers, which collect a wide band the target platform. In the event an in- to installation, and the size and the pow- at the ADC for processing by several trusion is detected, the PFP monitor re- er requirements of the PC are far too ex- digital tuners (commonly called digi- sponds according to an application-spe- treme to be viable for most installations. tal downconverters, or DDCs). These cific policy by alerting operators, logging And although PCs can be cost-effective, DDCs tune to smaller bands of interest event data to a central monitoring station the cost and complexity of integrating within the broader band and then fil- and/or engaging active measures. the remaining components with the PC

GigE NIC PC-Based Monitor Alerts to GigE NIC Central Monitor

Sensor Hard Drive Baseline Comms (Baseline Compare / and References) Threshold Control

Custom Statistical Control Analysis, Algorithm Analog Average, Trigger Front-End Clustering Detect PCB FFT

Feature Extraction OTS Data Acquisition

GigE NIC Unit Digital Down- Trace Converter (DDC) Alignment, Bank Synchron- ization

Processing System

Figure 1 – Blocks in the PC-based monitoring system include front-end analog, data acquisition and processing-system functions.

66 Xcell Journal Third Quarter 2015 XCELLENCE IN CYBERSECURITY

GigE NIC do they have the digital signal process- Zynq SoC-Based Monitor ing (DSP) capability to do anything of significance with that data.

PS LPDDR2 NAND Flash PL LPDDR2 LEVERAGING THE ZYNQ SOC FOR PFP CYBERSECURITY These more stringent requirements are what make the Zynq SoC an ideal fit for

PS PL Sensor this application. The Zynq SoC combines Alerts to Processing Processing a dual-core ARM processing system with

Central Monitor GigE high-performance programmable log- Zynq-7000 All-Programmable SoC ic in a single full-featured device. The combination delivers a heterogeneous Atlas-I-Z7e SoM computing architecture that’s capable of handling the processing demands of the

250- application while simplifying code porta- MSPS Analog Front-End bility from the PC-based system. 16-bit Circuitry The Zynq SoC’s processing system ADC provides all the benefits of an embedded ARM processor as previously mentioned,

Figure 2 – The Zynq SoC-based monitor system is built but the addition of the programmable log- using iVeia’s Atlas-I-Z7e system-on-module. ic offers several advantages. They include glueless connection to the ADC and the become cost-prohibitive. Not to mention, small, low power and low cost. It must ability to process the full data rate of the being a PC makes the monitor node itself be able to process and buffer data from ADC. What’s more, the Zynq SoC contains vulnerable to cyber-attacks. a high-rate ADC, and to handle the com- hundreds of DSP blocks and tens of thou- Architecturally, one option might be putational demands of the algorithm. sands of logic blocks in the programma- to transmit all the raw digital informa- The unit must be small enough to place ble-logic fabric that can be harnessed to tion over a standard network to a cen- in close proximity to the target device greatly accelerate the detection and train- tral processor or server. But due to the and thereby limit the cable length and in- ing algorithms. The Zynq SoC also fulfills high sampling rates of the ADC, the net- crease the noise immunity of the sensor. the requirements for low power, low cost work infrastructure required to support The size and potentially limited installa- and small size. such a volume of data would not like- tion space dictate that the unit operate With a 28-nanometer programma- ly be available at the installation and without a fan; hence, it must be designed ble-logic fabric and ARM processing would be complex and cost-prohibitive for low power consumption. system, the power consumption of to purchase and install. Since there are potentially hundreds the device is comparatively low. The Therefore, a distributed computing ar- of target devices in a given installation high level of integration in the Zynq chitecture is the most desirable choice, that would require monitoring, the unit SoC eliminates the need for much of with one compute node for every sensor. must be low in cost to keep overall in- the supporting circuitry and periph- A distributed architecture also reduces stallation costs low. A number of em- erals that would otherwise be re- cost and complexity by making it possi- bedded processors could satisfy most quired, resulting in a smaller overall ble to combine the sensor analog front of these criteria, including some of system design and lower cost. Fur- end and the algorithm processing into a those based on the popular ARM® ar- ther, a small Zynq SoC-based system- single unit. Also, the existing network in- chitecture. In addition to the low pow- on-a-module (SoM) is desirable in the frastructure for most installations is more er and low cost of most ARM-based de- design to reduce risk and accelerate than adequate to support what would now vices, the ARM has the added benefit time-to-market. be very low data rates. By distributing the of large community support, availabili- The Atlas-I-Z7e from iVeia is ideal for processing, however, the design of the ty of embedded operating systems and the embedded monitor design because monitor node becomes more challenging, development tools, and for most devic- of its high performance-to-power ratio since it alone has to satisfy the combined es, native Gigabit Ethernet support. (due to the low-power Zynq 7020 device requirements of both the sensor node and Where almost all devices fall short, and LPDDR2 memory); dedicated pro- the monitor algorithm processing. however, is the ability to handle the grammable-logic memory for buffering The monitor node must therefore be raw ADC data (rates up to 8 Gbps). Nor ADC data without processor interven-

Third Quarter 2015 Xcell Journal 67 XCELLENCE IN CYBERSECURITY

tion; and reliable operation in industrial to be accelerated in the programma- to 2 Gbps but is easily handled by the environments. Atlas’ flexible glueless in- ble logic vs. code running on the ARM high-performance AXI ports, which con- terfaces simplify baseboard design. The processors. However, with an emphasis nect the Zynq SoC’s programmable logic SoM development kit also includes a roy- on time-to-market, our initial approach to ARM memory. alty-free signal-processing IP repository was to partition the design by moving The DDC core is configured from the with reference designs, which provides a those functions with equivalent, readily ARM processor through the general-pur- significant portion of the monitor’s appli- available IP cores (and that are known pose AXI bus using an application pro- cation code and allows for a rapid design to be compute-intensive) into program- gramming interface. The API allows the ramp-up. Figure 2 illustrates the resulting mable logic. Next, we restructured and software running on the ARM to change Zynq SoC-based monitor design. ported the PC code, and then profiled the DDC parameters on the fly so that up- the remaining code to determine if any dates in center frequency, bandwidth and HANDLING COMPUTE-INTENSIVE additional acceleration was needed. decimation rates can occur in real time as SYSTEM FUNCTIONS The scheme is illustrated in Figure 3. the control algorithm commands. With the hardware selected, the focus The DDCs are an obvious choice to With the data rate significantly re- now shifts to porting the code from the implement in programmable logic, since duced by offloading the DDC, the dual PC-based design to the Zynq SoC-based a DDC core is included as part of the ARM central processing units (CPUs), embedded platform. The computation- SoM development kit and the combined running at 766 MHz, have more than al load on the PC is significant, so the computational requirement for the bank adequate performance to handle the programmable-logic portion of the Zynq of DDCs exceeds 20 gigaflops. The DDC follow-on processing. Using the Linux SoC must be used to accelerate the bank is a part of the intrusion-detection operating system in symmetric-mul- code and cannot simply serve as glue algorithm, which must run in real time tiprocessing (SMP) mode, the design logic. One possible approach would be to avoid missing an intrusion event. splits the processing between the two to port the PC code to the ARM proces- The decimated output of the DDC bank ARM CPU cores, one handling intrusion sor, profile the code to identify compu- passes to the ARM processor for further detection and the other the control al- tational bottlenecks and develop a plan processing of the intrusion algorithm gorithm and communication interface for partitioning the software into code in software. The output rate can run up to the central monitoring station. Linux

Alerts to PL LPDDR2 Memory GigE MAC PS/PL Programmable GigE NIC Central Monitor (Raw Samples) Boundary Logic (PL)

NAND Flash Baseline Comms (Baseline Compare / and References) Threshold Control Multi- Trigger AXI-HP port Detect / Memory Memory Statistical Capture Interface Arbiter / Analysis, Control Average, AXI-HP Port MIG Clustering Control Status / Algorithm 250- AXI Control ADC MSPS Inter- Registers Intf Feature FFT on 16-bit connect Logic Extraction NEON ADC AXI-GP Port

Digital Down- Trace Packetizer Converter (DDC) Alignment, Processing and Bank Synchron- Arbiter

System (PS) ization AXI-HP Port

Core 1 Core 0

Figure 3 – Chart shows the functional partitioning of the Zynq SoC’s PS and PL blocks, with data flow.

68 Xcell Journal Third Quarter 2015 XCELLENCE IN CYBERSECURITY

also has robust networking support and fixed-point math operations. Although Enclustra security, allowing for remote network the FFT function from the Ne10 library Everything FPGA. management (as required by most in- does not operate in real time as does the stallations) while disabling unneces- Xilinx IP core, it accelerates the control 1. MARS ZX2 sary features that might otherwise pres- algorithm adequately enough to main- Zynq-7020 SoC Module ent future vulnerabilities. tain detection accuracy. Xilinx Zynq-7010/7020 SoC FPGA Up to 1 GB DDR3L SDRAM The control processing requires a The resulting Zynq SoC-based moni- 64 MB quad SPI flash large contiguous block of raw ADC sam- tor design performs as well as, and in USB 2.0 Gigabit Ethernet ples. One consideration is to stream the some cases better than, the PC-based Up to 85,120 LUT4-eq 108 user I/Os raw ADC samples from the ADC inter- prototype. However, the resulting de- 3.3 V single supply face logic directly into the ARM mem- sign is significantly cheaper to manu- 67.6 × 30 mm SO-DIMM from $127 ory through the high-performance AXI facture than the PC-based design and ports. However, in order to preserve the eliminates the latter’s market barriers 2. MERCURY ZX5 ™ processor system’s memory bandwidth of larger size and power consumption. Zynq -7015/30 SoC Module Xilinx® Zynq-7015/30 SoC for processing the algorithms, we opt- Comparatively, the Zynq SoC design is 1 GB DDR3L SDRAM 64 MB quad SPI flash ed instead to buffer the ADC data in nearly an order of magnitude smaller PCIe® 2.0 ×4 endpoint the physical memory dedicated to the and lower in power. 4 × 6.25/6.6 Gbps MGT USB 2.0 Device programmable logic. This memory has PFP Cybersecurity developed the Gigabit Ethernet Up to 125,000 LUT4-eq a deterministic bandwidth and ensures Power Fingerprinting technology to 178 user I/Os 5-15 V single supply a large collection of contiguous ADC solve the complex problem of detect- 56 × 54 mm samples without interfering with the ing cyber-attacks in critical equipment, operation of the ARM CPUs. whose growth in number (and vulner- 3. MERCURY ZX1 The data collected in the dedicated ability) is fueled by the Industrial IoT Zynq-7030/35/45 SoC Module programmable-logic memory is trans- trend. With the technology proven, the Xilinx Zynq-7030/35/45 SoC 1 GB DDR3L SDRAM ferred to the ARM through one of the question arose of how to design a sys- 64 MB quad SPI flash PCIe 2.0 ×8 endpoint1 high-performance AXI ports to keep tem to realize the technology while sat- 8 × 6.6/10.3125 Gbps MGT2 USB 2.0 Device latency low and minimize overhead on isfying the needs of the market. With the Gigabit & Dual Fast Ethernet the ARM CPUs. We used a multiport Zynq SoC, the PFP technology became Up to 350,000 LUT4-eq 174 user I/Os memory arbiter to provide one port for commercially viable by providing a su- 5-15 V single supply 64 × 54 mm collection and one port for retrieval. perb intersection between the perfor- 1, 2: Zynq-7030 has 4 MGTs/PCIe lanes. This approach provides the arbitration mance demands of sophisticated and FPGA MANAGER required to simultaneously retrieve computationally intensive processing 4. IP Solution samples as they are being collected, fur- and the market demands for low cost, ther reducing latency. size and power. C/C++ When the newly partitioned design C#/.NET MATLAB® is profiled, the control algorithm does REFERENCE LabVIEW™ not operate frequently enough to ad- 1. ABC News, “‘Trojan Horse’ Bug Lurking in Vital U.S. Computers Since 2011,” Novem- equately maintain detection accura- USB 3.0 cy. The bottleneck in performance is ber 2014 PCIe® Gen2 Gigabit Ethernet largely due to the 16k-point FFT oper- 2. Kushner, D., “The Real Story of Stuxnet,” ation. With the FFT IP core provided IEEE Spectrum, Vol. 50, No. 3, pp. 48-53, with Xilinx’s Vivado® Design Suite, the March 2013 Streaming, FPGA performance of the FFT would be more made simple. than adequate, since it is designed to 3. S. Das, K. Kant and N. Zhang, Handbook on run in real time. But the additional re- Securing Cyber-Physical Critical Infrastruc- One tool for all FPGA communications. ture: Foundations and Challenges, Morgan Stream data from FPGA to host over USB 3.0, source demands on the programmable PCIe, or Gigabit Ethernet – all with one Kaufmann (Waltham, Mass.), 2012 simple API. logic would force the design into a larger Zynq 7030 device. 4. J. Reeves, A. Ramaswamy, M. Locasto, S. Bra- Fortunately, the open-source Ne10 tus, S. Smith, “Lightweight intrusion detection Design Center · FPGA Modules library from Project Ne10 provides an for resource-constrained embedded control Base Boards · IP Cores FFT function optimized for the ARM’s systems,” in Critical Infrastructure Protection We speak FPGA. NEON architecture extension, which V, J. Butts and S. Shenoi (Eds.), Springer, (Hei- accelerates common floating-point and delberg, Germany), pp. 31–46, 2011

Third Quarter 2015 Xcell Journal 69 XCLAMATIONS! Xpress Yourself in Our Caption Contest

MARTIN SCHULMAN, technical director at Symantec Corp. (Herndon, Va.), won a shiny new Digilent Zynq Zybo board with this caption for the robot cartoon in Issue 91 of Xcell Journal: DANIEL GUIDERA

hat do you do when you can’t make a decision? Heads-or-tails is “Art had never seen a robot flash one solution. Eeny, meeny, miny, moe is another. And then, you mob’s rendition of ‘YMCA.’” Wcan always throw darts. At least, that’s the route taken by the frustrated designer pictured in this issue’s cartoon. What’s his problem? We won’t know until a creative contributor writes an engineering- or technology-related caption putting the right spin on the situation. The image might Congratulations as well to inspire a caption like “Stumped by the boss’ verification plan, Bob resorted our two runners-up: to an old-school random-test generation technique.” Send your entries to [email protected]. Include your name, job title, com- “Bob discovers that ‘Klaatu barada pany affiliation and location, and indicate that you have read the contest nikto’ wasn’t gonna work this time.” rules at www.xilinx.com/xcellcontest. After due deliberation, we will print — Michael Brown, software engineering the submissions we like the best in the next issue of Xcell Journal. The manager, Lumenera Corp. winner will receive a Digilent Zynq Zybo board, featuring the Xilinx® Zynq®- (Ottawa, Canada) 7000 All Programmable SoC (http://www.xilinx.com/products/boards-and- kits/1-4AZFTE.htm). Two runners-up will gain notoriety, fame and have “Help! Help! I just saw a REAL bug, their captions, names and affiliations featured in the next issue. ick, I ran it over.” The contest begins at 12:01 a.m. Pacific Time on July 10, 2015. All entries – Frank Perron, software engineer, must be received by the sponsor by 5 p.m. PT on Oct. 1, 2015. Qualcomm Inc. (Boxborough, Mass.)

NO PURCHASE NECESSARY. You must be 18 or older and a resident of the fifty United States, the District of Columbia, or Canada (excluding Quebec) to enter. Entries must be entirely original. Contest begins on July 10, 2015. Entries must be received by 5:00 pm Pacific Time (PT) Oct. 1, 2015. Official rules are available online at www.xilinx.com/xcellcontest. Sponsored by Xilinx, Inc. 2100 Logic Drive, San Jose, CA 95124.

70 Xcell Journal Third Quarter 2015 Multi-Fabric Timing Analysis Industry standard STA for the entire system Closure

PCB Close timing faster What happens when you have a timing problem that cannot be resolved in the SI FPGA itself or the trade-offs required would affect performance and power beyond acceptable limits? How do you communicate these issues with the rest of the product team? How do you determine if the issue would be best solved with a board routing Analog update instead of a logic change? How do you receive and communicate your initial Digital I/O timing requirements to begin with? TimingDesigner® is designed to solve these timing communication and analysis Libraries challenges by providing a common graphical static timing analysis platform that allows the entire product team to quickly and accurately build up timing scenarios for Environmental Compliance effective review and analysis.

Training Get your TimingDesigner trial today Request a free trial today to see how TimingDesigner improves communication and Engineering Data Management { speeds up timing closure. www.ema-eda.com/TDtoday n Xilinx and China Mobile Research Institute Collaborate on Next-Gen Fronthaul for 5G Wireless n A real-time pedestrian detector running on Zynq? Here’s one generated automatically by MathWorks’ Simulink and HDL Coder n CESNET and INVEA-TECH develop FPGA-based bifurcated PCIe Gen3 x16 interface for 100G Ethernet designs n White Rabbit module for NI’s cRIO based on Spartan-6 FPGA: One clock makes you faster and one clock makes you slow n Peel me a grape—and then watch the da Vinci surgical robot suture the grape back together