BRKINI-2390.Pdf
Total Page:16
File Type:pdf, Size:1020Kb
BRKINI-2390 Data Center security within modern compute and attached fabrics - servers, IO, management Dan Hanson Director UCS Architectures and Technical Marketing Cisco Spark Questions? Use Cisco Spark to communicate with the speaker after the session How 1. Find this session in the Cisco Live Mobile App 2. Click “Join the Discussion” 3. Install Spark or go directly to the space 4. Enter messages/questions in the space cs.co/ciscolivebot#BRKINI-2390 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Recent Press Items © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Agenda • x86 Architecture Review • BIOS and Kernel Manipulation • Device Firmware Manipulation • On Server Data Storage Manipulation • Segmentation and Device access • Root of Trust Flow • Policy Control vs. Component Configuration (Cisco UCS and ACI) • Example of Security Offloading: Skyport Systems • Conclusion x86 Architecture Review Many Points of possible attack X86 Reference Legacy Elements • You may see many terms in various articles shown here • Over time, items moving on the CPU itself • Memory • PCIe • Processors/Servers differentiation in some areas • Front Side Bus speeds • Direct Media Interface • Southbridge Configurations BRKINI-2390 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 7 X86 Reference Fundamental Architecture • Not shown: Quick Path Interconnect/UltraPath Interconnect for CPU to CPU communications • Varied counts and speeds for multi-socket systems • Current Designs have On-Die PCIe and Memory controllers • Varied numbers and DIMMs in memory channels by CPU • Platform Controller Hub varies and can even offer server acceleration and security functions BRKINI-2390 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 8 Processing Cache Fundamental Caching on Processor • Representative diagram here • Caching was key to enhanced performance in industry • Caching in 3 levels typical on Intel and others • L1 is on-core and typically smallest with fastest access to CPU registers (scratchpad) • L2 is also on-core with larger size, but slower access than L1 • L3 shared over all cores on die • Data recursively searched from L1 through L3 (or L4) then DRAM BRKINI-2390 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 9 System DRAM Fundamental Memory Access • This drawing is illustrative, showing Skylake layout example • Memory and PCIe lanes now directly on CPU • UPI for multi-socket shown here • DMI3 for PCH connection • Omni-Path HFI is on certain models for that IO technology BRKINI-2390 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 10 Basic Input Output System (BIOS) Fundamental Device Access • Non-volatile basic storage for loading images • Typically stored in dual bank flash (for updating) • BIOS rootkits are a method for attack • Hardware initialization and boot loader • Run time services to the Operating System • Modern systems do not typically use BIOS after bootup • OS directly talk with hardware • Cannot talk to external components typically BRKINI-2390 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 11 UEFI Modern Fundamental Device Access • Unified Extensible Firmware Interface – UEFI is a low level process run pre-boot (replaces the BIOS) • UEFI spec 2.3.1 errata C (http://www.uefi.org/specs) • UEFI can test images, including native OS loaders, prior to allowing them to execute • Images are signed with OS Vendor Key (Lock-in FUD out there) • UEFI can also define the trusted boot paths UCS will then move onto next device in boot list if not trusted • UEFI can halt the execution based on exceptions – or allow boot in non TXT mode • UEFI allows for networking and storage functions at this low level • UEFI drivers for our VIC Adapters coming – to support SAN booting BRKINI-2390 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 12 Processor Kernel vs. User Modes Access to system information and configuration – Physical Servers • Most modern CPUs support a hardware enforced protected mode • This mode is only ring 0 in the diagram • X86 CPUs support 3 rings with different privledges • Ring 0: OS kernel System Call • Ring 1,2: Device Drivers • Ring 3: User space • Most OS will use only rings 0 and 3 • Transitions between rings at well defined calls only ** ** Recent press articles around ring 0 memory structures being directly accessible from ring 3 (circumvent system call to read data) BRKINI-2390 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 13 Processor Kernel vs. User Modes Access to system information and configuration – Virtual Servers • Hypervisor offer shared ring 0 device access to Virtual Machines • Commonly called Ring -1 • Virtualization Extensions in CPU to assist hypervisor in handling memory page table switching, IO mapping, etc. System Call Ring -1 • Modern CPU VirtualizationExtensions also isolate data between these VMs • One VM cannot inspect other VM’s memory resident data • Hypervisor itself can (more on addressing these with newer extensions later) ** Recent press articles around ring 0 memory structures being directly accessible from ring 3 (circumvent system call to read data) BRKINI-2390 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 14 x86 System Componentry Platform Controller • Platform Controller has multiple capabilities to server system • Onboard LOM ports • Optional co-processing • USB access • PCI and PCIe access • Serial Peripheral Interface • Onboard storage like SATA/SAS • Low Pin Count bus for some items like management controller • Attacks can be partially mitigated through BIOS control BRKINI-2390 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 15 x86 System Componentry Baseboard Management Controller • Key differentiation between desktop and server • Integrated IO to world • Ethernet • Async Serial • Storage (Flash, etc.) • Internal control of server • Environmentals and Power control • Storage controller config (on certain server models) • Virtual USB/Keyboard/Video/Mouse/Disk • Component firmware • Completely server vendor owned, and typically hardened BRKINI-2390 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 16 x86 System Componentry Peripheral Component Interconnect • Gen3 speeds shown • Gen3 roughly 2x of Gen2 • Gen2 roughly 2x of Gen1 • Went from 8B/10B to 128/130B encoding for less overhead in Gen3 • Multiple parallel lines with serial data • Can have differing physical vs. electrical slots • Not all pins enabled example • x16 physically but x8 electrically • Power specified along with speeds • 25W standard • 70W after poweron • 300 and 350W with added power pins BRKINI-2390 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 17 x86 System Componentry Storage Controller • Dedicated device for multi-disk configuration • SATA/SAS links to disk, PCIe to host • Caching of storage typical • Original battery backup now typically supercap • Scale is to maintain data in cache over a weekend • Advanced geometries of multiple units • Caching of data in unencrypted format had typically led to targeting of this area BRKINI-2390 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 18 x86 System Componentry Trusted Platform Module • Add-On Chip which Enables Secure Generation and Storage of Authentication Artifacts (Passwords/Certificates/Encryption Keys) • Also provides pseudo-random values for various crypto functions • Has Secure NVRAM location for storage of secret platform information (Checksums, Signed Keys, Platform Control Registers (PCRs), etc.) • Installed with a one-way screw, and goal is to be tamper-resistant • UCSM Inventory and Qualification Policies can trigger on TPM presence • Establishes a Root of Trust for Measurement and Storage (RTM, RTS) on this NVRAM • Advocates Suggest Geo-Location in Cloud to validate where services exist BRKINI-2390 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 19 x86 System Booting Starting Boot • BIOS invokes boot loader for OS • Traditional systems pointed to boot image on persistent media to execute • System level OPROM and other utilities typically tied to physical peripheral in host • Trusted boot added in modern system to measure and attest the validity of boot images BRKINI-2390 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 20 x86 System Booting OPtion Read Only Memory (OPROM) • Peripherals and Optimization Hardware with many options • Inserts into the boot sequence • Pre-boot environment to make low level configuration and operational validations • Multiple human interactions typically with function keys to enter configuration utilities • Security within this executing code on server left to peripheral vendor BRKINI-2390 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 21 x86 System Componentry Device Firmware • Firmware to be run on peripherals with processing capabilities • Different but often matched to OS side specific drivers to kernel • Increasing point of inspection and attack possibilities • Open nature of systems leaves security enforcement model to peripheral vendor BRKINI-2390 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 22 x86 System Componentry Device Drivers • Software in OS that controls the hardware element • Version maintained by OS and patched over time • Typically plugs into OS side framework for configuration and control • Maintained individually on machines