Security Implications of the Virtualized Datacenter

Total Page:16

File Type:pdf, Size:1020Kb

Security Implications of the Virtualized Datacenter F5 WHITE PAPER - BY ALAN MURPHY - F5 TECHNICAL MARKETING MANAGER, SECURITY SECURITY IMPLICATIONS OF THE VIRTUALIZED DATACENTER Introduction The concepts behind application and operation system virtualization are not new concepts, they have been around long before server appliances and desktop PCs were readily available in our daily vocabulary. The recent rate of virtualization adoption however, especially that of software operating system virtualization, has grown exponentially in the past few years. “Data does not leak According to Joe Tucci, the CEO at EMC®, most VMware® customers are planning on across virtual machines virtualizing 50% of their IT infrastructure within the next three years2. Virtual machines and applications can have fi nally come into their own, and are quickly moving into the enterprise data center and only communicate becoming a universal tool for all people and groups within IT departments everywhere. over confi gured So what exactly is a virtual machine? VMware defi nes a virtualization as “an abstraction network connections.” layer that decouples the physical hardware from the operating system...”3 Today, we 1 commonly think of virtual machines within the scope of one hardware platform running - vmware.com multiple software operating systems. Most often this concept is implemented in the form of one operating system on one hardware box (the host platform) running multiple independent operating systems on virtual hardware platforms in tandem (the guests). Platform virtualization usually relies on full hardware segmentation: allowing individual guest platforms to use specifi c portions of the physical host hardware without confl icting or impacting the host platform. For example, even though guest operating systems require the same CPU and RAM access that the host system requires, the guests use different hardware locations and addresses than the host. This allows the host and guest(s) to run in tandem without stepping on top of each other. There are two primary types of platform virtualization: transparent and host-aware (often referred to as paravirtualization). Transparent virtualization is implemented so that the guest is not aware that it’s running in a virtualized state. The guest consumes resources as if it were natively running on the hardware platform, oblivious to the fact that it’s being managed by an additional component, called the VMM (Virtual Machine Monitor), or hypervisor. The more standard forms of virtualization today, such as those by VMware, implement transparent hypervisor systems. These systems can be thought of as proxies: the hypervisor will transparently proxy all communication between the guest and the host hardware, hiding its existence from the guest so the guest believes it’s the only system running on that hardware. Host-aware implementations differ in that the guest has some form of virtualized knowledge built into the kernel; these can be considered “virtual self-aware” environments. There is some portion of the guest operating system kernel that knows about the existence of the hypervisor and communicates with it directly. Rather than transparent proxying of all communication, the guest OS will call the hypervisor directly, which will in turn manage the communication to the hardware. Xen (pronounced ‘zen’), a popular virtualization implementation for Linux, uses a host- aware architecture, requiring special hypervisor command code actively running in both the host and all running virtualized guests. Each form of virtualization comes with pros and cons, but both work equally as well. Transparent systems are the most portable for the guest, but sacrifi ce speed and are typically designed around much heavier hypervisors; host-aware systems are faster and more lightweight, but require guest modifi cations and can introduce security issues that transparent systems may not suffer from. One of the driving factors in virtualization adoption is the open nature of hardware support for VMMs: Hardware platforms, which run and manage the primary host operating system, and the VMM are not specialized devices or appliances. Virtual host platforms can be any type of hardware that used today: single CPU desktop machines; laptops; x86 servers; SPARC servers; rack mounted appliances; etc. A normal user running Microsoft® Windows® XP Professional on their laptop can run multiple virtual instances of other operating systems—such as Linux, BSD, or Windows Vista— using any number of freely available VMM software implementations. This fl exibility, the move of virtualization software to everyday hardware, has allowed everyone direct and inexpensive access to run virtualized environments. While at fi rst this access was relegated to technology professionals, such as Unix users who were required to run Windows as their base OS, it has quickly become the topic of IT managers. Platform virtualization provides an inexpensive mechanism to substantially expand server farms and data centers. Virtualization allows a company to purchase one high- end hardware device to run 20 virtual operating systems instead of purchasing 20 commoditized lower-end devices, one for each single operating platform. Virtualized Threat Vectors The benefi ts of virtualization are obvious: more bang for your buck. But everything has a pro/con list, and virtualization is no exception. The pro column is a large one, allowing progress and fl exibility for multiple projects and environments. But the con list isn’t so obvious. What could be bad about running 20 servers for the price of one? Although by no means considered to be a large threat today, security of virtual machines and environments is typically not considered, not because the security of these implementations is a technological mystery, but because it is generally an unknown vector by the groups that are implementing wide-spread virtualization. In other words, virtualization is usually implemented with no specifi c regard to the new security risks it brings. On the surface, a virtualized BSD guest carries with it the same security threats and issues that a real, single-device copy has, and this is mostly correct. However the glaring difference is that extra management layer: the hypervisor. The 1 F5 WHITE PAPER SECURITY IMPLICATIONS OF THE VIRTUALIZED DATACENTER hypervisor, is in effect, another operating system that manages the communication between the host OS and the guest OS. Rather than worrying about a single BSD implementation on a single device, administrators now have to worry about the security of three operating environments. For example, if you harden the virtual BSD guest installation, but leave the host and VMM unaddressed, then you are ignoring critical components. If the host is compromised, then security on the guest devices is irrelevant. While the hypervisor is the “Master Controller Program” of virtualization, it isn’t the only virtualized abstraction layer that carries with it implied security risks. Another critical piece to any virtualization system is the networking layer. Different virtualization software platforms have different methods for building and handling networking between the guest and the host, ranging from the more basic, generic network emulation of Qemu4 through VMware’s extremely sophisticated, and proprietary, software switching platform. Often accepted as “just working,” virtualized networking should be considered one of the most critical areas of security research for virtual machines and environments. Unlike hardware network devices, a software-based network brings with it security issues that typically are not seen in hardware. For example, until recently VMware’s guest networking subsystem functioned in “hub only” mode, meaning multiple guest instances on the same physical host within the same software network domain had open and free access to all network data shared within that domain. It was a standard hub implemented via software. If two guests were both a member of the same host-only network and shared the same virtual interface, both machines could see all traffi c between the host and guest. This is still true today for VMware’s bridge mode confi guration, where all guests map to the same physical network devices. And although there are barriers for more robust confi gurations, such as using multiple VMware software networking devices (known as vmnet0, vmnet1, vmnet2, etc.), their entire network management subsystem still exists in software. Using multiple virtual network interfaces to segment traffi c equates to plugging two machines into the same switch, each with unique subnets yet neither segmented by VLANs. VMware is also a great example of where software network security development is rapidly moving ahead. Having created a very robust software switching network in more recent enterprise releases, VWware is moving towards full software-only segmentation of virtual switching from layer 2 VLANs through layer 4 routing. Figure 1 shows a VMware ESX Virtual Switch (vSwitch0) with multiple virtual VLAN segments. As the diagram shows, ‘vm_oracle’ and ‘vm_sharepoint’ are on separate VLANs, but both networks are part of the same virtual switch and traffi c fl ows through the same physical network card. But even with these security-minded advances, software switching still carries a greater risk than hardware. Unlike a real networked environment where proximity is usually a substantial barrier to attack, all guests on one host share the same software stack across all other guests and the hosts. Network stack sharing is a key issue
Recommended publications
  • Understanding Full Virtualization, Paravirtualization, and Hardware Assist
    VMware Understanding Full Virtualization, Paravirtualization, and Hardware Assist Contents Introduction .................................................................................................................1 Overview of x86 Virtualization..................................................................................2 CPU Virtualization .......................................................................................................3 The Challenges of x86 Hardware Virtualization ...........................................................................................................3 Technique 1 - Full Virtualization using Binary Translation......................................................................................4 Technique 2 - OS Assisted Virtualization or Paravirtualization.............................................................................5 Technique 3 - Hardware Assisted Virtualization ..........................................................................................................6 Memory Virtualization................................................................................................6 Device and I/O Virtualization.....................................................................................7 Summarizing the Current State of x86 Virtualization Techniques......................8 Full Virtualization with Binary Translation is the Most Established Technology Today..........................8 Hardware Assist is the Future of Virtualization, but the Real Gains Have
    [Show full text]
  • Enabling Intel® Virtualization Technology Features and Benefits
    WHITE PAPER Intel® Virtualization Technology Enterprise Server Enabling Intel® Virtualization Technology Features and Benefits Maximizing the benefits of virtualization with Intel’s new CPUs and chipsets EXECUTIVE SUMMARY Although virtualization has been accepted in most data centers, some users have not yet taken advantage of all the virtualization features available to them. This white paper describes the features available in Intel® Virtualization Technology (Intel® VT) that work with Intel’s new CPUs and chipsets, showing how they can benefit the end user and how to enable them. Intel® Virtualization Technology Goldberg. Thus, developers found it difficult Feature Brief and Usage Model to implement a virtual machine platform on Intel VT combines with software-based the x86 architecture without significant virtualization solutions to provide maximum overhead on the host machine. system utilization by consolidating multiple environments into a single server or PC. In 2005 and 2006, Intel and AMD, working By abstracting the software away from the independently, each resolved this by creat- underlying hardware, a world of new usage ing new processor extensions to the x86 models opens up that can reduce costs, architecture. Although the actual implemen- increase management efficiency, and tation of processor extensions differs strengthen security—all while making your between AMD and Intel, both achieve the computing infrastructure more resilient in same goal of allowing a virtual machine the event of a disaster. hypervisor to run an unmodified operating system without incurring significant emula- During the last four years, Intel has intro- tion performance penalties. duced several new features to Intel VT. Most of these features are well known, but others Intel VT is Intel’s hardware virtualization for may not be.
    [Show full text]
  • PCI DSS Virtualization Guidelines
    Standard: PCI Data Security Standard (PCI DSS) Version: 2.0 Date: June 2011 Author: Virtualization Special Interest Group PCI Security Standards Council Information Supplement: PCI DSS Virtualization Guidelines Information Supplement • PCI DSS Virtualization Guidelines • June 2011 Table of Contents 1 Introduction ....................................................................................................................... 3 1.1 Audience ................................................................................................................ 3 1.2 Intended Use .......................................................................................................... 4 2 Virtualization Overview .................................................................................................... 5 2.1 Virtualization Concepts and Classes ..................................................................... 5 2.2 Virtual System Components and Scoping Guidance ............................................. 7 3 Risks for Virtualized Environments .............................................................................. 10 3.1 Vulnerabilities in the Physical Environment Apply in a Virtual Environment ....... 10 3.2 Hypervisor Creates New Attack Surface ............................................................. 10 3.3 Increased Complexity of Virtualized Systems and Networks .............................. 11 3.4 More Than One Function per Physical System ................................................... 11 3.5 Mixing VMs of
    [Show full text]
  • Virtualization and Shared Infrastructure Data Storage for IT in Kosovo Institutions Gani Zogaj
    Rochester Institute of Technology RIT Scholar Works Theses Thesis/Dissertation Collections 2012 Virtualization and shared Infrastructure data storage for IT in Kosovo institutions Gani Zogaj Follow this and additional works at: http://scholarworks.rit.edu/theses Recommended Citation Zogaj, Gani, "Virtualization and shared Infrastructure data storage for IT in Kosovo institutions" (2012). Thesis. Rochester Institute of Technology. Accessed from This Master's Project is brought to you for free and open access by the Thesis/Dissertation Collections at RIT Scholar Works. It has been accepted for inclusion in Theses by an authorized administrator of RIT Scholar Works. For more information, please contact [email protected]. AMERICAN UNIVERSITY IN KOSOVO MASTER OF SCIENCE IN PROFESSIONAL STUDIES Virtualization and Shared Infrastructure Data Storage for IT in Kosovo institutions “Submitted as a Capstone Project Report in partial fulfillment of a Master of Science degree in Professional Studies at American University in Kosovo” By Gani ZOGAJ November, 2012 Virtualization and Shared Infrastructure Data Storage for IT in Kosovo institutions ACKNOWLEDGEMENTS First, I would like to thank God for giving me health for completing my Master’s Degree studies. I would like to express my gratitude to my supervisor of capstone proposal, Bryan, for his support and for giving me suggestions and recommendations throughout the Capstone Project work. Finally, I would like to thank my wife, Shkendije, and my lovely daughters, Elisa and Erona, who I love so much, for their understanding while I was preparing the Capstone Project and I couldn’t spend enough time with them. 2 Virtualization and Shared Infrastructure Data Storage for IT in Kosovo institutions Table of Contents Figures and Tables………………………………………………….……………….……….……5 List of Acronyms………………………………………………………………....………….……6 Executive Summary……………………………………………………..………………...………8 Chapter 1…………………………………………………………………………...…………….11 1.
    [Show full text]
  • Oracle® Linux Virtualization Manager Getting Started Guide
    Oracle® Linux Virtualization Manager Getting Started Guide F25124-11 September 2021 Oracle Legal Notices Copyright © 2019, 2021 Oracle and/or its affiliates. This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing. If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, then the following notice is applicable: U.S. GOVERNMENT END USERS: Oracle programs (including any operating system, integrated software, any programs embedded, installed or activated on delivered hardware, and modifications of such programs) and Oracle computer documentation or other Oracle data delivered to or accessed by U.S. Government end users are "commercial computer software" or "commercial computer software documentation" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, the use, reproduction, duplication, release, display, disclosure, modification, preparation of derivative works, and/or adaptation of i) Oracle programs (including any operating system, integrated software, any programs embedded, installed or activated on delivered hardware, and modifications of such programs), ii) Oracle computer documentation and/or iii) other Oracle data, is subject to the rights and limitations specified in the license contained in the applicable contract.
    [Show full text]
  • Firecracker: Lightweight Virtualization for Serverless Applications
    Firecracker: Lightweight Virtualization for Serverless Applications Alexandru Agache, Marc Brooker, Andreea Florescu, Alexandra Iordache, Anthony Liguori, Rolf Neugebauer, Phil Piwonka, and Diana-Maria Popa, Amazon Web Services https://www.usenix.org/conference/nsdi20/presentation/agache This paper is included in the Proceedings of the 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI ’20) February 25–27, 2020 • Santa Clara, CA, USA 978-1-939133-13-7 Open access to the Proceedings of the 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI ’20) is sponsored by Firecracker: Lightweight Virtualization for Serverless Applications Alexandru Agache Marc Brooker Andreea Florescu Amazon Web Services Amazon Web Services Amazon Web Services Alexandra Iordache Anthony Liguori Rolf Neugebauer Amazon Web Services Amazon Web Services Amazon Web Services Phil Piwonka Diana-Maria Popa Amazon Web Services Amazon Web Services Abstract vantage over traditional server provisioning processes: mul- titenancy allows servers to be shared across a large num- Serverless containers and functions are widely used for de- ber of workloads, and the ability to provision new func- ploying and managing software in the cloud. Their popularity tions and containers in milliseconds allows capacity to be is due to reduced cost of operations, improved utilization of switched between workloads quickly as demand changes. hardware, and faster scaling than traditional deployment meth- Serverless is also attracting the attention of the research com- ods. The economics and scale of serverless applications de- munity [21,26,27,44,47], including work on scaling out video mand that workloads from multiple customers run on the same encoding [13], linear algebra [20, 53] and parallel compila- hardware with minimal overhead, while preserving strong se- tion [12].
    [Show full text]
  • Reference Architecture: Lenovo Client Virtualization (LCV) with Thinksystem Servers
    Reference Architecture: Lenovo Client Virtualization (LCV) with ThinkSystem Servers Last update: 10 June 2019 Version 1.3 Base Reference Architecture Describes Lenovo clients, document for all LCV servers, storage, and networking solutions hardware used in LCV solutions LCV covers both virtual Contains system performance desktops and hosted considerations and performance desktops testing methodology and tools Mike Perks Pawan Sharma Table of Contents 1 Introduction ............................................................................................... 1 2 Business problem and business value ................................................... 2 3 Requirements ............................................................................................ 3 4 Architectural overview ............................................................................. 6 5 Component model .................................................................................... 7 5.1 Management services ............................................................................................ 10 5.2 Support services .................................................................................................... 11 5.2.1 Lenovo Thin Client Manager ...................................................................................................... 11 5.2.2 Chromebook management console ........................................................................................... 12 5.3 Storage .................................................................................................................
    [Show full text]
  • Paravirtualization (PV)
    Full and Para Virtualization Dr. Sanjay P. Ahuja, Ph.D. Fidelity National Financial Distinguished Professor of CIS School of Computing, UNF x86 Hardware Virtualization The x86 architecture offers four levels of privilege known as Ring 0, 1, 2 and 3 to operating systems and applications to manage access to the computer hardware. While user level applications typically run in Ring 3, the operating system needs to have direct access to the memory and hardware and must execute its privileged instructions in Ring 0. x86 privilege level architecture without virtualization Technique 1: Full Virtualization using Binary Translation This approach relies on binary translation to trap (into the VMM) and to virtualize certain sensitive and non-virtualizable instructions with new sequences of instructions that have the intended effect on the virtual hardware. Meanwhile, user level code is directly executed on the processor for high performance virtualization. Binary translation approach to x86 virtualization Full Virtualization using Binary Translation This combination of binary translation and direct execution provides Full Virtualization as the guest OS is completely decoupled from the underlying hardware by the virtualization layer. The guest OS is not aware it is being virtualized and requires no modification. The hypervisor translates all operating system instructions at run-time on the fly and caches the results for future use, while user level instructions run unmodified at native speed. VMware’s virtualization products such as VMWare ESXi and Microsoft Virtual Server are examples of full virtualization. Full Virtualization using Binary Translation The performance of full virtualization may not be ideal because it involves binary translation at run-time which is time consuming and can incur a large performance overhead.
    [Show full text]
  • Red Hat Virtualization 4.3 Planning and Prerequisites Guide
    Red Hat Virtualization 4.3 Planning and Prerequisites Guide Planning for the Installation and Configuration of Red Hat Virtualization 4.3 Last Updated: 2019-05-09 Red Hat Virtualization 4.3 Planning and Prerequisites Guide Planning for the Installation and Configuration of Red Hat Virtualization 4.3 Red Hat Virtualization Documentation Team Red Hat Customer Content Services [email protected] Legal Notice Copyright © 2019 Red Hat, Inc. The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/ . In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version. Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law. Red Hat, Red Hat Enterprise Linux, the Shadowman logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries. Linux ® is the registered trademark of Linus Torvalds in the United States and other countries. Java ® is a registered trademark of Oracle and/or its affiliates. XFS ® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries. MySQL ® is a registered trademark of MySQL AB in the United States, the European Union and other countries.
    [Show full text]
  • IT Modernization with Nutanix – a Cloud-Smart Approach
    SOLUTION BRIEF IT Modernization with Nutanix - A Cloud- Smart Approach Nutanix simplifies legacy IT infrastructure while simultaneously enabling IT-as-a-Service by providing the secure and intelligent automation capabilities cloud-based datacenter architectures require THE BUSINESS VALUE Federal agencies are increasingly being asked to spend less time on infrastructure OF NUTANIX and allocate more time and budget to application services that add business value. Despite a continuous stream of IT hardware and software enhancements, • 60% lower cost of operations the infrastructure challenges and costs faced by agencies continue to rise. • 39% lower IT infrastructure costs Traditional IT infrastructure and virtualization software required to meet agency needs are complex and expensive, and datacenter management has become • 534% 5-year ROI painful. Far too much time, effort, and budget are focused on just keeping the • 61% more IT staff efficiencies lights on and the ability to support an agile continuous development environment with no downtime is not possible today. • 7 month payback Today, IT is distributed. There’s no longer just one cloud. Business applications run • 97% fewer outages in private, public, and distributed clouds. But with many clouds, come many • 73% less staff time to deploy silos–different management tools, infrastructure technologies, and consumption compute resources models – all of which make IT more complex. • 68% less staff time to deploy Legacy infrastructure—with separate silos of proprietary hardware-based servers, storage resources storage, storage networks, and virtualization—is not well suited to meet the growing demand or agility needed to service cloud or numerous enterprise workloads including virtualized, bare metal, container-based, DevSecOps automation use cases, or hybrid cloud architectures.
    [Show full text]
  • Malware Collection and Analysis Via Hardware Virtualization Tamas K
    University of Connecticut OpenCommons@UConn Doctoral Dissertations University of Connecticut Graduate School 12-1-2015 Malware Collection and Analysis via Hardware Virtualization Tamas K. Lengyel [email protected] Follow this and additional works at: https://opencommons.uconn.edu/dissertations Recommended Citation Lengyel, Tamas K., "Malware Collection and Analysis via Hardware Virtualization" (2015). Doctoral Dissertations. 964. https://opencommons.uconn.edu/dissertations/964 Malware Collection and Analysis via Hardware Virtualization Tamas Kristof Lengyel University of Connecticut 2015 Abstract Malware is one of the biggest security threat today and deploying effective defen- sive solutions requires the collection and rapid analysis of a continuously increasing number of samples. The collection and analysis is greatly complicated by the pro- liferation of metamorphic malware as the efficacy of signature-based static analysis systems is greatly reduced. While honeypots and dynamic malware analysis has been effectively deployed to combat the problem, significant challenges remain. The rapidly increasing number of malware samples poses a particular challenge as it greatly inflates the cost of the hardware required to process the influx. As modern malware also deploys anti-debugging and obfuscation techniques, the time it takes to formulate effective solutions is further exacerbated. There is a clear need for effective scalability in automated malware collection and analysis. At the same time, modern malware can both detect the monitoring environment and hide in unmonitored corners of the system. It has also been observed that mal- ware modifies its run-time behavior to lead the analysis system astray when it detects a monitoring environment. Consequently, it is critical to create a stealthy environment to hide the presence of the data collection from the external attacker.
    [Show full text]
  • BEST PRACTICES for DATA CENTRE VIRTUALIZATION (ITSP.70.010) March 2020
    UNCLASSIFIED CYBER CENTRE DATA CENTRE VIRTUALIZATION REPORT: BEST PRACTICES FOR DATA CENTRE VIRTUALIZATION (ITSP.70.010) March 2020 PRACTITIONER TLP:WHITE 1 ITSP.70.010 UNCLASSIFIED FOREWORD ITSP.70.010 Cyber Centre Data Centre Virtualization Report: Best Practices for Data Centre Virtualization is an UNCLASSIFIED publication, issued under the authority of the Head, Canadian Centre for Cyber Security (Cyber Centre). For more information, email or phone our Contact Centre: Cyber Centre Contact Centre [email protected] 613-949-7048 or 1-833-CYBER-88 EFFECTIVE DATE This publication takes effect on (03/27/2020). REVISION HISTORY Revision Amendments Date 1 First release. March 27, 2020 2 ITSP.70.010 UNCLASSIFIED OVERVIEW To improve efficiency and streamline operations, many organizations are virtualizing their data centres. However, some are doing so without a proper understanding of the security implications of their decisions on the overall security posture of the data centre. By increasing their consolidation ratios (i.e. the number of virtual machines that can operate on each physical host machine), many organizations may unintentionally compromise their network security architecture. Moreover, any attempt to replace many of the physical isolation mechanisms with logical equivalents may introduce unacceptable risk into their network architecture. Not only are virtualized data centres (VDCs) susceptible to most of the vulnerabilities inherent in traditional data centres, but they introduce new vulnerabilities specific to this environment. While complex virtualization threats (e.g. System Management Mode (SMM) rootkits, side-channel attacks, and hyperjacking) must be addressed, it is the more practical threats that are of greater risk to the VDC.
    [Show full text]