DOCSLIB.ORG

Algorithms Designs and Implementations for Page Allocation in SSD Firmware and SSD Caching in Storage Systems

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Algorithms Designs and Implementations for Page Allocation in SSD Firmware and SSD Caching in Storage Systems Algorithms Designs and Implementations for Page Allocation in SSD Firmware and SSD Caching in Storage Systems Thesis Presented in Partial Fulfillment of the Requirements for the Degre Master of Science in the Graduate School of The Ohio State University By Shuang Liang, B.E., M.E. Graduate Program in Computer Science and Engineering The Ohio State University 2010 Thesis Committee: Xiaodong Zhang, Adviser Feng Qin c Copyright by Shuang Liang 2010 ABSTRACT The emerging flash memory based SSD technology poses a new challenge for us to effectively use it in storage systems. Despite of its many advantages that promise a performance breakthrough for the next generation storage systems, the technical limitations of flash memory write operations call for innovative designs that can overcome the long latency of small random write pattern and the limited lifetime of flash memory cells. Previous studies have attempted to address the write performance limitations by trading ca- pacity at the firmware-level such that the write access pattern can be transformed to avoid small random write patterns. Although it may improve the write performance, it does not utilize the data access locality information available at system level to achieve optimal performance. In this thesis, we study techniques to improve the performance via. system-level designs. We consider a major application scenario of flash SSDs – SSD based disk cache. Based on a new disk cache model for SSD based cache, we propose cache management algorithms which utilize the data access local- ity for write access pattern transformation with minimal cost to cache hit ratio. We implemented the proposed algorithm in an SSD based second-level disk cache system. The results show that the proposed algorithm can significantly improve the performance of existing cache management algorithms. To maximize the lifetime of flash SSDs, we study techniques that can reduce the number of write operations, which consume the life-cycle of flash memory cells. Unlike previous ap- proaches of flash memory life-cycle management, which balance the wear-level of flash memory ii cells through data migration, we focus on reducing the internal data movement overhead introduced by flash memory SSD firmware to mask the long latency of small random write. We found that the internal data movement overhead can be managed effectively by capacity over-provisioning based on the write request working-set size, and we show a theoretical analysis to illustrate the rela- tionship between the minimal over-provisioning capacity and the page allocation policy to achieve zero data movement overhead. Based on these findings, two SSD firmware page allocation al- gorithms are proposed to improve the capacity efficiency. Our trace-driven simulation using the Microsoft SSD extension of the well-known Disksim simulator shows that the proposed algorithm can significantly reduce the internal data movement overhead. iii To My Family iv ACKNOWLEDGMENTS I am indebted to many great people. First of all, I would like to thank my advisor, Prof. Xiaodong Zhang for his patience and thoughtful advice during my graduate study. It is Prof. Zhang’s help, motivation, support, and encouragement that has made me come to this step. I would like to thank my research collaborators: Song Jiang, Ke Chen, Weikuan Yu, Ranjit Noronha and Prof. D.K. Panda. They have inspired and helped me in various aspects of my research. I am also very grateful to my lab-mates in The Ohio State University: Feng Chen, Xiaoning Ding, Lei Guo, Rubao Li, Enhua Tan, and Qi Zhang. Their friendship and support have brought me joyful and unforgettable memories in these years. I would like to thank Prof. Feng Qin for his advice as a committee member so that I can complete this thesis. I am also grateful to Peg Steele and Nikki Strader. Their support and friendship provided me an environment to be productive in research while I was working as a graduate administrative assistant. Finally, I am indebted to my family for their unconditional love and support so that I can go through the toughest moments in these years. v VITA 1999 . B.E. in Computer Science, Huazhong University of Science and Technology 2002 . M.E. in Computer Science, Huazhong University of Science and Technology 2003-present . Graduate student in Computer Science and Engineering, The Ohio State Univer- sity FIELDS OF STUDY Major Field: Computer Science and Engineering PUBLICATIONS Research Publications Song Jiang, Xuechen Zhang, Shuang Liang, and Kei Davis. “Improving Networked File System Performance Using a Locality-Aware Cooperative Cache Protocol”. To appear In IEEE Transac- tion on Computers, 2010 Shuang Liang, Ke Chen, Song Jiang, and Xiaodong Zhang. “Cost-Aware Caching Algorithms for Distributed Storage Servers”. In Proceedings of 21st International Symposium on Distributed Computing (DISC’07) Shuang Liang, Song Jiang, and Xiaodong Zhang. “STEP: Sequentiality and Thrashing Detection Based Prefetching to Improve Performance of Networked Storage Servers”. In Proceedings of 27th International Conference on Distributed Computing Systems (ICDCS’07) vi Shuang Liang, Weikuan Yu, and D.K. Panda. “High Performance Block I/O for Global File System (GFS) with InfiniBand RDMA”. In Proceedings of 35th International Conference for Parallel Processing (ICPP’06) Shuang Liang, Ranjit Noronha, and D.K. Panda. “Swapping to Remote Memory over Infini- Band: An Approach using a High Performance Network Block Device”. In Proceedings of IEEE International Conference on Cluster Computing (Cluster’05) Weikuan Yu, Shuang Liang and D.K. Panda. “High Performance Support of Parallel Virtual File System (PVFS2) over Quadrics”. In Proceedings of 19th ACM International Conference on Supercomputing (ICS ’05) Weikuan Yu, Ranjit Noronha, Shuang Liang and D.K. Panda. “Benefits of High Speed Inter- connects to Cluster File Systems: A Case Study with Lustre”. In Proceedings of International Workshop on Communication Architecture for Clusters (CAC ’06) in conjunction with IPDPS 06 vii TABLE OF CONTENTS Page Abstract . ii Dedication . iv Acknowledgments . v Vita ............................................... vi List of Tables . x List of Figures . xi Chapters: 1. Introduction . 1 1.1 Rationale and Challenge of SSD based Disk Caches . 2 1.2 Challenge of Maximizing the Lifetime of Flash SSD . 3 1.3 Contribution of the Thesis . 4 1.3.1 Flash SSD based Disk Cache . 4 1.3.2 Maximizing the Lifetime of Flash SSD . 5 1.4 Thesis Organization . 5 2. Background and Related Work . 7 2.1 Flash Memory SSDs Basics . 7 2.2 Enhancing Flash SSDs via. FTL . 9 2.2.1 Performance Optimization . 9 2.2.2 Managing the Life-cycles of SSDs . 11 2.3 Related Work . 11 2.3.1 Caching Models and Algorithms . 12 viii 2.3.2 Flash Related Systems . 13 3. Flash SSD based Disk Cache . 16 3.1 An SSD-based Caching Model . 16 3.2 Semi-Random Writes for SSD based Disk Cache . 18 3.3 Our SSD-based L2 Cache Design . 20 3.3.1 SSD Aware Placement . 20 3.3.2 Zone based Admission . 25 3.3.3 On Exclusive Cache . 28 3.4 Experimental Studies . 29 3.4.1 Simulation . 30 3.4.2 System Implementation . 36 3.5 Summary . 40 4. Maximizing Flash SSD Lifetime . 42 4.1 Overview . 42 4.2 Capacity, Access Pattern, Performance and Wear . 42 4.2.1 Over-provisioning Capacity and Write Amplification . 42 4.2.2 Access Pattern Impacts . 44 4.3 Workload Measurements . 45 4.3.1 Experiment Environment . 45 4.3.2 Evaluation Results . 46 4.4 Write Amplification Rate Reduction Algorithms . 51 4.4.1 Algorithm Preliminaries . 52 4.4.2 WFWD Allocation Algorithm . 55 4.4.3 Predicting the Write Forward Distance . 56 4.4.4 Implementation in DiskSim-SSD . 57 4.4.5 Performance Evaluations . 58 4.5 Summary . 62 5. Conclusions . 64 Bibliography . 65 ix LIST OF TABLES Table Page 3.1 Product Specification of SSDs .............................. 30 4.1 Workload Description .................................. 45 4.2 Parameters of DiskSim-SSD ............................... 46 4.3 Over-provisioning Capacity vs. Workload Storage Address Space Size . 50 x LIST OF FIGURES Figure Page 1.1 SSD endurance/ECC vs. progress geometry (source: Micron at Memcon’09) . 4 2.1 An example SSD Internal Organization(source:Microsoft at Usenix’08) . 9 3.1 The SSD based two level cache architecture. 17 3.2 Histogram of the evicted page address for MIN, LRU, and DULO replacement algorithms. 19 3.3 Selection of new active placement blocks. 22 3.4 Detailed Placement Algorithm . 24 3.5 Zone based Admission Control Algorithm . 27 3.6 Overall Performance of SSD based L2 cache . 31 3.7 Detail Performance of Placement and Admission Control . 32 3.8 Effects of Exclusive Cache . 33 3.9 α and Placement Block Size on Miss Ratio . 34 3.10 Comparison of High-end and Low-end SSDs . 35 3.11 The architecture of the system implementation of SSD based L2 cache in Linux kernel. 36 3.12 Replaying the OLTP Traces on Real System for 3 Runs. 37 xi 3.13 Evaluation Results of TPC-H on MySQL . 40 4.1 Sensitivity to Write Working-set based OPC . 47 4.2 Sensitivity to Disk Capacity based OPC . 48 4.3 State Transitions of Physical Pages in SSDs . 52 4.4 An Example for Write Forward Distance based Allocation. 57 4.5 Comparison of Algorithms (a) . 60 4.6 Comparison of Algorithms (b) . 61 xii CHAPTER 1 Introduction The exponential growth of information in modern society has been constantly demanding the advancement of storage technology to provide better performance and larger capacity for various data intensive applications. In the past 50 years, the magnetic disk as the major online system storage technology has been successfully keeping up with the capacity requirements, however, it has not been able to catch up with the performance scaling trend of data processing speed.
Load more

Recommended publications
  • Copy on Write Based File Systems Performance Analysis and Implementation
    Copy On Write Based File Systems Performance Analysis And Implementation Sakis Kasampalis Kongens Lyngby 2010 IMM-MSC-2010-63 Technical University of Denmark Department Of Informatics Building 321, DK-2800 Kongens Lyngby, Denmark Phone +45 45253351, Fax +45 45882673 [email protected] www.imm.dtu.dk Abstract In this work I am focusing on Copy On Write based file systems. Copy On Write is used on modern file systems for providing (1) metadata and data consistency using transactional semantics, (2) cheap and instant backups using snapshots and clones. This thesis is divided into two main parts. The first part focuses on the design and performance of Copy On Write based file systems. Recent efforts aiming at creating a Copy On Write based file system are ZFS, Btrfs, ext3cow, Hammer, and LLFS. My work focuses only on ZFS and Btrfs, since they support the most advanced features. The main goals of ZFS and Btrfs are to offer a scalable, fault tolerant, and easy to administrate file system. I evaluate the performance and scalability of ZFS and Btrfs. The evaluation includes studying their design and testing their performance and scalability against a set of recommended file system benchmarks. Most computers are already based on multi-core and multiple processor architec- tures. Because of that, the need for using concurrent programming models has increased. Transactions can be very helpful for supporting concurrent program- ming models, which ensure that system updates are consistent. Unfortunately, the majority of operating systems and file systems either do not support trans- actions at all, or they simply do not expose them to the users.
    [Show full text]
  • The Page Cache Today’S Lecture RCU File System Networking(Kernel Level Syncmem
    2/21/20 COMP 790: OS Implementation COMP 790: OS Implementation Logical Diagram Binary Memory Threads Formats Allocators User System Calls Kernel The Page Cache Today’s Lecture RCU File System Networking(kernel level Syncmem. Don Porter management) Memory Device CPU Management Drivers Scheduler Hardware Interrupts Disk Net Consistency 1 2 1 2 COMP 790: OS Implementation COMP 790: OS Implementation Recap of previous lectures Background • Page tables: translate virtual addresses to physical • Lab2: Track physical pages with an array of PageInfo addresses structs • VM Areas (Linux): track what should be mapped at in – Contains reference counts the virtual address space of a process – Free list layered over this array • Hoard/Linux slab: Efficient allocation of objects from • Just like JOS, Linux represents physical memory with a superblock/slab of pages an array of page structs – Obviously, not the exact same contents, but same idea • Pages can be allocated to processes, or to cache file data in memory 3 4 3 4 COMP 790: OS Implementation COMP 790: OS Implementation Today’s Problem The address space abstraction • Given a VMA or a file’s inode, how do I figure out • Unifying abstraction: which physical pages are storing its data? – Each file inode has an address space (0—file size) • Next lecture: We will go the other way, from a – So do block devices that cache data in RAM (0---dev size) physical page back to the VMA or file inode – The (anonymous) virtual memory of a process has an address space (0—4GB on x86) • In other words, all page
    [Show full text]
  • Dynamically Tuning the JFS Cache for Your Job Sjoerd Visser Dynamically Tuning the JFS Cache for Your Job Sjoerd Visser
    Dynamically Tuning the JFS Cache for Your Job Sjoerd Visser Dynamically Tuning the JFS Cache for Your Job Sjoerd Visser The purpose of this presentation is the explanation of: IBM JFS goals: Where was Journaled File System (JFS) designed for? JFS cache design: How the JFS File System and Cache work. JFS benchmarking: How to measure JFS performance under OS/2. JFS cache tuning: How to optimize JFS performance for your job. What do these settings say to you? [E:\]cachejfs SyncTime: 8 seconds MaxAge: 30 seconds BufferIdle: 6 seconds Cache Size: 400000 kbytes Min Free buffers: 8000 ( 32000 K) Max Free buffers: 16000 ( 64000 K) Lazy Write is enabled Do you have a feeling for this? Do you understand the dynamic cache behaviour of the JFS cache? Or do you just rely on the “proven” cachejfs settings that the eCS installation presented to you? Do you realise that the JFS cache behaviour may be optimized for your jobs? November 13, 2009 / page 2 Dynamically Tuning the JFS Cache for Your Job Sjoerd Visser Where was Journaled File System (JFS) designed for? 1986 Advanced Interactive eXecutive (AIX) v.1 based on UNIX System V. for IBM's RT/PC. 1990 JFS1 on AIX was introduced with AIX version 3.1 for the RS/6000 workstations and servers using 32-bit and later 64-bit IBM POWER or PowerPC RISC CPUs. 1994 JFS1 was adapted for SMP servers (AIX 4) with more CPU power, many hard disks and plenty of RAM for cache and buffers. 1995-2000 JFS(2) (revised AIX independent version in c) was ported to OS/2 4.5 (1999) and Linux (2000) and also was the base code of the current JFS2 on AIX branch.
    [Show full text]
  • NOVA: a Log-Structured File System for Hybrid Volatile/Non
    NOVA: A Log-structured File System for Hybrid Volatile/Non-volatile Main Memories Jian Xu and Steven Swanson, University of California, San Diego https://www.usenix.org/conference/fast16/technical-sessions/presentation/xu This paper is included in the Proceedings of the 14th USENIX Conference on File and Storage Technologies (FAST ’16). February 22–25, 2016 • Santa Clara, CA, USA ISBN 978-1-931971-28-7 Open access to the Proceedings of the 14th USENIX Conference on File and Storage Technologies is sponsored by USENIX NOVA: A Log-structured File System for Hybrid Volatile/Non-volatile Main Memories Jian Xu Steven Swanson University of California, San Diego Abstract Hybrid DRAM/NVMM storage systems present a host of opportunities and challenges for system designers. These sys- Fast non-volatile memories (NVMs) will soon appear on tems need to minimize software overhead if they are to fully the processor memory bus alongside DRAM. The result- exploit NVMM’s high performance and efficiently support ing hybrid memory systems will provide software with sub- more flexible access patterns, and at the same time they must microsecond, high-bandwidth access to persistent data, but provide the strong consistency guarantees that applications managing, accessing, and maintaining consistency for data require and respect the limitations of emerging memories stored in NVM raises a host of challenges. Existing file sys- (e.g., limited program cycles). tems built for spinning or solid-state disks introduce software Conventional file systems are not suitable for hybrid mem- overheads that would obscure the performance that NVMs ory systems because they are built for the performance char- should provide, but proposed file systems for NVMs either in- acteristics of disks (spinning or solid state) and rely on disks’ cur similar overheads or fail to provide the strong consistency consistency guarantees (e.g., that sector updates are atomic) guarantees that applications require.
    [Show full text]
  • MULTI-STREAM SSD TECHNOLOGY 2X READ PERFORMANCE & WRITE ENDURANCE T10 SCSI STANDARD
    MULTI-STREAM SSD TECHNOLOGY 2x READ PERFORMANCE & WRITE ENDURANCE T10 SCSI STANDARD NAND flash solid-state drives (SSDs) are increasingly deployed within enterprise datacenters thanks to their high performance and low power consumption. Decreasing NAND flash cost-per-gigabyte is also accelerating SSD adoption to replace hard disk drives (HDDs) in storage applications. One SSD drawback is that, as a device continually writes data, valid data can be fragmented across the NAND flash medium (See Figure 1). To reclaim free space, garbage collection activity copies user data to new storage blocks and erases invalid data storage blocks, thereby allowing the media to store new write data. However, garbage collection processing decreases both SSD read and write performance. In addition, garbage collection increases write amplification because individual host data write requests can result in multiple internal SSD writes to the NAND medium. Here, valid data is first read from a media block about to be erased, then rewritten to another media storage block, accompanied by the write to store new host data. Consequently, write amplification decreases SSD lifetime because each NAND chip can endure a certain number of writes before it begins to fail. MULTI-STREAM SSD TECHNOLOGY With multi-stream technology, an innovative new technology System Configuration FIO Configuration standardized in T10, implemented in Samsung PM953S NVMe SSD, garbage collection can be eliminated or reduced by storing • Hardware system • I/O workload: associated or similar lifetime data in the same erase block. This Quad Core Intel i7-4790 70% Read/ 30% Write helps avoid NAND erase block fragmentation for data with the CPU 3.60GHz • 4 sequential write jobs same lifetime.
    [Show full text]
  • Hibachi: a Cooperative Hybrid Cache with NVRAM and DRAM for Storage Arrays
    Hibachi: A Cooperative Hybrid Cache with NVRAM and DRAM for Storage Arrays Ziqi Fan, Fenggang Wu, Dongchul Parkx, Jim Diehl, Doug Voigty, and David H.C. Du University of Minnesota–Twin Cities, xIntel Corporation, yHewlett Packard Enterprise Email: [email protected], ffenggang, park, [email protected], [email protected], [email protected] Abstract—Adopting newer non-volatile memory (NVRAM) Application Application Application technologies as write buffers in slower storage arrays is a promising approach to improve write performance. However, due OS … OS … OS to DRAM’s merits, including shorter access latency and lower Buffer/Cache Buffer/Cache Buffer/Cache cost, it is still desirable to incorporate DRAM as a read cache along with NVRAM. Although numerous cache policies have been Application, web or proposed, most are either targeted at main memory buffer caches, database servers or manage NVRAM as write buffers and separately manage DRAM as read caches. To the best of our knowledge, cooperative hybrid volatile and non-volatile memory buffer cache policies specifically designed for storage systems using newer NVRAM Storage Area technologies have not been well studied. Network (SAN) This paper, based on our elaborate study of storage server Hibachi Cache block I/O traces, proposes a novel cooperative HybrId NVRAM DRAM NVRAM and DRAM Buffer cACHe polIcy for storage arrays, named Hibachi. Hibachi treats read cache hits and write cache hits differently to maximize cache hit rates and judiciously adjusts the clean and the dirty cache sizes to capture workloads’ tendencies. In addition, it converts random writes to sequential writes for high disk write throughput and further exploits storage server I/O workload characteristics to improve read performance.
    [Show full text]
  • Things You Should Know About Solid State Storage
    ThingsPRESENTATION You Should TITLE Know GOES HERE About Solid State Storage Snippets from SNIA Tutorials and other Giblets Marty Czekalski - President SCSI Trade Association, Sr. Staff Program Manager - Seagate Technology SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA unless otherwise noted. Member companies and individual members may use this material in presentations and literature under the following conditions: Any slide or slides used must be reproduced in their entirety without modification The SNIA must be acknowledged as the source of any material used in the body of any document containing material from these presentations. This presentation is a project of the SNIA Education Committee. Neither the author nor the presenter is an attorney and nothing in this presentation is intended to be, or should be construed as legal advice or an opinion of counsel. If you need legal advice or a legal opinion please contact your attorney. The information presented herein represents the author's personal opinion and current understanding of the relevant issues involved. The author, the presenter, and the SNIA do not assume any responsibility or liability for damages arising out of any reliance on or use of this information. NO WARRANTIES, EXPRESS OR IMPLIED. USE AT YOUR OWN RISK. What You Should Know About Solid State Storage © 2013 Storage Networking Industry Association. All Rights Reserved. 2 Abstract What You Should Know About Solid State Storage This session will appeal to Data Center Managers, Development Managers, and those that are seeking an overview of Solid State Storage. It’s comprised of excerpts from SNIA Solid State Tutorials and other sources.
    [Show full text]
  • How Controllers Maximize SSD Life
    SSSI TECH NOTES How Controllers Maximize SSD Life January 2013 by SNIA SSSI Member: Jim Handy Objective Analysis “The SSD Guy” www.snia.org1 About the Solid State Storage Initiative The SNIA Solid State Storage Initiative (SSSI) fosters the growth and success of the market for solid state storage in both enterprise and client environ- ments. Members of the SSSI work together to promote the development of technical standards and tools, educate the IT communities about solid state storage, perform market outreach that highlights the virtues of solid state storage, and collaborate with other industry associations on solid state stor- age technical work. SSSI member companies come from a wide variety of segments in the SSD industry www.snia.org/forums/sssi/about/members. How Controllers Maximize SSD Life by SNIA SSSI Member: Jim Handy “The SSD Guy”, Objective Analysis Table of Contents Introduction 2 How Controllers Maximize SSD Life 2 Better Wear Leveling 3 External Data Buffering 6 Improved ECC 7 Other Error Management 9 Reduced Write Amplification 10 Over Provisioning 11 Feedback on Block Wear 13 Internal NAND Management 14 1 Introduction This booklet contains a collection of posts from Jim Handy’s SSD Guy blog www.TheSSDGuy.com which explores the various techniques designers use to increase SSD life. How Controllers Maximize SSD Life How do controllers maximize the life of an SSD? After all, MLC flash has a lifetime of only 10,000 erase/write cycles or fewer and that is a very small number compared to the write traffic an SSD is expected to see in a high- workload environment, especially in the enterprise.
    [Show full text]
  • Write Amplification Analysis in Flash-Based Solid State Drives
    Write Amplification Analysis in Flash-Based Solid State Drives Xiao-Yu Hu, Evangelos Eleftheriou, Robert Haas, Ilias Iliadis, Roman Pletka IBM Research IBM Zurich Research Laboratory CH-8803 Rüschlikon, Switzerland {xhu,ele,rha,ili,rap}@zurich.ibm.com ABSTRACT age computer architecture, ranging from notebooks to en- Write amplification is a critical factor limiting the random terprise storage systems. These devices provide random I/O write performance and write endurance in storage devices performance and access latency that are orders of magnitude based on NAND-flash memories such as solid-state drives better than that of rotating hard-disk drives (HDD). More- (SSD). The impact of garbage collection on write amplifica- over, SSDs significantly reduce power consumption and dra- tion is influenced by the level of over-provisioning and the matically improve robustness and shock resistance thanks to choice of reclaiming policy. In this paper, we present a novel the absence of moving parts. probabilistic model of write amplification for log-structured NAND-flash memories have unique characteristics that flash-based SSDs. Specifically, we quantify the impact of pose challenges to the SSD system design, especially the over-provisioning on write amplification analytically and by aspects of random write performance and write endurance. simulation assuming workloads of uniformly-distributed ran- They are organized in terms of blocks, each block consist- dom short writes. Moreover, we propose modified versions ing of a fixed number of pages, typically 64 pages of 4 KiB of the greedy garbage-collection reclaiming policy and com- each. A block is the elementary unit for erase operations, pare their performance.
    [Show full text]
  • Why Data Retention Is Becoming More Critical in Automotive
    White Paper Why Data Retention is Becoming More Critical in Automotive Applications Understanding and Improving Managed NAND Flash Memory for Higher Data Retention and Extended Product Life Christine Lee – Kevin Hsu – Scott Harlin KIOXIA America, Inc. Assisted and self-driving vehicles are fully loaded with electronics that support the infrastructure within. They have become mobile data centers that require an immense amount of computing power to capture, process and analyze data ‘near-instantaneously’ from a myriad of sensors, recorders, algorithms and external connections. A massive amount of this data is either stored locally or uploaded to cloud storage to be leveraged and transformed into real-time intelligence and value. Data storage has now become a critical part of automotive design, placing a precedence on high data retention and continual data integrity. The automotive environment creates unique challenges and presents a much different scenario from computing equipment in air-conditioned server rooms under controlled temperatures. Due to the extreme temperatures that can affect the NAND flash storage used within vehicles, there are information technology (IT) considerations that require different design approaches. Understanding how data wears out, how temperature and NAND flash memory characteristics can affect data retention and product life, and what practices can be used to improve them, are the focuses of this paper. ‘Under the Hood’ in Automotive Data Storage Depending on the source, assisted and self-driving vehicles generate terabytes (TB)1 of data daily. One prediction2 forecasts that between 5 TB and 20 TB of data will be consumed per day per vehicle, which is overwhelmingly more data than the average person consumes on a smartphone3.
    [Show full text]
  • Getting the Most out of SSD: Sometimes, Less Is More
    Getting the Most Out of SSD: Sometimes, Less is More Bruce Moxon Chief Solutions Architect STEC Flash Memory Summit 2012 Santa Clara, CA 1 Overview • A Quick Solid State Backgrounder • NAND Flash, Log structured file systems, GC, and Overprovisioning – less really is more! • Benchmarking • Operationally representative testing • Applications • Why Flash, why now? Enough • Caching – Less *is* more • Optimizing the Stack and Changing Application Architectures Flash Memory Summit 2012 Santa Clara, CA 2 Solid State Performance Characteristics (General) HDD (SAS) Sequential Random General Performance Characteristics. Read 200 MB/s 200 IOPS YMMV, depending on Write 200 MB/s 200 IOPS . Device architecture . Interface (SATA, SAS, PCIe) 2-8x 100-1000x • 2-4x performance range SSD / PCIe Sequential 4K Random . Transfer sizes Read .5 – 1.5 GB/s 60-200K IOPS . QDs (concurrency) Write .3 – 1 GB/s 15-40K IOPS Cost Differential . $0.50 - $1.50 / GB SAS HDD . $2 - $12 / GB SSD/PCIe Rand Read Write Sweet Spot Response . High Random IOPS (esp. Read) HDD 8 ms 0.5 ms* . Low Latency SSD 60 us 20 us Solid State Storage Fundamentals Everything I Needed to Know I learned at FMS … . Data is read/written in pages (typically 4-8 KB) . Data is *erased* in multi-page blocks (e.g., 128) . Data can only be written (programmed) into a previously erased block (no “overwrite”) . Background garbage collection (GC) copies valid pages to “squeeze out” deleted pages and make room for new data • OS/FS integration (TRIM) . Additional write amplification can occur in support of wear leveling and GC . Flash memory can only be programmed/erased (P/E) a limited number of times (Endurance) • Performance degrades over device lifetime .
    [Show full text]
  • UNIVERSITY of CALIFORNIA, SAN DIEGO Model and Analysis of Trim
    UNIVERSITY OF CALIFORNIA, SAN DIEGO Model and Analysis of Trim Commands in Solid State Drives A dissertation submitted in partial satisfaction of the requirements for the degree Doctor of Philosophy in Electrical Engineering (Intelligent Systems, Robotics, and Control) by Tasha Christine Frankie Committee in charge: Ken Kreutz-Delgado, Chair Pamela Cosman Philip Gill Gordon Hughes Gert Lanckriet Jan Talbot 2012 The Dissertation of Tasha Christine Frankie is approved, and it is acceptable in quality and form for publication on microfilm and electronically: Chair University of California, San Diego 2012 iii TABLE OF CONTENTS Signature Page . iii Table of Contents . iv List of Figures . vi List of Tables . x Acknowledgements . xi Vita . xii Abstract of the Dissertation . xiv Chapter 1 Introduction . 1 Chapter 2 NAND Flash SSD Primer . 5 2.1 SSD Layout . 5 2.2 Greedy Garbage Collection . 6 2.3 Write Amplification . 8 2.4 Overprovisioning . 10 2.5 Trim Command . 11 2.6 Uniform Random Workloads . 11 2.7 Object Based Storage . 13 Chapter 3 Trim Model . 15 3.1 Workload as a Markov Birth-Death Chain . 15 3.1.1 Steady State Solution . 17 3.2 Interpretation of Steady State Occupation Probabilities . 18 3.3 Region of Convergence of Taylor Series Expansion . 22 3.4 Higher Order Moments . 23 3.5 Effective Overprovisioning . 24 3.6 Simulation Results . 26 Chapter 4 Application to Write Amplification . 35 4.1 Write Amplification Under the Standard Uniform Random Workload . 35 4.2 Write Amplification Analysis of Trim-Modified Uniform Random Workload . 37 4.3 Alternate Models for Write Amplification Under the Trim-Modified Uniform Random Workload .
    [Show full text]