DOCSLIB.ORG

I/O Schedulers for Proportionality and Stability on Flash-Based Ssds in Multi-Tenant Environments

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
I/O Schedulers for Proportionality and Stability on Flash-Based Ssds in Multi-Tenant Environments Received November 19, 2019, accepted December 10, 2019, date of publication December 30, 2019, date of current version January 8, 2020. Digital Object Identifier 10.1109/ACCESS.2019.2963081 I/O Schedulers for Proportionality and Stability on Flash-Based SSDs in Multi-Tenant Environments JAEHO KIM 1, EUNJAE LEE 2, AND SAM H. NOH 2, (Senior Member, IEEE) 1Department of Electrical and Computer Engineering, Virginia Tech, Blacksburg, VA 24061, USA 2School of Electrical and Computer Engineering, Ulsan National Institute of Science and Technology (UNIST), Ulsan 44919, South Korea Corresponding author: Sam H. Noh ([email protected]) This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea Government (MSIT) under Grant NRF-2019R1A2C2009476. ABSTRACT The use of flash based Solid State Drives (SSDs) has expanded rapidly into the cloud computing environment. In cloud computing, ensuring the service level objective (SLO) of each server is the major criterion in designing a system. In particular, eliminating performance interference among virtual machines (VMs) on shared storage is a key challenge. However, studies on SSD performance to guarantee SLO in such environments are limited. In this paper, we present analysis of I/O behavior for a shared SSD as storage in terms of proportionality and stability. We show that performance SLOs of SSD based storage systems being shared by VMs or tasks are not satisfactory. We present and analyze the reasons behind the unexpected behavior through examining the components of SSDs such as channels, DRAM buffer, and Native Command Queuing (NCQ). We introduce two novel SSD-aware host level I/O schedulers on Linux, called ACCFQ and HCBFQ, based on our analysis and findings. Through experiments on Linux, we analyze I/O proportionality and stability in multi-tenant environments. In addition, through experiments using real workloads, we analyze the performance interference between workloads on a shared SSD. We then show that the proposed I/O schedulers almost eliminate the interference effect seen in CFQ and BFQ, while still providing I/O proportionality and stability for various I/O weighted scenarios. INDEX TERMS Flash memory based SSDs, cloud computing, virtual machines, I/O schedulers, I/O performance. I. INTRODUCTION and verification of the use of flash memory are needed [8], Flash memory semiconductors are increasingly being used [12], [13]. Although the use of flash memory based storage is as storage devices in smart devices, consumer electronics, rapidly increasing, studies on practical use cases in this area and various computing platforms [1]. Recently, the growth of are still insufficient. new industries such as Internet of Things (IoT), autonomous In particular, I/O performance is of critical importance for vehicles, big-data analysis and cloud computing is expected cloud service providers [14]: ``Amazon found every 100ms to further increase the use of flash memory [2]–[7]. of latency cost them 1% in sales. Google found an extra Although flash memory based storage devices have various 0.5 seconds in search page generation time dropped traf- advantages such as high performance and low power con- fic by 20%. A broker could lose $4 million in revenues sumption, performance variation may occur due to physical per millisecond if their electronic trading platform is characteristics of storage media, sophisticated internal mech- 5 milliseconds behind the competition.'' Virtualization has anisms and complicated management software [8]–[11]. been a key technology in providing enhanced I/O perfor- Therefore, in areas where stable and consistent I/O perfor- mance to customers. To further satisfy customers, it is imper- mance has a critical impact on the system, various analysis ative that virtualization platform providers satisfy the service level objectives (SLOs) of each virtual machine (VM). There- The associate editor coordinating the review of this manuscript and fore, we analyze the I/O performance and present solutions in approving it for publication was Xiao-Sheng Si . the cloud computing platform, where the use of flash memory This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see http://creativecommons.org/licenses/by/4.0/ VOLUME 8, 2020 4451 J. Kim et al.: I/O Schedulers for Proportionality and Stability on Flash-Based SSDs in Multi-Tenant Environments FIGURE 1. Proportionality of I/O throughput (y-axis) resulting from using Cgroup for various workloads on a shared SSD. (Notation: VM-x, where x is the I/O weight used to assign Cgroup proportion with higher x meaning more throughput is requested). based SSDs has greatly increased but where research in the performance impact of the internal components of the area is still limited. flash SSDs as shared storage environment [16]. In this work, we empirically investigate internal compo- • We find that the Linux I/O schedulers do not meet the nents of an SSD among VMs (or tasks) sharing the SSDs as SLO of each user in a shared SSD. Then, we present storage in terms of I/O proportionality and stability. Through two I/O schedulers, ACCFQ and HCBFQ, which are the analysis, we show the performance characteristics of designed to satisfy SLO requirements, based on the SSDs in a shared storage environment and figure out the rea- lessons learned from the analysis. sons of performance interference among VMs by examining • We extensively evaluate the existing and proposed I/O the components of an SSD such as channels, DRAM buffer, schedulers through various workloads in terms of I/O and Native Command Queueing (NCQ). From the results, performance, proportionality, and stability on Linux we learn lessons about how these components influence SLO kernel 3.18.x. satisfaction of VMs or tasks. In particular, we find that the • We also evaluate and analyze the performance interfer- current Linux default I/O schedulers supporting I/O weights ence between tasks competing on a shared SSD. do not satisfy I/O proportionality requirements as requested The remainder of this paper is organized as follows. by users and do not provide I/O stability due to performance In the next section, we present our motivation and summarize interference between VMs or tasks. related work. In Section III, we perform experiments and ana- I/O proportionality here refers to the relation among the lyze I/O proportionality on a shared storage. We discuss and I/O throughput as observed by VMs (or tasks) competing for present the design of the ACCFQ and HCBFQ I/O schedulers I/O resources in a shared storage environment. I/O is said in SectionsIV and V, respectively. In Section VI, we dis- to be proportional if each VM observes I/O throughput in cuss evaluation results. In addition, we analyze the perfor- proportion to the relative weight of I/O resources that the mance interference between concurrently running workloads VMs request [15]. I/O disproportionality and instability are on SSDs in Section VII. Finally, in Section VIII, we conclude particularly noticeable in fast storage devices such as SSDs. the paper. We find that components of SSDs, the NCQ and the cache in the SSD controller, hurt I/O proportionality and II. MOTIVATION AND RELATED WORK stability. Based on our analysis, we solve the SLO problem In this section, we present the motivation of this work by with two novel SSD-aware host level I/O schedulers [16], conducting experiments. Then, an overview of flash memory called ACCFQ and HCBFQ, that extend the CFQ (Com- is presented, followed by a discussion on related work. plete Fairness Queueing) [17] and BFQ (Budget Fair Queue- ing) [18] schedulers; these are the only two I/O schedulers A. DISPROPORTIONAL I/O PERFORMANCE that provide I/O proportionality on Linux. We show sig- Figure 1 depicts the motivation of this work, which shows the nificant improvements in I/O performance proportionality disproportionality of I/O throughput of four VMs running the and stability through minimizing performance interference same workload simultaneously for six different workloads. between VMs with the proposed I/O schedulers. Then we Using Cgroup [19] in Linux, we assign different I/O weights experimentally analyze the interference effects between con- to the VMs denoted as VM-x, where x refers to the I/O currently executing tasks in terms of I/O throughput, latency, weight, and observe the I/O throughput allotted and measured and proportionality with the characteristics of the workloads. by each VM. Here, higher I/O weight means we expect to In existing I/O schedulers, severe performance interference be allotted higher bandwidth by Cgroup proportional to the between tasks is observed. We show that the proposed sched- x value. Cgroup is known to match well in proportion to ulers eliminate performance interference almost completely. CPU and memory resources as well as HDDs [20]. However, The primary contributions of this paper are as follows: the effectiveness of Cgroup with flash based SSDs has not • We empirically examine internal components of a flash been explored much. Currently, we are aware of only one based SSD shared by multiple VMs or tasks in terms study [21] in this regard, which we review in Section II-C. of I/O performance and proportionality. To the best of To conduct experiments, we create VMs, which are our knowledge, our study is the first to analyze the kernel-based virtual machines (KVM), and run the same 4452 VOLUME 8, 2020 J. Kim et al.: I/O Schedulers for Proportionality and Stability on Flash-Based SSDs in Multi-Tenant Environments TABLE 1. KVM environment. TABLE 2. Characteristics of I/O workloads. FIGURE 2. Proportionality of I/O performance relative to VM-1 for various workloads. Also, it is interesting to note that though the MSN, Prj, and Prn workload results in Figure 1 show frequent, high spikes and reversals of proportionality, the averaged out proportion- ality results in Figure 2 appear not be as disproportionate.
Load more

Recommended publications
  • Self-Learning Disk Scheduling
    50 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 21, NO. 1, JANUARY 2009 Self-Learning Disk Scheduling Yu Zhang and Bharat Bhargava, Fellow, IEEE Abstract—The performance of disk I/O schedulers is affected by many factors such as workloads, file systems, and disk systems. Disk scheduling performance can be improved by tuning scheduler parameters such as the length of read timers. Scheduler performance tuning is mostly done manually. To automate this process, we propose four self-learning disk scheduling schemes: Change-sensing Round-Robin, Feedback Learning, Per-request Learning, and Two-layer Learning. Experiments show that the novel Two-layer Learning Scheme performs best. It integrates the workload-level and request-level learning algorithms. It employs feedback learning techniques to analyze workloads, change scheduling policy, and tune scheduling parameters automatically. We discuss schemes to choose features for workload learning, divide and recognize workloads, generate training data, and integrate machine learning algorithms into the Two-layer Learning Scheme. We conducted experiments to compare the accuracy, performance, and overhead of five machine learning algorithms: decision tree, logistic regression, naı¨ve Bayes, neural network, and support vector machine algorithms. Experiments with real-world and synthetic workloads show that self-learning disk scheduling can adapt to a wide variety of workloads, file systems, disk systems, and user preferences. It outperforms existing disk schedulers by as much as 15.8 percent while consuming less than 3 percent - 5 percent of CPU time. Index Terms—Machine learning, application-transparent adaptation, I/O, operating system. Ç 1INTRODUCTION UE to the physical limitations such as time-consuming selection, and parameter tuning.
    [Show full text]
  • A New I/O Scheduler for Solid State Devices
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Texas A&M Repository A NEW I/O SCHEDULER FOR SOLID STATE DEVICES A Thesis by MARCUS PAUL DUNN Submitted to the Office of Graduate Studies of Texas A&M University in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE August 2009 Major Subject: Computer Engineering A NEW I/O SCHEDULER FOR SOLID STATE DEVICES A Thesis by MARCUS PAUL DUNN Submitted to the Office of Graduate Studies of Texas A&M University in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE Approved by: Chair of Committee, Narasimha Annapareddy Committee Members, Sunil P. Khatri Riccardo Bettati Head of Department, Costas N. Georghiades August 2009 Major Subject: Computer Engineering iii ABSTRACT A New I/O Scheduler for Solid State Devices. (August 2009) Marcus Paul Dunn, B.S., Texas A&M University Chair of Advisory Committee: Dr. Narasimha Annapareddy Since the emergence of solid state devices onto the storage scene, improvements in capacity and price have brought them to the point where they are becoming a viable alternative to traditional magnetic storage for some applications. Current file system and device level I/O scheduler design is optimized for rotational magnetic hard disk drives. Since solid state devices have drastically different properties and structure, we may need to rethink the design of some aspects of the file system and scheduler levels of the I/O subsystem. In this thesis, we consider the current approach to I/O scheduling and show that the current scheduler design may not be ideally suited to solid state devices.
    [Show full text]
  • Predicting Disk Scheduling Performance with Virtual Machines
    Predicting Disk Scheduling Performance with Virtual Machines Robert Geist, Zachary H. Jones, James Westall Clemson University Abstract. A method for predicting the performance of disk schedul- ing algorithms on real machines using only their performance on virtual machines is suggested. The method uses a dynamically loaded kernel in- tercept probe (iprobe) to adjust low-level virtual device timing to match that of a simple model derived from the real device. An example is pro- vided in which the performance of a newly proposed disk scheduling algorithm is compared with that of standard Linux algorithms. The ad- vantage of the proposed method is that reasonable performance predic- tions may be made without dedicated measurement platforms and with only relatively limited knowledge of the performance characteristics of the targeted devices. 1 Introduction In the last five years, the use of virtual computing systems has grown rapidly, from nearly non-existent to commonplace. Nevertheless, system virtualization has been of interest to the computing community since at least the mid-1960s, when IBM developed the CP/CMS (Control Program/Conversational Monitor System or Cambridge Monitor System) for the IBM 360/67 [1]. In this design, a low-level software system called a hypervisor or virtual machine monitor sits between the hardware and multiple guest operating systems, each of which runs unmodified. The hypervisor handles scheduling and memory management. Priv- ileged instructions, those that trap if executed in user mode, are simulated by the hypervisor’s trap handlers when executed by a guest OS. The specific architecture of the host machine essentially determines the dif- ficulty of constructing such a hypervisor.
    [Show full text]
  • Alleviation of Disk I/O Contention in Virtualized Settings for Data-Intensive Computing
    Alleviation of Disk I/O Contention in Virtualized Settings for Data-Intensive Computing Matthew Malensek, Sangmi Lee Pallickara, and Shrideep Pallickara Department of Computer Science Colorado State University, Fort Collins, Colorado, USA Email: fmalensek, sangmi, [email protected] Abstract—Steady growth in storage and processing capabilities Disk I/O contention incurs higher costs in terms of latency has led to the accumulation of large-scale datasets that contain and throughput than either CPU or memory contention, and valuable insight into the interactions of complex systems, long- adversely impacts completion times and throughput for data- and short-term trends, and real-world phenomena. Converged in- frastructure, operating on cloud deployments and private clusters, intensive tasks. This stems from the nature of the underly- has emerged as an energy-efficient and cost-effective means of ing storage technology and associated seek times of HDDs, coping with these computing demands. However, increased collo- which are six orders of magnitude slower than CPU clock cation of storage and processing activities often leads to greater cycles. With mechanical disks, processes often contend for contention for resources in high-use situations. This issue is a single disk head. Stable storage must also satisfy different particularly pronounced when running distributed computations (such as MapReduce applications), because overall execution performance objectives; for example, in the case of HDDs the times are dependent on the completion time of the slowest task(s). objective of disk scheduling algorithms is to minimize disk In this study, we propose a framework that makes opinionated head movements, while in the case of SSDs the objective disk scheduling decisions to ensure high throughput for tasks is to minimize write amplifications and make considerations that use I/O resources conservatively, while still maintaining for wear leveling.
    [Show full text]
  • Improving Disk I/O Performance on Linux Master Thesis Carl Henrik
    UNIVERSITY OF OSLO Department of Informatics Improving Disk I/O Performance on Linux Master thesis Carl Henrik Lunde May 2009 Contents 1 Introduction1 1.1 Background and Motivation...................1 1.2 Problem Definition........................1 1.3 Main Contributions........................2 1.4 Outline...............................3 2 Overview of Disk Storage5 2.1 The Rotating Disk Drive.....................6 2.2 The I/O Scheduler........................9 2.2.1 Basic I/O Scheduling Algorithms...........9 2.2.2 Existing I/O Schedulers in Linux............ 10 The Deadline I/O Scheduler.............. 10 Anticipatory I/O Scheduler (AS)............ 12 Complete Fairness Queueing (CFQ) I/O Scheduler. 14 2.2.3 Performance of Linux Schedulers............ 16 Analyzing and Visualization Tool........... 16 Using the Visualization................. 18 The Base Workload.................... 22 Deadline I/O Scheduler Performance......... 23 Anticipatory I/O Scheduler (AS) Performance.... 25 Complete Fairness Queueing (CFQ) Performance.. 27 2.3 The ext4 File System....................... 33 2.3.1 The Virtual File System................. 33 2.3.2 Disk Layout........................ 34 Group Descriptors.................... 34 Inode and Block Bitmaps................ 35 Virtual Block Groups (flex bg)............. 35 2.3.3 Inodes........................... 35 2.3.4 Directories......................... 36 2.3.5 Large, Indexed Directories................ 36 2.3.6 Name Look-up...................... 39 i 2.3.7 Extents........................... 39 2.3.8 Allocation......................... 42 Inode Allocation..................... 42 Extent Allocation..................... 42 Dirent Allocation..................... 44 2.3.9 Journaling......................... 45 2.3.10 Readahead and Caching................. 46 File Data Readahead................... 46 Inode Readahead..................... 47 2.3.11 Implications of the File System Design......... 48 Metadata / Inodes.................... 49 File Data Placement................... 49 Evaluation of Data Ordering Cost..........
    [Show full text]
  • A New I/O Scheduler for Solid State Devices
    A NEW I/O SCHEDULER FOR SOLID STATE DEVICES A Thesis by MARCUS PAUL DUNN Submitted to the Office of Graduate Studies of Texas A&M University in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE August 2009 Major Subject: Computer Engineering A NEW I/O SCHEDULER FOR SOLID STATE DEVICES A Thesis by MARCUS PAUL DUNN Submitted to the Office of Graduate Studies of Texas A&M University in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE Approved by: Chair of Committee, Narasimha Annapareddy Committee Members, Sunil P. Khatri Riccardo Bettati Head of Department, Costas N. Georghiades August 2009 Major Subject: Computer Engineering iii ABSTRACT A New I/O Scheduler for Solid State Devices. (August 2009) Marcus Paul Dunn, B.S., Texas A&M University Chair of Advisory Committee: Dr. Narasimha Annapareddy Since the emergence of solid state devices onto the storage scene, improvements in capacity and price have brought them to the point where they are becoming a viable alternative to traditional magnetic storage for some applications. Current file system and device level I/O scheduler design is optimized for rotational magnetic hard disk drives. Since solid state devices have drastically different properties and structure, we may need to rethink the design of some aspects of the file system and scheduler levels of the I/O subsystem. In this thesis, we consider the current approach to I/O scheduling and show that the current scheduler design may not be ideally suited to solid state devices. We also present a framework for extracting some device parameters of solid state drives.
    [Show full text]
  • Alleviation of Disk I/O Contention in Virtualized Settings for Data-Intensive Computing
    Alleviation of Disk I/O Contention in Virtualized Settings for Data-Intensive Computing Matthew Malensek, Sangmi Lee Pallickara, and Shrideep Pallickara Department of Computer Science Colorado State University, Fort Collins, Colorado, USA Email: fmalensek, sangmi, [email protected] Abstract—Steady growth in storage and processing capabilities Disk I/O contention incurs higher costs in terms of latency has led to the accumulation of large-scale datasets that contain and throughput than either CPU or memory contention, and valuable insight into the interactions of complex systems, long- adversely impacts completion times and throughput for data- and short-term trends, and real-world phenomena. Converged in- frastructure, operating on cloud deployments and private clusters, intensive tasks. This stems from the nature of the underly- has emerged as an energy-efficient and cost-effective means of ing storage technology and associated seek times of HDDs, coping with these computing demands. However, increased collo- which are six orders of magnitude slower than CPU clock cation of storage and processing activities often leads to greater cycles. With mechanical disks, processes often contend for contention for resources in high-use situations. This issue is a single disk head. Stable storage must also satisfy different particularly pronounced when running distributed computations (such as MapReduce applications), because overall execution performance objectives; for example, in the case of HDDs the times are dependent on the completion time of the slowest task(s). objective of disk scheduling algorithms is to minimize disk In this study, we propose a framework that makes opinionated head movements, while in the case of SSDs the objective disk scheduling decisions to ensure high throughput for tasks is to minimize write amplifications and make considerations that use I/O resources conservatively, while still maintaining for wear leveling.
    [Show full text]
  • Essential Roles of Exploiting Internal Parallelism of Flash Memory Based Solid State Drives in High-Speed Data Processing
    Essential Roles of Exploiting Internal Parallelism of Flash Memory based Solid State Drives in High-Speed Data Processing Feng Chen2∗ Rubao Lee1,2 Xiaodong Zhang2 1Institute of Computing Technology 2Dept. of Computer Science & Engineering Chinese Academy of Sciences The Ohio State University [email protected] {fchen, zhang}@cse.ohio-state.edu Abstract speed data processing, due to the excessively high latency Flash memory based solid state drives (SSDs) have of HDDs for random data accesses and low throughput of shown a great potential to change storage infrastructure HDDs for handling multiple concurrent requests. Recently fundamentally through their high performance and low Flash Memory based Solid State Drives (SSDs) have been power. Most recent studies have mainly focused on address- incorporated into storage systems. Unlike HDDs, an SSD ing the technical limitations caused by special requirements is built entirely of semiconductor chips with no moving for writes in flash memory. However, a unique merit of an parts. Such an architectural difference provides a great po- SSD is its rich internal parallelism, which allows us to offset tential to address fundamentally the technical issues of ro- for the most part of the performance loss related to techni- tating media. In fact, researchers have made extensive ef- cal limitations by significantly increasing data processing forts to adopt this solid state storage technology in storage throughput. systems and proposed solutions for performance optimiza- In this work we present a comprehensive study of essen- tions (e.g. [8,18,27,31]). Most of the work has focused tial roles of internal parallelism of SSDs in high-speed data on leveraging the high random data access performance of processing.
    [Show full text]
  • Enhancements to Linux I/O Scheduling
    Enhancements to Linux I/O Scheduling Seetharami Seelam, Rodrigo Romero, Patricia Teller University of Texas at El Paso {seelam, romero, pteller}@cs.utep.edu Bill Buros IBM Linux Technology Center [email protected] Abstract the literature and this study reports, no one scheduler can provide the best possible perfor- mance for all workloads; accordingly, Linux The Linux 2.6 release provides four disk I/O provides four I/O schedulers from which to se- schedulers: deadline, anticipatory, noop, and lect. Even when dealing with just four, in sys- completely fair queuing (CFQ), along with an tems that service concurrent workloads with option to select one of these four at boot time different I/O behaviors, a priori selection of or runtime. The selection is based on a pri- the scheduler with the best possible perfor- ori knowledge of the workload, file system, mance can be an intricate task. Dynamic se- and I/O system hardware configuration, among lection based on workload needs, system con- other factors. The anticipatory scheduler (AS) figuration, and other parameters can address is the default. Although the AS performs this challenge. Accordingly, we are developing well under many situations, we have identi- metrics and heuristics that can be used for this fied cases, under certain combinations of work- purpose. The paper concludes with a descrip- loads, where the AS leads to process starvation. tion of our efforts in this direction, in particular, To mitigate this problem, we implemented an we present a characterization function, based extension to the AS (called Cooperative AS or on metrics related to system behavior and I/O CAS) and compared its performance with the requests, that can be used to measure and com- other four schedulers.
    [Show full text]
  • A Qos Aware Non-Work-Conserving Disk Scheduler
    A QoS Aware Non-work-conserving Disk Scheduler Pedro Eugenioˆ Rocha Luis C. E. Bona Federal University of Parana,´ Brazil Federal University of Parana,´ Brazil [email protected] [email protected] Abstract—Disk schedulers should provide QoS guarantees to these schedulers is that a request that is soon to arrive might applications, thus sharing proportionally the storage resource and be closer to the current disk head position than other pending enforcing performance isolation. Disk schedulers must execute requests. If the seek time needed to serve other pending requests in an efficient order though, preventing poor disk usage. Non-work-conserving disk schedulers help to increase disk requests is greater then the cost of keeping the disk idle while throughput by predicting future requests’ arrival and therefore waiting for future requests (assuming that such request does exploiting disk spatial locality. Previous work are limited to either arrive), than the idle waiting is justified. provide QoS guarantees or exploit disk spatial locality. In this This performance gain is commonly observed when the paper, we propose a new non-work-conserving disk scheduler scheduler must deal with concurrent applications issuing syn- called High-throughput Token Bucket Scheduler (HTBS), which can provide both QoS guarantees and high throughput by (a) chronous requests. In the meanwhile between synchronous assigning tags to requests in a fair queuing-like fashion and (b) requests, a work-conserving scheduler would serve a pending predicting future requests’ arrival. We show through experiments request from another application, thus losing spatial locality. with our Linux Kernel implementation that HTBS outperforms This behavior, commonly known as deceptive idleness [7], previous QoS aware work-conserving disk schedulers throughput causes unnecessary seek time and harms disk performance.
    [Show full text]
  • Self-Tuning of Disk Input–Output in Operating Systems
    The Journal of Systems and Software 85 (2012) 77–86 Contents lists available at ScienceDirect The Journal of Systems and Software journal homepage: www.elsevier.com/locate/jss Self-tuning of disk input–output in operating systems A. Santos a, J. Romero a,∗, J. Taibo b, C. Rodriguez c, A. Carballal a a Artificial Neural Networks and Adaptive Systems LAB, University of A Coru˜na, A Coru˜na, Spain b VideaLAB, University of A Coru˜na, A Coru˜na, Spain c Computing and Communications Service, University of A Coru˜na, A Coru˜na, Spain article info abstract Article history: One of the most difficult and hard to learn tasks in computer system management is tuning the kernel Received 18 October 2010 parameters in order to get the maximum performance. Traditionally, this tuning has been set using Received in revised form 23 May 2011 either fixed configurations or the subjective administrator’s criteria. The main bottleneck among the Accepted 13 July 2011 subsystems managed by the operating systems is disk input/output (I/O). An evolutionary module has Available online 26 July 2011 been developed to perform the tuning of this subsystem automatically, using an adaptive and dynamic approach. Any computer change, both at the hardware level, and due to the nature of the workload itself, Keywords: will make our module adapt automatically and in a transparent way. Thus, system administrators are Operating system Genetic algorithms released from this kind of task and able to achieve some optimal performances adapted to the framework Evolutionary computation of each of their systems. The experiment made shows a productivity increase in 88.2% of cases and an IO optimization average improvement of 29.63% with regard to the default configuration of the Linux operating system.
    [Show full text]
  • NCQ Vs. I/O Scheduler
    NCQ vs. I/O Scheduler: Preventing Unexpected Misbehaviors 2 YOUNG JIN YU, DONG IN SHIN, HYEONSANG EOM, and HEON YOUNG YEOM Distributed Computing System Lab., Seoul National University Native Command Queueing (NCQ) is an optimization technology to maximize throughput by re- ordering requests inside a disk drive. It has been so successful that NCQ has become the standard in SATA 2 protocol specification, and the great majority of disk vendors have adopted it for their recent disks. However, there is a possibility that the technology may lead to an information gap between the OS and a disk drive. A NCQ-enabled disk tries to optimize throughput without realizing the in- tention of an OS, whereas the OS does its best under the assumption that the disk will do as it is told without specific knowledge regarding the details of the disk mechanism. Let us call this expectation discord, which may cause serious problems such as request starvations or performance anomaly. In this article, we (1) confirm that expectation discord actually occurs in real systems; (2) propose software-level approaches to solve them; and (3) evaluate our mechanism. Experimental results show that our solution is simple, cheap (no special hardware required), portable, and effective. Categories and Subject Descriptors: C.4 [Computer Systems Organization]: Performance of Systems—Performance attributes; D.4.2 [Operating Systems]: Storage Management—Secondary storage; D.4.8 [Operating Systems]: Performance—Measurements General Terms: Measurement, Performance Additional Key Words and Phrases: NCQ, SATA 2, hybrid scheduling, starvation detection, I/O prioritization ACM Reference Format: Yu, Y. J., Shin, D.
    [Show full text]