DOCSLIB.ORG

Predicting Disk Scheduling Performance with Virtual Machines

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Predicting Disk Scheduling Performance with Virtual Machines Predicting Disk Scheduling Performance with Virtual Machines Robert Geist, Zachary H. Jones, James Westall Clemson University Abstract. A method for predicting the performance of disk schedul- ing algorithms on real machines using only their performance on virtual machines is suggested. The method uses a dynamically loaded kernel in- tercept probe (iprobe) to adjust low-level virtual device timing to match that of a simple model derived from the real device. An example is pro- vided in which the performance of a newly proposed disk scheduling algorithm is compared with that of standard Linux algorithms. The ad- vantage of the proposed method is that reasonable performance predic- tions may be made without dedicated measurement platforms and with only relatively limited knowledge of the performance characteristics of the targeted devices. 1 Introduction In the last five years, the use of virtual computing systems has grown rapidly, from nearly non-existent to commonplace. Nevertheless, system virtualization has been of interest to the computing community since at least the mid-1960s, when IBM developed the CP/CMS (Control Program/Conversational Monitor System or Cambridge Monitor System) for the IBM 360/67 [1]. In this design, a low-level software system called a hypervisor or virtual machine monitor sits between the hardware and multiple guest operating systems, each of which runs unmodified. The hypervisor handles scheduling and memory management. Priv- ileged instructions, those that trap if executed in user mode, are simulated by the hypervisor’s trap handlers when executed by a guest OS. The specific architecture of the host machine essentially determines the dif- ficulty of constructing such a hypervisor. Popek and Goldberg [2] characterize as sensitive those machine instructions that may modify or read resource config- uration data. They show that an architecture is most readily virtualized if the sensitive instructions are a subset of the privileged instructions. The principal roadblock to widespread deployment of virtual systems has been the basic Intel x86 architecture, the de facto standard, in which a rela- tively large collection of instructions are sensitive but not privileged. Thus a guest OS running at privilege level 3 may execute one of them without gen- erating a trap that would allow the hypervisor to virtualize the effect of the instruction. A detailed analysis of the challenges to virtualization presented by these instructions is given by Robin and Irvine [3]. 2 Robert Geist, Zachary H. Jones, James Westall The designers of VMWare provided the first solution to this problem by us- ing a binary translation of guest OS code [4]. Xen [5] provided an open-source virtualization of the x86 using para-virtualization, in which the hypervisor pro- vides a virtual machine interface that is similar to the hardware interface but avoids the instructions whose virtualization would be problematic. Each guest OS must then be modified to run on the virtual machine interface. The true catalysts to the widespread development and deployment of virtual machines appeared in 2005-2006 as extensions to the basic x86 architecture, the Intel VT-x and AMD-V. The extensions include a “guest” operating mode, which carries all the privilege levels of the normal operating mode, except that system software can request that certain instructions be trapped. The hardware state switch to/from guest mode includes control registers, segment registers, and instruction pointer. Exit from guest mode includes the cause of the exit [6]. These extensions have allowed the development of a full virtualization Xen, in which the guest operating systems can run unmodified, and the Kernel-based Virtual Machine (KVM), which uses a standard Linux kernel as hypervisor. Nevertheless, VMWare, Xen, and KVM are principally aimed at facilitating user-level applications, and thus they export only a fairly generic view of the guest operating system(s), in which the devices to be managed are taken from a small, fixed collection of emulated components. This would seem to preclude the use of virtual machines in testing system-level performance, such as in a comparative study of scheduling algorithms. The devices of interest are unlikely to be among those emulated, and, even if they are, the emulated devices may have arbitrary implementation, thus exhibiting performance characteristics that bear little or no resemblance to those of the real devices. For example, an entire virtual disk may be cached in the main memory of a large NAS device supporting the virtual machine, thus providing a disk with constant (O(1)) service times. In [7], we introduced the iprobe, an extremely light-weight extension to the Linux kernel that can be dynamically loaded and yet allows the interception and replacement of arbitrary kernel functions. The iprobe was seen to allow a straightforward implementation of PCI device emulators that could be dynam- ically loaded and yet were suitable for full, system-level driver design, develop- ment, and testing. The goal of this effort is to extend the use of the iprobe to allow performance prediction for real devices using only virtual machines. We suggested this possi- bility in [8]. In particular, we will predict the performance of a collection of disk scheduling algorithms, one of which is new, for a targeted file-server workload on a specific SCSI device. We then compare the results with measurements of the same algorithms on the real hardware. We will see that the advantage of our approach is that reasonable performance predictions may be made without dedicated measurement platforms and with relatively limited knowledge of the performance characteristics of the targeted devices. The remainder of the paper is organized as follows. In the next section we provide background on both kernel probes and disk scheduling. In section 3, we propose a new disk scheduling algorithm that will serve as the focus of our tests. Lecture Notes in Computer Science 3 In section 4, we show how the kernel probes may be used to implement a virtual performance throttle, the mechanism that allows performance prediction from virtual platforms. In section 5 we describe our virtual machine, the targeted (real) hardware, and a test workload. Section 6 contains results of algorithm performance prediction from the virtual machine and measurements of the same algorithms on the real hardware. Conclusions follow in section 7. 2 Background 2.1 Kernel Probes The Linux kernel probe or kprobe utility was designed to facilitate kernel debug- ging [10]. All kprobes have the same basic operation. A kprobe structure is ini- tialized, usually by a kernel module, i.e., a dynamically loaded kernel extension, to identify a target instruction and specify both pre-handler and post-handler functions. When the kprobe is registered, it saves the targeted instruction and replaces it with a breakpoint. When the breakpoint is hit, the pre-handler is executed, then the saved instruction is executed in single step mode, then the post-handler is executed. A return resumes execution after the breakpoint. A variation on the kprobe, also supplied with Linux, is the jprobe, or jump probe, which is intended for probing function calls, rather than arbitrary kernel instructions. It is a kprobe with a two-stage pre-handler and an empty post- handler. On registration, it copies the first instruction of the registered function and replaces that with the breakpoint. When this breakpoint is hit, the first- stage pre-handler, which is a fixed code sequence, is invoked. It copies both registers and stack, in addition to loading the saved instruction pointer with the address of the supplied, second-stage pre-handler. The second-stage pre-handler then sees the same register values and stack as the original function. In [7], we introduced the intercept probe or iprobe, which is a modified jprobe. Our iprobe second-stage pre-handler decides whether or not to replace the origi- nal function. If it decides to do so, it makes a backup copy of the saved (function entry) instruction and then overwrites the saved instruction with a no-op. As is standard with a jprobe, the second-stage pre-handler then executes a jprobe return, which traps again to restore the original register values and stack. The saved instruction (which now could be a no-op) is then executed in single step mode. Next the post-handler runs. On a conventional jprobe, this is empty, but on the iprobe, the post-handler checks to see if replacement was called for by the second-stage pre-handler. If this is the case, the single-stepped instruction was a no-op. The registers and stack necessarily match those of the original function call. We simply load the instruction pointer with the address of the replacement function, restore the saved instruction from the backup copy (overwrite the no- op) and return. With this method, we can intercept and dynamically replace any kernel function of our choice. Note that it is possible to have two calls to the same probed function, one that is to be intercepted and one that is not. A discussion of the handling of potential attendant race conditions in SMP systems may be found in [9]. 4 Robert Geist, Zachary H. Jones, James Westall 2.2 Disk Scheduling Scheduling algorithms that re-order pending requests for disks have been studied for at least four decades. Such scheduling algorithms represent a particularly attractive area for investigation in that the algorithms are not constrained to be work-conserving. Further, it is easy to dismiss naive (but commonly held) beliefs about such scheduling, in particular, that a greedy or shortest-access-time-first algorithm will deliver performance that is optimal with respect to any common performance measure, such as mean service time or mean response time. Consider a hypothetical system in which requests are identified by their starting blocks and service time between blocks is equal to distance.
Load more

Recommended publications
  • Self-Learning Disk Scheduling
    50 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 21, NO. 1, JANUARY 2009 Self-Learning Disk Scheduling Yu Zhang and Bharat Bhargava, Fellow, IEEE Abstract—The performance of disk I/O schedulers is affected by many factors such as workloads, file systems, and disk systems. Disk scheduling performance can be improved by tuning scheduler parameters such as the length of read timers. Scheduler performance tuning is mostly done manually. To automate this process, we propose four self-learning disk scheduling schemes: Change-sensing Round-Robin, Feedback Learning, Per-request Learning, and Two-layer Learning. Experiments show that the novel Two-layer Learning Scheme performs best. It integrates the workload-level and request-level learning algorithms. It employs feedback learning techniques to analyze workloads, change scheduling policy, and tune scheduling parameters automatically. We discuss schemes to choose features for workload learning, divide and recognize workloads, generate training data, and integrate machine learning algorithms into the Two-layer Learning Scheme. We conducted experiments to compare the accuracy, performance, and overhead of five machine learning algorithms: decision tree, logistic regression, naı¨ve Bayes, neural network, and support vector machine algorithms. Experiments with real-world and synthetic workloads show that self-learning disk scheduling can adapt to a wide variety of workloads, file systems, disk systems, and user preferences. It outperforms existing disk schedulers by as much as 15.8 percent while consuming less than 3 percent - 5 percent of CPU time. Index Terms—Machine learning, application-transparent adaptation, I/O, operating system. Ç 1INTRODUCTION UE to the physical limitations such as time-consuming selection, and parameter tuning.
    [Show full text]
  • A New I/O Scheduler for Solid State Devices
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Texas A&M Repository A NEW I/O SCHEDULER FOR SOLID STATE DEVICES A Thesis by MARCUS PAUL DUNN Submitted to the Office of Graduate Studies of Texas A&M University in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE August 2009 Major Subject: Computer Engineering A NEW I/O SCHEDULER FOR SOLID STATE DEVICES A Thesis by MARCUS PAUL DUNN Submitted to the Office of Graduate Studies of Texas A&M University in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE Approved by: Chair of Committee, Narasimha Annapareddy Committee Members, Sunil P. Khatri Riccardo Bettati Head of Department, Costas N. Georghiades August 2009 Major Subject: Computer Engineering iii ABSTRACT A New I/O Scheduler for Solid State Devices. (August 2009) Marcus Paul Dunn, B.S., Texas A&M University Chair of Advisory Committee: Dr. Narasimha Annapareddy Since the emergence of solid state devices onto the storage scene, improvements in capacity and price have brought them to the point where they are becoming a viable alternative to traditional magnetic storage for some applications. Current file system and device level I/O scheduler design is optimized for rotational magnetic hard disk drives. Since solid state devices have drastically different properties and structure, we may need to rethink the design of some aspects of the file system and scheduler levels of the I/O subsystem. In this thesis, we consider the current approach to I/O scheduling and show that the current scheduler design may not be ideally suited to solid state devices.
    [Show full text]
  • I/O Schedulers for Proportionality and Stability on Flash-Based Ssds in Multi-Tenant Environments
    Received November 19, 2019, accepted December 10, 2019, date of publication December 30, 2019, date of current version January 8, 2020. Digital Object Identifier 10.1109/ACCESS.2019.2963081 I/O Schedulers for Proportionality and Stability on Flash-Based SSDs in Multi-Tenant Environments JAEHO KIM 1, EUNJAE LEE 2, AND SAM H. NOH 2, (Senior Member, IEEE) 1Department of Electrical and Computer Engineering, Virginia Tech, Blacksburg, VA 24061, USA 2School of Electrical and Computer Engineering, Ulsan National Institute of Science and Technology (UNIST), Ulsan 44919, South Korea Corresponding author: Sam H. Noh ([email protected]) This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea Government (MSIT) under Grant NRF-2019R1A2C2009476. ABSTRACT The use of flash based Solid State Drives (SSDs) has expanded rapidly into the cloud computing environment. In cloud computing, ensuring the service level objective (SLO) of each server is the major criterion in designing a system. In particular, eliminating performance interference among virtual machines (VMs) on shared storage is a key challenge. However, studies on SSD performance to guarantee SLO in such environments are limited. In this paper, we present analysis of I/O behavior for a shared SSD as storage in terms of proportionality and stability. We show that performance SLOs of SSD based storage systems being shared by VMs or tasks are not satisfactory. We present and analyze the reasons behind the unexpected behavior through examining the components of SSDs such as channels, DRAM buffer, and Native Command Queuing (NCQ). We introduce two novel SSD-aware host level I/O schedulers on Linux, called ACCFQ and HCBFQ, based on our analysis and findings.
    [Show full text]
  • Alleviation of Disk I/O Contention in Virtualized Settings for Data-Intensive Computing
    Alleviation of Disk I/O Contention in Virtualized Settings for Data-Intensive Computing Matthew Malensek, Sangmi Lee Pallickara, and Shrideep Pallickara Department of Computer Science Colorado State University, Fort Collins, Colorado, USA Email: fmalensek, sangmi, [email protected] Abstract—Steady growth in storage and processing capabilities Disk I/O contention incurs higher costs in terms of latency has led to the accumulation of large-scale datasets that contain and throughput than either CPU or memory contention, and valuable insight into the interactions of complex systems, long- adversely impacts completion times and throughput for data- and short-term trends, and real-world phenomena. Converged in- frastructure, operating on cloud deployments and private clusters, intensive tasks. This stems from the nature of the underly- has emerged as an energy-efficient and cost-effective means of ing storage technology and associated seek times of HDDs, coping with these computing demands. However, increased collo- which are six orders of magnitude slower than CPU clock cation of storage and processing activities often leads to greater cycles. With mechanical disks, processes often contend for contention for resources in high-use situations. This issue is a single disk head. Stable storage must also satisfy different particularly pronounced when running distributed computations (such as MapReduce applications), because overall execution performance objectives; for example, in the case of HDDs the times are dependent on the completion time of the slowest task(s). objective of disk scheduling algorithms is to minimize disk In this study, we propose a framework that makes opinionated head movements, while in the case of SSDs the objective disk scheduling decisions to ensure high throughput for tasks is to minimize write amplifications and make considerations that use I/O resources conservatively, while still maintaining for wear leveling.
    [Show full text]
  • Improving Disk I/O Performance on Linux Master Thesis Carl Henrik
    UNIVERSITY OF OSLO Department of Informatics Improving Disk I/O Performance on Linux Master thesis Carl Henrik Lunde May 2009 Contents 1 Introduction1 1.1 Background and Motivation...................1 1.2 Problem Definition........................1 1.3 Main Contributions........................2 1.4 Outline...............................3 2 Overview of Disk Storage5 2.1 The Rotating Disk Drive.....................6 2.2 The I/O Scheduler........................9 2.2.1 Basic I/O Scheduling Algorithms...........9 2.2.2 Existing I/O Schedulers in Linux............ 10 The Deadline I/O Scheduler.............. 10 Anticipatory I/O Scheduler (AS)............ 12 Complete Fairness Queueing (CFQ) I/O Scheduler. 14 2.2.3 Performance of Linux Schedulers............ 16 Analyzing and Visualization Tool........... 16 Using the Visualization................. 18 The Base Workload.................... 22 Deadline I/O Scheduler Performance......... 23 Anticipatory I/O Scheduler (AS) Performance.... 25 Complete Fairness Queueing (CFQ) Performance.. 27 2.3 The ext4 File System....................... 33 2.3.1 The Virtual File System................. 33 2.3.2 Disk Layout........................ 34 Group Descriptors.................... 34 Inode and Block Bitmaps................ 35 Virtual Block Groups (flex bg)............. 35 2.3.3 Inodes........................... 35 2.3.4 Directories......................... 36 2.3.5 Large, Indexed Directories................ 36 2.3.6 Name Look-up...................... 39 i 2.3.7 Extents........................... 39 2.3.8 Allocation......................... 42 Inode Allocation..................... 42 Extent Allocation..................... 42 Dirent Allocation..................... 44 2.3.9 Journaling......................... 45 2.3.10 Readahead and Caching................. 46 File Data Readahead................... 46 Inode Readahead..................... 47 2.3.11 Implications of the File System Design......... 48 Metadata / Inodes.................... 49 File Data Placement................... 49 Evaluation of Data Ordering Cost..........
    [Show full text]
  • A New I/O Scheduler for Solid State Devices
    A NEW I/O SCHEDULER FOR SOLID STATE DEVICES A Thesis by MARCUS PAUL DUNN Submitted to the Office of Graduate Studies of Texas A&M University in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE August 2009 Major Subject: Computer Engineering A NEW I/O SCHEDULER FOR SOLID STATE DEVICES A Thesis by MARCUS PAUL DUNN Submitted to the Office of Graduate Studies of Texas A&M University in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE Approved by: Chair of Committee, Narasimha Annapareddy Committee Members, Sunil P. Khatri Riccardo Bettati Head of Department, Costas N. Georghiades August 2009 Major Subject: Computer Engineering iii ABSTRACT A New I/O Scheduler for Solid State Devices. (August 2009) Marcus Paul Dunn, B.S., Texas A&M University Chair of Advisory Committee: Dr. Narasimha Annapareddy Since the emergence of solid state devices onto the storage scene, improvements in capacity and price have brought them to the point where they are becoming a viable alternative to traditional magnetic storage for some applications. Current file system and device level I/O scheduler design is optimized for rotational magnetic hard disk drives. Since solid state devices have drastically different properties and structure, we may need to rethink the design of some aspects of the file system and scheduler levels of the I/O subsystem. In this thesis, we consider the current approach to I/O scheduling and show that the current scheduler design may not be ideally suited to solid state devices. We also present a framework for extracting some device parameters of solid state drives.
    [Show full text]
  • Alleviation of Disk I/O Contention in Virtualized Settings for Data-Intensive Computing
    Alleviation of Disk I/O Contention in Virtualized Settings for Data-Intensive Computing Matthew Malensek, Sangmi Lee Pallickara, and Shrideep Pallickara Department of Computer Science Colorado State University, Fort Collins, Colorado, USA Email: fmalensek, sangmi, [email protected] Abstract—Steady growth in storage and processing capabilities Disk I/O contention incurs higher costs in terms of latency has led to the accumulation of large-scale datasets that contain and throughput than either CPU or memory contention, and valuable insight into the interactions of complex systems, long- adversely impacts completion times and throughput for data- and short-term trends, and real-world phenomena. Converged in- frastructure, operating on cloud deployments and private clusters, intensive tasks. This stems from the nature of the underly- has emerged as an energy-efficient and cost-effective means of ing storage technology and associated seek times of HDDs, coping with these computing demands. However, increased collo- which are six orders of magnitude slower than CPU clock cation of storage and processing activities often leads to greater cycles. With mechanical disks, processes often contend for contention for resources in high-use situations. This issue is a single disk head. Stable storage must also satisfy different particularly pronounced when running distributed computations (such as MapReduce applications), because overall execution performance objectives; for example, in the case of HDDs the times are dependent on the completion time of the slowest task(s). objective of disk scheduling algorithms is to minimize disk In this study, we propose a framework that makes opinionated head movements, while in the case of SSDs the objective disk scheduling decisions to ensure high throughput for tasks is to minimize write amplifications and make considerations that use I/O resources conservatively, while still maintaining for wear leveling.
    [Show full text]
  • Essential Roles of Exploiting Internal Parallelism of Flash Memory Based Solid State Drives in High-Speed Data Processing
    Essential Roles of Exploiting Internal Parallelism of Flash Memory based Solid State Drives in High-Speed Data Processing Feng Chen2∗ Rubao Lee1,2 Xiaodong Zhang2 1Institute of Computing Technology 2Dept. of Computer Science & Engineering Chinese Academy of Sciences The Ohio State University [email protected] {fchen, zhang}@cse.ohio-state.edu Abstract speed data processing, due to the excessively high latency Flash memory based solid state drives (SSDs) have of HDDs for random data accesses and low throughput of shown a great potential to change storage infrastructure HDDs for handling multiple concurrent requests. Recently fundamentally through their high performance and low Flash Memory based Solid State Drives (SSDs) have been power. Most recent studies have mainly focused on address- incorporated into storage systems. Unlike HDDs, an SSD ing the technical limitations caused by special requirements is built entirely of semiconductor chips with no moving for writes in flash memory. However, a unique merit of an parts. Such an architectural difference provides a great po- SSD is its rich internal parallelism, which allows us to offset tential to address fundamentally the technical issues of ro- for the most part of the performance loss related to techni- tating media. In fact, researchers have made extensive ef- cal limitations by significantly increasing data processing forts to adopt this solid state storage technology in storage throughput. systems and proposed solutions for performance optimiza- In this work we present a comprehensive study of essen- tions (e.g. [8,18,27,31]). Most of the work has focused tial roles of internal parallelism of SSDs in high-speed data on leveraging the high random data access performance of processing.
    [Show full text]
  • Enhancements to Linux I/O Scheduling
    Enhancements to Linux I/O Scheduling Seetharami Seelam, Rodrigo Romero, Patricia Teller University of Texas at El Paso {seelam, romero, pteller}@cs.utep.edu Bill Buros IBM Linux Technology Center [email protected] Abstract the literature and this study reports, no one scheduler can provide the best possible perfor- mance for all workloads; accordingly, Linux The Linux 2.6 release provides four disk I/O provides four I/O schedulers from which to se- schedulers: deadline, anticipatory, noop, and lect. Even when dealing with just four, in sys- completely fair queuing (CFQ), along with an tems that service concurrent workloads with option to select one of these four at boot time different I/O behaviors, a priori selection of or runtime. The selection is based on a pri- the scheduler with the best possible perfor- ori knowledge of the workload, file system, mance can be an intricate task. Dynamic se- and I/O system hardware configuration, among lection based on workload needs, system con- other factors. The anticipatory scheduler (AS) figuration, and other parameters can address is the default. Although the AS performs this challenge. Accordingly, we are developing well under many situations, we have identi- metrics and heuristics that can be used for this fied cases, under certain combinations of work- purpose. The paper concludes with a descrip- loads, where the AS leads to process starvation. tion of our efforts in this direction, in particular, To mitigate this problem, we implemented an we present a characterization function, based extension to the AS (called Cooperative AS or on metrics related to system behavior and I/O CAS) and compared its performance with the requests, that can be used to measure and com- other four schedulers.
    [Show full text]
  • A Qos Aware Non-Work-Conserving Disk Scheduler
    A QoS Aware Non-work-conserving Disk Scheduler Pedro Eugenioˆ Rocha Luis C. E. Bona Federal University of Parana,´ Brazil Federal University of Parana,´ Brazil [email protected] [email protected] Abstract—Disk schedulers should provide QoS guarantees to these schedulers is that a request that is soon to arrive might applications, thus sharing proportionally the storage resource and be closer to the current disk head position than other pending enforcing performance isolation. Disk schedulers must execute requests. If the seek time needed to serve other pending requests in an efficient order though, preventing poor disk usage. Non-work-conserving disk schedulers help to increase disk requests is greater then the cost of keeping the disk idle while throughput by predicting future requests’ arrival and therefore waiting for future requests (assuming that such request does exploiting disk spatial locality. Previous work are limited to either arrive), than the idle waiting is justified. provide QoS guarantees or exploit disk spatial locality. In this This performance gain is commonly observed when the paper, we propose a new non-work-conserving disk scheduler scheduler must deal with concurrent applications issuing syn- called High-throughput Token Bucket Scheduler (HTBS), which can provide both QoS guarantees and high throughput by (a) chronous requests. In the meanwhile between synchronous assigning tags to requests in a fair queuing-like fashion and (b) requests, a work-conserving scheduler would serve a pending predicting future requests’ arrival. We show through experiments request from another application, thus losing spatial locality. with our Linux Kernel implementation that HTBS outperforms This behavior, commonly known as deceptive idleness [7], previous QoS aware work-conserving disk schedulers throughput causes unnecessary seek time and harms disk performance.
    [Show full text]
  • Self-Tuning of Disk Input–Output in Operating Systems
    The Journal of Systems and Software 85 (2012) 77–86 Contents lists available at ScienceDirect The Journal of Systems and Software journal homepage: www.elsevier.com/locate/jss Self-tuning of disk input–output in operating systems A. Santos a, J. Romero a,∗, J. Taibo b, C. Rodriguez c, A. Carballal a a Artificial Neural Networks and Adaptive Systems LAB, University of A Coru˜na, A Coru˜na, Spain b VideaLAB, University of A Coru˜na, A Coru˜na, Spain c Computing and Communications Service, University of A Coru˜na, A Coru˜na, Spain article info abstract Article history: One of the most difficult and hard to learn tasks in computer system management is tuning the kernel Received 18 October 2010 parameters in order to get the maximum performance. Traditionally, this tuning has been set using Received in revised form 23 May 2011 either fixed configurations or the subjective administrator’s criteria. The main bottleneck among the Accepted 13 July 2011 subsystems managed by the operating systems is disk input/output (I/O). An evolutionary module has Available online 26 July 2011 been developed to perform the tuning of this subsystem automatically, using an adaptive and dynamic approach. Any computer change, both at the hardware level, and due to the nature of the workload itself, Keywords: will make our module adapt automatically and in a transparent way. Thus, system administrators are Operating system Genetic algorithms released from this kind of task and able to achieve some optimal performances adapted to the framework Evolutionary computation of each of their systems. The experiment made shows a productivity increase in 88.2% of cases and an IO optimization average improvement of 29.63% with regard to the default configuration of the Linux operating system.
    [Show full text]
  • NCQ Vs. I/O Scheduler
    NCQ vs. I/O Scheduler: Preventing Unexpected Misbehaviors 2 YOUNG JIN YU, DONG IN SHIN, HYEONSANG EOM, and HEON YOUNG YEOM Distributed Computing System Lab., Seoul National University Native Command Queueing (NCQ) is an optimization technology to maximize throughput by re- ordering requests inside a disk drive. It has been so successful that NCQ has become the standard in SATA 2 protocol specification, and the great majority of disk vendors have adopted it for their recent disks. However, there is a possibility that the technology may lead to an information gap between the OS and a disk drive. A NCQ-enabled disk tries to optimize throughput without realizing the in- tention of an OS, whereas the OS does its best under the assumption that the disk will do as it is told without specific knowledge regarding the details of the disk mechanism. Let us call this expectation discord, which may cause serious problems such as request starvations or performance anomaly. In this article, we (1) confirm that expectation discord actually occurs in real systems; (2) propose software-level approaches to solve them; and (3) evaluate our mechanism. Experimental results show that our solution is simple, cheap (no special hardware required), portable, and effective. Categories and Subject Descriptors: C.4 [Computer Systems Organization]: Performance of Systems—Performance attributes; D.4.2 [Operating Systems]: Storage Management—Secondary storage; D.4.8 [Operating Systems]: Performance—Measurements General Terms: Measurement, Performance Additional Key Words and Phrases: NCQ, SATA 2, hybrid scheduling, starvation detection, I/O prioritization ACM Reference Format: Yu, Y. J., Shin, D.
    [Show full text]