FLIN: Enabling Fairness and Enhancing Performance in Modern NVMe Solid State Drives Arash Tavakkol† Mohammad Sadrosadati† Saugata Ghose‡ Jeremie S. Kim‡† Yixin Luo‡ Yaohua Wang†§ Nika Mansouri Ghiasi† Lois Orosa†∗ Juan Gómez-Luna† Onur Mutlu†‡ †ETH Zürich ‡Carnegie Mellon University §NUDT ∗Unicamp
Modern solid-state drives (SSDs) use new host–interface pro- To overcome this bottleneck, modern enterprise SSDs (e.g., tocols, such as NVMe, to provide applications with fast access [30–33,65,66,87,103,104,110,111]) use new high-performance to storage. These new protocols make use of a concept known protocols, such as the Non-Volatile Memory Express (NVMe) as the multi-queue SSD (MQ-SSD), where the SSD has direct protocol [21, 83]. These new protocols make use of the multi- access to the application-level I/O request queues. This removes queue SSD (MQ-SSD) [8,48,101] concept, where multiple host- most of the OS software stack that was used in older protocols side I/O request queues (in software) are directly exposed to to control how and when the I/O requests were dispatched to the SSD controller. There are two benets to directly expos- storage devices. Unfortunately, while the elimination of the OS ing the request queues to the SSD controller: (1) there is no software stack leads to a signicant performance improvement, longer any need for the OS software stack to manage the I/O we show in this paper that it introduces a new problem: unfair- requests; and (2) the SSD can make more e ective I/O request ness. This is because the elimination of the OS software stack scheduling decisions than the OS, since the SSD knows ex- eliminates the mechanisms that were used to provide fairness actly which of its internal resources are available or are busy among applications in older SSDs. serving other requests. Thus, the protocols eliminate the OS To study application-level unfairness, we perform experiments software stack, enabling MQ-SSDs to provide signicantly using four real state-of-the-art MQ-SSDs. We demonstrate that higher performance than traditional SSDs [101, 114]. the lack of fair scheduling mechanisms leads to high unfairness Unfortunately, eliminating the OS software stack also elim- among concurrently-executing applications due to the interfer- inates critical mechanisms that were previously implemented ence among them. For instance, when one of these applications as part of the stack, such as fairness control [6, 8,35, 55, 56,84, issues many more I/O requests than others, the other applications 89,106,118]. Fairness control mechanisms work to equalize are slowed down signicantly. We perform a comprehensive the e ects of interference across applications when multiple analysis of interference in real MQ-SSDs, and nd four major applications concurrently access a shared device. Fairness is interference sources: (1) the intensity of requests sent by each a critical requirement in multiprogrammed computers and application, (2) di erences in request access patterns, (3) the multi-tenant cloud environments, where multiple I/O ows ratio of reads to writes, and (4) garbage collection. (i.e., series of I/O requests) from di erent applications concur- rently access a single, shared SSD [36, 43, 84, 91, 94, 101]. To alleviate unfairness in MQ-SSDs, we propose the Flash- For older host–interface protocols, the OS software stack Level INterference-aware scheduler (FLIN). FLIN is a lightweight provides fairness by limiting the number of requests that each I/O request scheduling mechanism that provides fairness among application can dispatch from the host to the SSD. Since fair- requests from di erent applications. FLIN uses a three-stage ness was handled by the OS software stack, the vast majority scheduling algorithm that protects against all four major sources of state-of-the-art SSD I/O request schedulers did not con- of interference, while respecting the application-level priorities sider request fairness [20,36,37,40,77,90,112]. Surprisingly, assigned by the host. FLIN is implemented fully within the even though new host–interface protocols, such as NVMe, do SSD controller rmware, requiring no new hardware, and has not use the OS software stack, modern MQ-SSDs still do not negligible (<0.06%) storage cost. Compared to a state-of-the-art contain any fairness mechanisms. To demonstrate this, we I/O scheduler, FLIN improves the fairness and performance of a perform experiments using four real state-of-the-art enter- wide range of enterprise and datacenter storage workloads, with prise MQ-SSDs. We make two major ndings. First, when two an average improvement of 70% and 47%, respectively. applications share the same SSD, one of the applications typi- cally slows down (i.e., it takes more time to execute compared 1. Introduction to if it were accessing the SSD alone) signicantly. When we Solid-state drives (SSDs) are widely used as a storage run a representative workload on our four SSDs, we observe medium today due to their high throughput, low response that such slowdowns range from 2x to 106x. Second, with the time, and low power consumption compared to conventional removal of the fairness control mechanisms that were in the hard disk drives (HDDs). As more SSDs are deployed in data- software stack, an application that makes a large number of centers and enterprise platforms, there has been a continued I/O requests to an MQ-SSD can starve requests from other need to improve SSD performance. One area where SSD man- applications, which can hurt overall system performance and ufacturers have innovated on SSD performance is the host– lead to denial-of-service issues [8, 84, 101]. Therefore, we con- interface protocol, which coordinates communication between clude that there is a pressing need to introduce fairness control applications and the SSD. SSDs initially adopted existing host– mechanisms within modern SSD I/O request schedulers. interface protocols (e.g., SATA [88]) that were originally de- In order to understand how to control fairness in a modern signed for lower-performance HDDs. As the performance of SSD, we experimentally analyze the sources of interference the underlying storage technology (e.g., NAND ash memory) among I/O ows from di erent applications in an SSD using used by the SSD increased, these host–interface protocols the detailed MQSim simulator [101]. Our experimental results became a signicant performance bottleneck [114], mainly enable us to identify four major types of interference: because these protocols rely on the OS to manage I/O requests 1. I/O Intensity: The SSD controller breaks down each I/O and data transfers between the host system and the SSD. request into multiple page-size (e.g., 4 kB) transactions. An
1 I/O ow with a greater ow intensity (i.e., the rate at which concurrently-running I/O ows with di erent characteristics. the ow generates transactions) causes a ow with a lower We show that FLIN improves both fairness and performance ow intensity to slow down, by as much as 49x in our across all of our tested MQ-SSD congurations. experiments (Section 3.1). In this paper, we make the following major contributions: 2. Access Pattern: The SSD contains per-chip transaction queues that expose the high internal parallelism that is • We provide an in-depth experimental analysis of unfairness available. Some I/O ows exploit this parallelism by in- in state-of-the-art multi-queue SSDs. We identify four major serting transactions that can be performed concurrently sources of interference that contribute to unfairness. into each per-chip queue. Such ows slow down by as • We propose FLIN, a new I/O request scheduler for mod- much as 2.4x when run together with an I/O ow that does ern SSDs that e ectively mitigates interference among not exploit parallelism. This is because an I/O ow that concurrently-running I/O ows to provide both high fair- does not exploit parallelism inserts transactions into only ness and high performance. a small number of per-chip queues, which prevents the • We comprehensively evaluate FLIN using a wide variety of highly-parallel ow’s transactions from completing at the storage workloads consisting of concurrently-running I/O same time (Section 3.2). ows, and demonstrate that FLIN signicantly improves 3. Read/Write Ratio: Because SSD writes are typically 10–40x fairness and performance over a state-of-the-art I/O request slower than reads [62–64,67,92], a ow with a greater frac- scheduler across a variety of MQ-SSD congurations. tion of write requests unfairly slows down a concurrently- running ow with a smaller fraction of write requests, i.e., 2. Background and Motivation a ow that is more read-intensive (Section 3.3). We rst provide a brief background on multi-queue SSD 4. Garbage Collection: SSDs perform garbage collection (MQ-SSD) device organization and I/O request management (GC) [10,13,16,23], where invalidated pages within the SSD (Section 2.1). We then motivate the need for a fair MQ-SSD are reclaimed (i.e., erased) some time after the invalidation. I/O scheduler (Section 2.2). When an I/O ow requires frequent GC operations, these operations can block the I/O requests of a second ow that 2.1. SSD Organization performs GC only infrequently, slowing down the second Figure 1 shows the internal organization of an MQ-SSD. The ow by as much as 3.5x in our experiments (Section 3.4). components within an SSD are split into two groups: (1) the To address the lack of OS-level fairness control mechanisms, back end, which consists of data storage units; and (2) the front prior work [36] implements a fair I/O request scheduling pol- end, which consists of control and communication units. The icy in the SSD controller that is similar to the software-based front end and back end work together to service the multiple policies previously used in the OS [120]. Unfortunately, this page-size I/O transactions that constitute an I/O request. approach has two shortcomings: (1) it cannot control two Front end Back end types of interference, access pattern interference or GC inter- Host DRAM HIL FTL Channel0 ference; and (2) it signicantly decreases the responsiveness Transaction FCC Chip 0 Chip 1 Request i, Address Scheduling Page 1 Translation Channel1
and overall throughput of an MQ-SSD [89]. Our goal in Unit (TSU) FCC Chip 2 Chip 3
SSD
-
BusBus InterfaceInterface
Microprocessor Bus Interface
Side
MultiplexedMultiplexedMultiplexed
DieDie 0 0
this work is to design an e cient I/O request scheduler for -
InterfaceInterface i Request i, Interface Plane0 Page M Chip 0 Queue WRQ a modern SSD that (1) provides fairness among I/O ows by Host Flash Plane1
MQ RDQ Die 1 Chip 1 Queue Die 1 Management Plane0 mitigating all four types of interference within an SSD, and Device-level Data Chip 2 Queue GC-WRQ I/O Request Queues Request I/O Plane1 (2) maximizes the performance and throughput of the SSD. Request Queues Chip 3 Queue GC-RDQ DRAM To this end, we propose the Flash-Level INterference-aware scheduler (FLIN) for modern SSDs. FLIN considers the prior- Figure 1: Internal organization of an MQ-SSD. ity class of each ow (which is determined by the OS), and The back end contains multiple channels (i.e., request buses), carefully reorders transactions within the SSD controller to where the channels can service I/O transactions in parallel. balance the slowdowns incurred by ows belonging to the Each channel is connected to one or more chips of the underly- same priority class. FLIN uses three stages to schedule re- ing memory (e.g., NAND ash memory). Each chip consists of quests. The rst stage, Intensity- and Parallelism-aware Queue one or more dies, where each die contains one or more planes. Insert, reorders transactions to mitigate the impact of di erent Each plane can service an I/O transaction concurrently with I/O intensities and di erent access patterns. The second stage, the other planes. In all, this provides four levels of parallelism Flow-level Priority Arbitration, prioritizes transactions based for servicing I/O transactions (channel, chip, die, and plane). on the priority class of their corresponding ow. The third stage, Read/Write Wait-Balancing, balances the overall stall The front end manages the back end resources and issues time of read and write requests, and schedules GC activities I/O transactions to the back end channels. There are three to distribute the GC overhead proportionally across di erent major components within the front end [37,61,101]: (1) the ows. FLIN is fully implementable in the rmware of a mod- host–interface logic (HIL), which implements the protocol used ern SSD. As such, it requires no additional hardware, and in to communicate with the host and fetches I/O requests from the worst case consumes less than 0.06% of the DRAM storage the host-side request queues in a round-robin manner [83]; space that is already available within the SSD controller. (2) the ash translation layer (FTL), which manages SSD re- Our evaluations show that FLIN signicantly improves fair- sources and processes I/O requests [10,13] using an embedded ness and performance across a wide variety of workloads and processor and DRAM; and (3) the ash channel controllers MQ-SSD congurations. Compared to a scheduler that uses a (FCCs), which are used to communicate between the other state-of-the-art scheduling policy [37] and fairness controls front end modules and the back end chips. similar to those previously used in the OS [36], FLIN improves There are seven major steps in processing an I/O request. fairness by 70%, and performance by 47%, on average, across (1) The host generates the request and inserts the request into 40 MQ-SSD workloads that contain various combinations of a dedicated in-DRAM I/O queue on the host side, where each
2 queue corresponds to a single I/O ow.1 (2) The HIL in the time of each request is measured from the time the request MQ-SSD fetches the request from the host-side queue and is generated on the host side to the time the host receives a inserts it into a device-level queue within the SSD. (3) The HIL response from the SSD. We dene fairness (F) [22] as the ratio parses the request and breaks it down into multiple transac- of the maximum and minimum slowdowns experienced by tions. (4) The address translation module in the FTL performs any ow in the system: a logical-to-physical address translation for each transaction. F = MIN{Si}/MAX{Si} (2) (5) The transaction scheduling unit (TSU) [101,102] in the FTL i i schedules each transaction for servicing [20,102], and places As dened, 0 < F ≤ 1. A lower F value indicates a greater the transaction into a chip-level software queue. (6) The TSU di erence between the ow that is slowed down the most and selects a transaction from the chip-level queue and dispatches the ow that is slowed down the least, which is more unfair to it to the corresponding FCC. (7) The FCC issues a command the ow that is slowed down the most. In other words, higher to the target back end chip to execute the transaction. values of F are desirable. In addition to processing I/O requests, the SSD controller We conduct a large set of real system experiments to study must perform maintenance operations. One such operation the fairness of scheduling policies implemented in four real is garbage collection (GC) [10,13,16,23]. Due to circuit-level MQ-SSDs [31, 33, 65, 111] (which we call SSD-A through SSD- limitations, many of the underlying memories used in SSDs D), using workloads that capture a wide range of I/O ow be- must perform writes to a logical page out of place (i.e., the data havior (see Section 6). Figure 2 shows the slowdowns and fair- is written to a new physical page). For example, in NAND ash ness for a representative workload where a moderate-intensity memory, before new data can be written to a physical page, read-dominant ow (generated by the tpce application; see the page must rst be erased. The erase operation can only be Table 2) runs concurrently with a high-intensity ow with performed at the granularity of a ash block, which contains an even mix of reads and writes (generated by the tpcc ap- hundreds of pages [10, 13]. Since many other pages in a block plication; see Table 2), as measured over a 60 s period. We may still contain valid data at the time of the update, the make two key observations from the gure. First, the amount block containing the page that is being updated is typically not of interference between the two ows uctuates greatly over erased at that time, and the page is simply marked as invalid. the period, which leads to high variability in the slowdown Periodically, the GC procedure (1) identies candidate blocks of each ow. Across the four SSDs, tpce experiences slow- that are ready to be erased (i.e., blocks where most of the pages downs as high as 2.8x, 69x, 661x, and 11x, respectively, while are marked as invalid), (2) generates read/write transactions its average slowdowns are 2x, 4.4x, 106x, and 5.3x. Second, the to move any remaining valid data in each candidate block to SSDs that we study do not provide any fairness guarantees. another location; and (3) generates an erase transaction for We observe that tpcc does not experience high slowdowns each block. throughout the entire period. We conclude that modern MQ- The TSU is responsible for scheduling both I/O transactions SSDs focus on providing high performance at the expense of and GC transactions. For I/O transactions, it maintains a large amounts of unfairness, because they lack mechanisms read queue (RDQ in Figure 1) and a write queue (WRQ) in to mitigate interference among ows. DRAM for each back end chip. GC transactions are kept in a 5 4 tpce tpcc 60 tpce tpcc separate set of read (GC-RDQ) and write (GC-WRQ) queues 3 45 in DRAM. The TSU typically employs a rst-come rst-serve 2 30
Slowdown 1
Slowdown 15 transaction scheduling policy, and typically prioritizes I/O 1.00 1.00 0.75 0.75 transactions over GC transactions [51, 117]. To better exploit 0.50 0.50 0.25 0.25
the parallelism available in the back end, some state-of-the-art Fairness 0.00 Fairness 0.00 0 10 20 30 40 50 60 0 10 20 30 40 50 60 TSUs partially reorder the transactions within a queue [37,40]. Time (s) Time (s) 2.2. The Need for Fairness Control in an SSD (a) SSD-A (b) SSD-B tpce tpcc tpce tpcc 12 When multiple I/O ows are serviced concurrently, the be- 600 450 9 havior of the requests of one ow can negatively interfere with 300 6 150 3 Slowdown the requests of another ow. In order to understand the impact Slowdown 1.00 1.00 of such inter- ow interference, and how the scheduler a ects 0.75 0.75 0.50 0.50 this type of interference, we would like to quantify how fairly 0.25 0.25 Fairness 0.00 Fairness 0.00 the requests of di erent ows are being treated. We say that 0 10 20 30 40 50 60 0 10 20 30 40 50 60 an I/O scheduler is completely fair if concurrently-running Time (s) Time (s) ows with equal priority experience the same slowdown due (c) SSD-C (d) SSD-D to interference [19,22,70,73,84]. The slowdown Si of ow i Figure 2: Slowdown and fairness of ows in a representative can be calculated as: workload on four real MQ-SSDs. shared alone Si = RTi /RTi (1) 3. An Analysis of Unfairness in MQ-SSDs shared where RTi is the average response time of ow i when it As Section 2.2 shows, real MQ-SSDs do not provide fair- alone ness across multiple queues. This is because state-of-the-art runs concurrently with other ows, and RTi is the same ow’s average response time when it runs alone. The response transaction scheduling units (TSUs) [20, 37, 40, 77, 90, 112] are designed to maximize the overall I/O request throughput of 1In this work, we use the term ow to indicate a series of I/O requests the entire SSD, and thus they do not take into account the that (1) are generated by a single user process; or (2) originate from the throughput of each ow. Our goal in this work is to design a same host-side I/O queue (e.g., for systems running virtual machines, where high-performance TSU for MQ-SSDs that delivers high SSD fairness should be enforced at the VM level [36], and where each VM can be throughput while providing fairness for each ow. assigned to a dedicated I/O queue). To simplify explanations without loss of generality, we associate each ow with a dedicated host-side I/O queue in In order to meet this goal, we must (1) identify the runtime this work. characteristics of each I/O ow, and (2) understand how these
3 characteristics lead to unfairness. An I/O ow is made up of a compared to when the base ow runs alone. We conclude that stream of I/O requests, where each request is dened by its the average response time of a low-intensity ow substantially arrival time, starting address, size of the data that it requires, increases due to interference from a high-intensity ow. and access type (i.e., read or write) [42, 79, 119]. Based on the requests that make up a particular ow, we dene four key 3.2. Flows with Di erent Access Patterns characteristics of the ow: (1) I/O intensity, (2) access pattern, The access pattern of a ow determines how its transactions (3) read/write ratio of the requests in the ow, and (4) garbage are distributed across the chip-level queues. Some ows can collection (GC) demands of the ow. In this section, we study take advantage of the parallelism available in the back end how di erences in each of these characteristics can lead to of the SSD, by distributing their transactions across all of unfairness, using the methodology described in Section 6. the chip-level queues equally. Other ows do not benet as much from the back-end parallelism, as their transactions are 3.1. Flows with Di erent I/O Intensities distributed unevenly across the chip-level queues. The access The I/O intensity of a ow refers to the rate at which the pattern depends on the starting addresses and sizes of each ow generates transactions. This is dependent on the arrival request in the ow. time and the size of the I/O requests (as larger I/O requests To analyze the interference between concurrent ows with generate a greater number of page-sized transactions). There di erent access patterns, we run a base ow with a stream- are three factors that contribute to the amount of time required ing access pattern [50, 70] together with an interfering ow to service each transaction, which is known as the transaction whose access pattern changes periodically between streaming turnaround time: (1) the wait time at the chip-level queue, and random (i.e., we can control the fraction of time the ow (2) the transfer time between the front end and the back end, performs random accesses). Figure 5 shows the slowdown of and (3) the time required to execute the transaction in the the two ows and the fairness of the SSD. The x-axis shows memory chip. As the intensity of a ow increases, the transfer the randomness of the interfering ow. We observe that as time and execution time do not change, but the wait time at the randomness of the interfering ow increases, the MQ-SSD the chip-level queue grows. becomes more unfair to the base ow. Both the probability Figure 3 shows how ow intensity a ects chip-level queue of resource contention and the transaction turnaround time length and average transaction turnaround time. We run a increase when the two ows are executed together. However, 100%-read synthetic ow where we vary the I/O intensity from the base ow, which highly exploits and benets from chip- 16 MB/s to 1024 MB/s. Figures 3a and 3b show, respectively, level parallelism, is more susceptible to interference from the the average length of the chip-level queues and the breakdown interfering ow. The uneven distribution of transactions from of the transaction turnaround time with respect to the I/O the interfering ow causes some, but not all, of the base ow’s intensity. We make three conclusions from the gures. First, transactions to stall. As a result, the base ow’s parallelism the lengths of the chip-level queues increase proportionately gets destroyed, and the ow slows down until its last trans- with the I/O intensity of the running ow. Second, a longer action is serviced. We conclude that ows with parallelism- queue leads to a longer wait time for transactions. Third, as friendlyBase access flow patterns are susceptible to interference from ows Interferingwith access flow patterns that do not exploit parallelism. the ow intensity increases, the contribution of the queue wait Fairness Base flow Interfering flow Fairness time to the transaction turnaround time increases, dominating Base flow Interfering flow Fairness the other two factors (shown in the gure). 3.0 1.0 3.0 1.0
6 3 49 1 10 106 240 240 2.4 0.8 Chip Queue Waiting 2.4 0.8
s) Chip Queue Waiting 5 5 s) 2.4 0.8 3678 μ 3678 10 10 200μ 200 BusBus Transfer Transfer 1.8 0.6 Flash Operation 1.8 0.6 4 Flash Operation 10 104 160 160 1.8 0.6 1.2 0.4 3 3 1.2 0.4 10 10 120 120 1.2 0.4 Fairness Slowdown Fairness Slowdown Fairness
2 Slowdown 0.6 0.2 10 102 80 80 0.6 0.6 0.2 0.2 10 40 0.0 0.0
Queue Length 10 40 0% 25% 50% 75% 100% Queue Length 0.0 0.0 16 320 0%64 12825%25650%512 75%1024 100% 0 1 1 0 0 Turnaround Time ( Turnaround Time ( Normalized Chip-level Normalized Chip-level 16 1632 3264 6412812825625651251210241024 16 1632 3264 6412812825625651251210241024 Randomness of Avg. Flash Transaction Avg. Flash Transaction Intensity of Randomness of the Interfering Flow FlowFlow I/O I/O Intensity Intensity (MB/s) (MB/s) FlowFlow I/O I/O Intensity Intensity (MB/s) (MB/s) the Interferingthe Interfering Flow (MB/s) Flow (a) Chip-level queue length (b) Transaction turnaround time Figure 4: E ects of concur- Figure 5: E ects of con- rently executing two ows currently executing two Figure 3: E ect of ow I/O intensity. with di erent I/O intensities ows with di erent access When two ows with di erent I/O intensities execute con- on fairness. patterns on fairness. currently, each ow experiences an increase in the average length of the chip-level queues, which can more adversely 3.3. Flows with Di erent Read/Write Ratios a ect the ow with a low I/O intensity than the ow with a As reads are 10-40x faster than writes, and are more likely high I/O intensity. To characterize the e ect, we execute a to fall on the critical path of program execution, state-of-the- low-intensity 16 MB/s synthetic ow, referred to as the base art MQ-SSD I/O schedulers tend to prioritize reads over writes ow, together with a synthetic ow of varying I/O intensities, to improve overall performance [20, 112]. To implement read referred to as the interfering ow. The I/O requests of the two prioritization, TSUs mainly adopt two approaches: (1) trans- ows are purely reads. Figure 4 depicts the slowdown and fair- actions in the read queue are always prioritized over those in ness results. The x-axis shows the intensity of the interfering the write queue, and (2) a read transaction can preempt an ow. We observe that even when the intensity of the inter- ongoing write transaction. fering ow exceeds 256 MB/s, the slowdown of the interfering Unfortunately, read prioritization increases the wait time ow is very small, while the base ow is slowed down by as of write transactions, and can potentially slow down write- much as 49x. The unfair slowdown of the base ow is due to a dominant ows in an MQ-SSD [84]. Figure 6 compares the drastic increase in the average length of the chip-level queues response time and fairness of an SSD using a read-prioritized
4 scheduler (RP) with that of an SSD with a rst-come rst- the fraction of time spent on I/O requests and on GC when serve scheduler (FCFS), which processes all transactions in the base ow (the leftmost column) and the interfering ow the order they are received. In this experiment, we use two (all other columns) are executed alone. Figures 7b, 7c, and concurrently-running synthetic ows, where we control the 7d show the slowdown of the base ow, the slowdown of the rate at which the ows can issue new requests to the SSD interfering ow, and the fairness of the MQ-SSD, respectively. by limiting the size of the host-side queue. We sweep the The x-axis in these gures represents the write intensity of write rate of the base ow (0%, 30%, 70%, and 100%), shown the interfering ow. as di erent curves, and set the host-side queue length to one 100% request. We set the host-side queue length for the interfering 80% User I/O ow GC Activities to six requests (i.e., it has 6x the request rate of the 60% 4 4 Idle No GC No GC base ow) to increase the probability of interference, and 3 4 3 4 40% Default GC Default GC we sweep the interfering ow’s read ratio from 0% to 100%. SPGCNo GC SPGCNo GC Flash Chip 2 3 Default GC 2 3 Default GC In Figures 6a and 6b, the x-axis marks the read ratio of the 20% SPGC SPGC Status Breakdown 2 2 Slowdown 1 Slowdown 1 interfering ow. We make three observations from the gures. 0% 2 4 8 16 32 64 0 Slowdown 1 0 Slowdown 1 First, RP maintains or improves the average response time for Intensity of 2 4 8 16 32 64 2 4 8 16 32 64 base flow Interfering Flow (MB/s) Intensity 0 of the Interfering Flow (MB/s) Intensity 0 of the Interfering Flow (MB/s) all ow congurations over FCFS. Second, RP leads to lower 2 4 8 16 32 64 2 4 8 16 32 64 (a) Breakdown of chip status (b)Intensity Slowdown of the ofInterferingbase ow Flow (MB/s) Intensity of the Interfering Flow (MB/s) fairness than FCFS when the interfering ow is heavily read- over the entire simulation time dominant (i.e., its read ratio > 80%). Third, fairness in an SSD with RP increases when the read ratio4 of the interfering ow 4 1.0 No GC No GC 0.8 1.0 decreases. We conclude that when 3 concurrently-running ows 3 Default GC Default GC 0.6 0.8 have di erent read/write ratios, existing schedulingSPGC policies are SPGC 0.6 2 2 0.4 No GC not e ective at providing fairness. Fairness 0.4 DefaultNo GCGC
0.2 Fairness Slowdown 1 Slowdown 1 Default GC 0.2 SPGC 100% WR in base flow 30% WR in base flow 0.0 SPGC 70% WR in base flow 00% WR in base flow 0 0.02 4 8 16 32 64 2 4 8 16 32 64 2 4 8 16 32 64 Intensity of2 the Interfering4 8 Flow16 (MB/s)32 64 60% 100%Intensity of the Interfering Flow (MB/s) Intensity of the Interfering Flow (MB/s) Intensity of the Interfering Flow (MB/s) (c) Slowdown of interfering ow (d) Fairness 50% 75% 50% Figure 7: E ects of GC demands on fairness. 40% 1.025% 30% We make four key observations. First, the GC activities 0.80% increase proportionately with write intensity. Second, as the 20% 0.6 -25% write intensity of the interfering ow increases, which triggers 0.4 No GC 10% Fairness -50% Default GC more GC operations, fairness decreases. Third, when the 0.2 SPGC 0% 0% 20% 40% 60% 80% 100% 0.0-75% 0% 20% 40% 60% 80% 100% di erence between the GC demands of the base ow and the Fairness Improvement 2 4 8 16 32 64 Intensity of the Interfering Flow (MB/s) interfering ow increases, the default GC mechanism leads to a Read Ratio of Read Ratio of Response Time Improvement the Interfering Flow the Interfering Flow higher slowdown of the base ow, degrading fairness. Fourth, SPGC improves fairness over the default GC mechanism, but (a) Response time improvement of (b) Fairness improvement of RP RP compared to FCFS compared to FCFS only by a small amount, as SPGC disables preemption when there are a large number of pending writes. We conclude that Figure 6: E ects of read prioritization. the GC activities of a high-GC ow can unfairly block ash 3.4. Flows with Di erent GC Demands transactions of a low-GC ow. GC is one of the major sources of performance uctuation in 4. FLIN: Fair, High-Performance Scheduling SSDs [26,27,38,41,51,54,108]. The rate at which GC is invoked As Section 3 shows, the four sources of interference that depends primarily on the write intensity and access pattern of slow down ows all result from interference among the trans- the ow [107]. A ow that causes GC to be invoked frequently actions that make up each ow. Based on this insight, we (high-GC) can disproportionately slow down a concurrently- design a new MQ-SSD transaction scheduling unit (TSU) running ow that does not invoke GC often (low-GC). Prior called the Flash-Level INterference-aware scheduler (FLIN). works that optimize GC performance [26,27,38,41,51,54,108] FLIN schedules and reorders transactions in a way that sig- can reduce interference between GC operations and trans- nicantly reduces each of the four sources of interference, in actions from I/O requests, and thus can improve fairness. order to provide fairness across concurrently-running ows. However, these works cannot completely eliminate interfer- FLIN consists of three stages, as shown in Figure 8. The rst ence between a high-GC ow and a low-GC ow that occurs stage, fairness-aware queue insertion, inserts each new trans- when a high-GC ow results in a write cli (i.e., when the action from an I/O request into the appropriate read/write high intensity of the write requests causes GC to be invoked queue (Section 4.1). The position at which the transaction is frequently, resulting in a large drop in performance [39]). inserted into the queue depends on the current intensity and Figure 7 shows the e ect of GC execution on MQ-SSD access pattern of the ow that corresponds to the transaction, fairness. We run a base ow with moderate GC demands (i.e., such that (1) a low-priority ow is not unfairly slowed down, 8 MB/s write rate) and an interfering ow where we change the and (2) a ow that can take advantage of parallelism in the GC demands by varying the write rate (2 MB/s to 64 MB/s). We back end does not experience interference that undermines evaluate three di erent scenarios for GC: (1) no GC execution, this parallelism. The second stage, priority-aware queue arbi- assuming the MQ-SSD capacity is innite; (2) default (i.e., tration, uses priority levels that are assigned to each ow by non-preemptive) GC execution; and (3) semi-preemptive GC the host to determine the next read request and the next write (SPGC) [51,52] execution. SPGC tries to mitigate the negative request for each back end memory chip (Section 4.2). The ar- e ect of GC on user I/O requests by allowing user transactions biter aims to ensure that all ows assigned to the same priority to preempt GC activities when the number of free pages in the level are slowed down equally. The third stage, wait-balancing SSD is above a specic threshold (GChard). Figure 7a shows transaction selection, selects the transaction to dispatch to the
5 ash chip controllers (FCCs, which issue commands to the Algorithm 1 Fairness-aware queue insertion in FLIN. memory chips in the back end; see Section 2.1) in a way that 1: Inputs: alleviates the interference introduced by read/write ratio dis- 2: Q: the chip-level queue to insert the new transaction parities among ows and by garbage collection (Section 4.3). 3: TRnew: the newly arriving ash transaction To achieve this, the scheduler chooses to dispatch a read trans- 4: Source: source ow of the new transaction 5: Fthr : fairness threshold for high-intensity ows action, a write transaction, or a garbage collection transaction, 6: in a way that (1) balances the slowdown for read transactions 7: Lastlow ← farthest transaction from head ofQ belonging to a low-intensity ow with the slowdown of write transactions, and (2) distributes 8: // Step 1: Prioritize Low-Intensity Flows the overhead of garbage collection transactions fairly across 9: if Source is a low-intensity ow then // more detail in Section 5.1 10: Insert(TRnew after Lastlow ) all ows. FLIN is designed to be easy to implement, with low 11: // Step 2a: Maximize Fairness Among Low-Intensity Flows. overhead, as we describe in Section 5. 12: Estimate waiting time ofTR new as if Source were running alone // Section 5.2 13: Reorder transactions of low-intensity ows inQ for fairness // Section 5.3 Chip 0 Chip 1 Chip 0 Chip 1 14: else Queue Queue Queue Queue 15: Insert(TRnew at the tail ofQ) Chip 2 Chip 3 Chip 2 Chip 3 Queue Queue Queue Queue 16: // Step 2b: Maximize Fairness Among High-Intensity Flows. 17: Estimate waiting time ofTR as if Source were running alone
aware new
aware - RDQ WRQ - Read Slot 18: F ← Estimate fairness from average slowdown of departed transactions (Q)
Q1 Q1
FCCFCC Write Slot balancing 19: if F < F and Source has experienced the maximum slowdown then
- thr
Q2 Q2 Stage2 Stage3 Stage1 GC-RDQ 20: MoveTR new to Lastlow + 1 GC-WRQ QP QP 21: else Reorder transactions from the tail ofQ to Lastlow + 1 for fairness
Priority If Source of TRnew TR If Source of TRnew TR Wait
Fairness new new QueueInsertion
QueueArbitration Ifis Sourcehigh-intensity of TRnew TR Ifis Sourcelow-intensity of TRnew TR Flash Transactions Flash new Tail newLast Head
Transaction Selection Transaction low DRAM DRAM Ifis Sourcehigh-intensity of TRnew isIf Sourcelow-intensity of TRnew TRnew Tail TRnewLastlow Head Chipis high-level-intensity Queue (Q) TR11 TR10 TR9 TR8is lowTR6-intensityTR5 TR4 TR3 TR2 TR1 Tail Lastlow Head Figure 8: High-level overview of FLIN. Chip-level Queue (Q) TR11 TR10 TR9 TR8 TR6 TR5 TR4 TR3 TR2 TR1 TR TR TransactionsTR TR fromTR TR TR Transactions from Chip-level Queue (Q) 11 10 highTransactions-9intensity8 fromflows6 5 4 lowTransactionsTR-3intensityTR2 TRflowsfrom1 4.1. Fairness-Aware Queue Insertion highTransactions-intensity fromflows lowTransactions-intensity flowsfrom high-intensity flows low-intensity flows We design fairness-aware queue insertion, the rst stage of (a) Step 1 of Algorithm 1 Reorder transactions to FLIN, to relieve interference that occurs due to the intensity Reorderimprove transactions fairness to and access pattern of concurrently-running ows. Within the Chip-level Queue (Q) TR11 TR10 TR9 TR8 TR6 TR5 TRReorder4 TRimprovenew transactionsTR3 fairnessTR2 TR to1 TR11 TR10 TR9 TR8 TR6 TR5 TR4 TRimproveTR fairnessTR TR rst stage, FLIN maintains separate per-chip read and write Chip-level Queue (Q) new 3 2 1 Chip-level Queue (Q) TR11 (b)TR10 StepTR 2a9 ofTR8 AlgorithmTR6 TR5 1 TR4 TRnew TR3 TR2 TR1 queues (RDQ and WRQ in Figure 8, respectively) for each Reorder transactions to improve fairness Reorder transactions to improve fairness ow priority level, since priorities are not processed until the Chip-level Queue (Q) TRnew TR11 TR10 TR9 TR8 TR6 TR5 TR4 TR3 TR2 TR1 Chip-level Queue (Q) ReorderTRnew TR11 transactionsTR10 TR9 to TRimprove8 TR6 fairnessTR5 TR4 TR3 TR2 TR1 second stage of FLIN. Fairness-aware queue insertion uses a Chip-level Queue (Q) TRnew TR11 TR10 TR9 TR8 TR6 TR5 TR4 TR3 TR2 TR1 heuristic O(t) algorithm to determine where each transaction (c) Step 2b of Algorithm 1 is inserted within the queue that corresponds to the access Figure 9: Fairness-aware queue insertion in FLIN. type (i.e., read or write) of the transaction and the priority TRnew belongs to a high-intensity ow, the heuristic inserts level of the ow that generated the transaction. Finding the the transaction at the tail of the queue (line 15), and advances optimal ordering of transactions in each queue is a variant to Step 2b (line 17). of the multidimensional assignment problem [85], which is NP-hard [82]. FLIN uses a heuristic that approximates the Step 2a: Maximize Fairness Among Low-Intensity optimal ordering, in order to bound the time required for the Flows. The heuristic begins this step by estimating how long rst stage and to minimize the fraction of the transaction TRnew would have to wait to be serviced if its source ow turnaround time that is spent on scheduling. were running alone (line 12). This estimate is also used to Algorithm 1 shows the heuristic used to insert a new trans- determine a transaction’s slowdown and is kept alongside the action (TRnew) into its corresponding chip-level queue (Q). transaction in the queue. The heuristic then reorders transac- The heuristic is based on three key insights: (1) the perfor- tions of low-intensity ows in the queue for fairness (line 13). mance of a low-intensity ow is very sensitive to the increased As Figure 9b shows, this involves moving TRnew from the tail queue length when a high-intensity ow inserts many trans- of the low-intensity- ow transactions closer to the head, in actions (Section 3.1); (2) some ows can change between being order to maximize the ratio of the minimum slowdown to high-intensity and low-intensity, based on the current phase maximum slowdown for all pending transactions from low- of the application [80]; and (3) some, but not all, of the transac- priority ows. This reordering step prioritizes a transaction tions from a ow that take advantage of back-end parallelism (i.e., moves it closer to the queue head) that has a greater experience high queue wait times when a ow with poor slowdown, which indicates that its source ow either (1) has a parallelism inserts requests into some, but not all, of the chip- lower intensity, or (2) better utilizes the parallelism available level queues (Section 3.2). The algorithm consists of two steps, in the MQ-SSD back end. We describe the details of the alone which we describe in detail. wait time estimation (line 12) and the reordering step (line 13) in Sections 5.2 and 5.3, respectively. Step 1: Prioritize Low-Intensity Flows. Transactions from a low-intensity ow are always prioritized over transac- Step 2b: Maximize Fairness Among High-Intensity tions from a high-intensity ow. Figure 9a shows an example Flows. Figure 9c shows a general overview of Step 2b. The snapshot of a chip-level queue when FLIN inserts a newly- heuristic begins this step by estimating the alone wait time of arrived transaction (TRnew) into the queue. The heuristic TRnew (line 17), using the same procedure in Step 2a. However, checks to see whether TRnew belongs to a low-intensity ow unlike the low-intensity- ow transactions, this step performs (line 9 of Algorithm 1; see Section 5.1 for details). If TRnew is slowdown-aware queue shu ing in order to prevent any one from a low-intensity ow, the heuristic inserts TRnew into the high-intensity ow from being treated much more unfairly queue immediately after Lastlow, the location of the farthest than the other high-intensity ows. After a new high-intensity transaction from the head of the queue that belongs to a low- transaction is inserted into the queue, the heuristic checks the intensity ow (line 10), and advances to Step 2a (line 12). If current fairness (line 18). If the fairness is lower than a prede-
6 termined threshold (Fthr ), and the new transaction belongs to Algorithm 2 Wait-balancing transaction selection in FLIN. the ow with the maximum slowdown (line 19), the heuristic 1: Inputs: moves the transaction ahead of all other high-intensity- ow 2: ReadSlot: the read transaction waiting to be issued to the SSD back end transactions (line 20). This approach can often help ows that 3: WriteSlot: the write transaction waiting to be issued to the SSD back end 4: GC-RDQ, GC-WRQ: the garbage collection read and write queues are transitioning from low-intensity to high-intensity, as such 5: ows are particularly susceptible to interference from other 6: PW read ← EstimateProportionalWait (ReadSlot) // Section 5.4 high-intensity ows. If fairness is above the threshold, which 7: PW write ← EstimateProportionalWait (WriteSlot) // Section 5.4 means that all high-intensity ows are being treated some- 8: if PW read > PW write then what fairly, the heuristic only reorders the high-intensity- ow 9: Dispatch ReadSlot to FCC ( ash chip controller) 10: else transactions for fairness (line 21). 11: if number of free pages < GCFLIN then 12: GCMigrationCount ← AssignGCMigrations(WriteSlot) // Section 5.4 4.2. Priority-Aware Queue Arbitration 13: while GCMigrationCount > 0 do Many host–interface protocols, such as NVMe [83], allow 14: Dispatch the transaction at the head of GC-RDQ to FCC the host to assign di erent priority levels to each ow. The 15: Dispatch the transaction at the head of GC-WRQ to FCC 16: GCMigrationCount ← GCMigrationCount - 1 second stage of FLIN enforces these priorities, and ensures 17: Dispatch WriteSlot to FCC that ows at the same priority level are slowed down by an equal amount. As we discuss in Section 4.1, FLIN maintains a We dene the proportional wait time (PW ) of a transaction read and write queue for each priority level (i.e., if there are P as the ratio of its wait time in the scheduler (Twait), from the priority levels as dened by the protocol, FLIN includes P read time that the transaction is received by the scheduler until queues and P write queues in the DRAM of the SSD, for each the time the transaction is dispatched to the FCC, over the ash chip). Priority-aware queue arbitration selects one ready sum of the time required to perform the operation in the read transaction from the transactions at the head of the P memory (Tmemory) and transfer data back to the front end read queues, and one ready write transaction from the trans- (Ttransfer ). The transaction in the read slot is prioritized over actions at the head of the P write queues. The two selected the transaction in the write slot when the read transaction’s transactions then move to the last stage of the scheduler. proportional wait time is greater than the write transaction’s Whenever more than one queue has a transaction at the proportional wait time (lines 6–8 in Algorithm 2). queue head that is ready to execute (i.e., the back end re- sources it requires are not being used by another transaction), Step 2: Dispatch Transactions. If the read transaction is the queue arbiter uses a weighted round-robin policy [76] to prioritized, it is dispatched to the ash channel controller (see select the read and write transactions that move to the next Section 2.1) right away (line 9). If the write transaction is pri- scheduling stage. If the protocol denes P priority levels, oritized, the scheduler then considers also dispatching garbage where level 0 is the lowest priority and level P – 1 is the collection operations, since a write transaction takes signi- highest priority, FLIN assigns the weight 2i to priority level cantly longer to complete than a read transaction and the cost i. Under weighted round-robin, this means that out of every of garbage collection operations can be amortized by serv- P i ing them together with the selected write transaction. If the i 2 scheduling decisions in the second stage, transactions i number of free pages available in the memory is lower than a in the priority level i queue receive 2 of these slots. pre-determined threshold (GCFLIN ), and there are garbage col- lection transactions waiting in the GC-RDQ or GC-WRQ, the 4.3. Wait-Balancing Transaction Selection scheduler (1) determines how many of these transactions to We design wait-balancing transaction selection, the last perform, which is represented by GCMigrationCount (line 12), stage of FLIN, to minimize interference that occurs due to the and (2) issues GCMigrationCount transactions (lines 13–16). read/write ratios (Section 3.3) and garbage collection demands Since garbage collection transactions read a valid page and of concurrently-running ows (Section 3.4). A transaction write the page somewhere else in the memory, FLIN always ex- stalls if the back-end resources that it needs are being used by ecutes a pair of read and write transactions. Once the garbage another transaction (which can be a transaction from another collection transactions are done, the scheduler dispatches the ow, or a garbage collection transaction). Wait-balancing write transaction (line 17). transaction selection attempts to distribute this stall time evenly across all read and write transactions. Optimizations. FLIN employs two optimizations beyond Wait-balancing transaction selection chooses one of the fol- the basic read-write wait-balancing algorithm. First, FLIN lowing transactions to dispatch to the FCC: (1) the ready read supports write transaction suspension [112] whenever the pro- transaction determined by priority-aware queue arbitration portional wait time (PW) of the transaction in the read slot is (Section 4.2), which is stored in a read slot in DRAM; (2) the larger than the PW of the currently-executing write operation. ready write transaction determined by priority-aware queue Second, when the read and write slots are empty, FLIN dis- arbitration, which is stored in a write slot in DRAM; (3) the patches a pair of garbage collection read/write transactions. transaction at the head of the garbage collection read queue If a transaction arrives at the read slot when the garbage col- (GC-RDQ); and (4) the transaction at the head of the garbage lection transactions are being executed, FLIN preempts the collection write queue (GC-WRQ). Algorithm 2 depicts the garbage collection write and executes the transaction at the selection heuristic, which consists of two steps. read slot, to avoid unnecessarily stalling the incoming read. Step 1: Estimate the Proportional Wait. FLIN uses a 5. Implementation of FLIN novel approach to determine when to prioritize reads over To realize the high-level behavior of FLIN (Section 4), we writes, which we call proportional waiting. Previous schedul- need to implement a number of functions, particularly for ing techniques [20, 112] always prioritize reads over writes, Stages 1 and 3 of the scheduler. We discuss each function in which as we show in Section 3.3 leads to unfairness. Propor- detail in this section, and then discuss the overhead required tional waiting avoids this unfairness. to implement FLIN in an MQ-SSD.
7 TR TR 5.1. Classifying Flow I/O Intensity To estimate Tshared, FLIN estimates Twait_shared based on The rst stage of FLIN dynamically categorizes ows as the current position of the transaction in the queue. For a either low-intensity or high-intensity, with separate classi- transaction sitting at the p-th position in the queue (where cations for the ow’s read behavior and its write behavior. the request at the head of the queue is at position 1), FLIN FLIN performs this classication over xed-length intervals. uses the following equation: During each interval, FLIN records the number of read and TR write transactions enqueued by each ow. At the end of the Twait_shared =Tnow – Tenqueued + Tchip_busy interval, FLIN uses the transaction counts to calculate the read p–1 (5) and write arrival rates of each ow. If the read arrival rate of X i i read + Tmemory + Ttransfer a ow is lower than a predetermined threshold αC , FLIN classies the ow as low-intensity for reads; otherwise, FLIN i=1 classies the ow as high-intensity for reads. FLIN performs Tnow – Tenqueued gives the time that has elapsed since the the same classication for write intensity using the write ar- transaction was enqueued (i.e., how long the transaction has rival rate of the ow, and a predetermined threshold αwrite. C already waited for). To compute T TR , FLIN adds the We empirically set the epoch length to 10 ms, which we nd wait_shared is short enough to detect changes in ow intensity, but long time elapsed so far to the amount of time left to service the enough to capture di erences in intensity among the ows. currently-executing transaction (Tchip_busy), and the time re- quired to execute and transfer data for all transactions ahead 5.2. Estimating Alone Wait Time and Slowdown in the queue that have not yet been issued to the back end. The rst stage of FLIN estimates the alone wait time of last each transaction when the transaction is enqueued. After- While FLIN calculates Talone only once (when a transaction TR TR wards, whenever the transactions are reordered for fairness is rst enqueued), it calculates Talone and Tshared every time in the rst stage (see Section 4.1), FLIN uses the alone wait the slowdown of an individual transaction needs to be used time estimate to predict how much each individual trans- during the rst stage. action will be slowed down due to ow-level interference. The slowdown of an individual transaction, STR, is calcu- 5.3. Reordering Transactions for Fairness TR TR TR TR When the rst stage of FLIN reorders transactions within a lated as S = Tshared/Talone, where Tshared is the transaction turnaround time when the ow corresponding to the trans- region of a transaction queue for fairness (in Steps 2a and 2b in Section 4.1), it scans through all transactions in the region action runs concurrently with other ows, and T TR is the alone to nd a position for the newly-arrived transaction (TRnew) transaction turnaround time when the ow runs alone. For that maximizes the ratio of minimum to maximum slowdown both the shared and alone turnaround times, FLIN estimates of the queued transactions. This requires FLIN to perform T TR using the following equation: two passes over the transactions within the queue region. T TR T TR T TR T TR During the rst pass, FLIN estimates the slowdown of each = memory + transfer + wait (3) transaction in the region (see Section 5.2). Then, it calculates T TR and T TR are the times required to execute a com- fairness across all enqueued requests by using Equation 2 over memory transfer the per-transaction slowdowns. mand in the memory chip and transfer the data between the During the second pass, FLIN starts with the last transaction back end and the front end, respectively. Both values are con- in the queue region (i.e., the one closest to the queue tail), and stants that are determined by the speed of the components TR uses the slowdown to determine the insertion position for within the MQ-SSD. Twait is the amount of time that the trans- the newly-arrived transaction that is expected to increase action must wait in the read or write queue before it is issued fairness by the largest amount. To do this, at every position p, to the back end. TR TR FLIN estimates the change in fairness by recalculating the To estimate Talone, FLIN must estimate Twait_alone, which is slowdown only for (1) the newly-arrived transaction, if it the alone wait time. If the ow were running alone, the only were to be inserted into position p; and (2) the transaction other requests that the transaction would wait on are other currently at position p, if it were moved to position p –1. FLIN requests in the same queue from the same ow. As a result, keeps track of the position that leads to the highest fairness FLIN uses the last request inserted into the same queue by as it checks all of the positions in the queue region. After the the same ow to determine how long the current transaction second pass is complete, FLIN moves TRnew to the position would have to wait if the other ows were not running. This that would provide the highest fairness. can be determined by calculating how much longer the last With this two-pass approach, if TRnew is from a ow that request needs to complete: has only a few transactions in the queue (e.g., the ow is TR last last low-intensity), the reordering moves the transaction closer Twait_alone = Tenqueue + Talone – Tnow (4) to the head of the queue. Hence, FLIN signicantly reduces Essentially, we take the timestamp at which the last re- the slowdown of the transaction, making its slowdown value last more balanced with respect to transactions from other ows. quest was enqueued (Tenqueue), add its estimated transaction last 5.4. Estimating Proportional Wait Time turnaround time (Talone) to determine the timestamp at which the request is expected to nish, and subtract the current The third stage of FLIN chooses whether to dispatch the transaction in the read slot or the transaction in the write slot timestamp (Tnow) to determine the remaining time needed to complete the request. If the queue contains no other transac- to the ash chip controller (FCC), by selecting the transaction tions from the same ow when the transaction arrives, then that has the higher proportional wait time. In Section 4.3, we the transaction would not have to wait if the ow was run dene the proportional wait time (PW ) as: TR alone, so FLIN sets Twait_alone to 0. PW = Twait ÷ (Tmemory + Ttransfer ) (6)
8 Tmemory and Ttransfer are the execution and data transfer time, FLIN approximates by using the fraction of all currently-valid respectively, of a read or write transaction that is issued to pages in the SSD that were written by ow f ), since each valid page in the selected block requires a GC operation: the back end of the SSD. Twait represents how long the total wait time of the transaction would be, starting from the time NumWritesf Validf GCM = × × length (10) the host made the I/O request that generated the transaction, P NumWrites P Valid GC–RDQ if FLIN chooses to issue the transaction in the other slot rst. i i i i In other words, if FLIN is evaluating PW for the transaction where NumWritesf is the number of write operations per- in the read slot, it calculates how long that transaction would formed by ow f since the last garbage collection operation, have to wait if the transaction in the write slot were performed Validf is the number of valid pages in the SSD for ow f rst. We can calculate Twait as: (which is determined using the history of valid pages that is Twait = Tnow – Tenqueued + Tother_slot + TGC (7) maintained by the FTL page management mechanism [10,13]), and length is the number of queued garbage collec- T – T gives the amount of time that the transac- GC–RDQ now enqueued tion read transactions (which represents the total number of tion has already waited for, Tother_slot is the time required to pending garbage collection migrations). perform the operation waiting in the other slot rst, and TGC While FLIN can use ne-grained data separation techniques is the amount of time required to perform pending garbage proposed for multi-stream SSDs [41] for more accurate GC collection transactions. Recall from Algorithm 2 (Section 4.3) cost estimation, we nd that our estimates from Equation 10 that if the transaction in the read slot is selected, no garbage are accurate enough for our purposes. collection transactions are performed, so TGC = 0. TGC can be non-zero if the transaction in the write slot is selected. 5.5. Implementation Overhead and Costs To determine Tother_slot, we add the time needed to exe- FLIN can be implemented in the rmware of a modern cute the transaction and transfer data together with the time SSD, and does not require specialized hardware. The re- needed for any garbage collection transactions that would be quirements of FLIN are very modest compared to the pro- performed along with the transaction based on Algorithm 2: cessing power and DRAM capacity of a modern SSD con- not T T T T troller [10–12,14,33,37,57–61,104]. FLIN does interfere other_slot = memory + transfer + GC (8) with any SSD management tasks [10], such as the GC candi- In Equations 7 and 8, TGC = 0 if either (1) the transaction date block selection policy [23,53,107], the address translation being considered is a read, or (2) there are no pending garbage scheme, or other FTL maintenance and reliability enhance- collection operations. Otherwise, TGC is estimated as: ment tasks, and hence it can be adopted independently of SSD GCM management tasks. X read write erase TGC = (2Ttransfer + Tmemory + Tmemory) + Tmemory (9) DRAM Overhead. FLIN requires only a small amount of i=1 space within the in-SSD DRAM to store four pieces of informa- In this equation, GCM stands for GCMigrationCount, which tion that are not kept by existing MQ-SSD schedulers: (1) the read and write arrival rates of each ow in the last interval is the number of page migrations that should be performed × ows × chips along with the transaction in the write slot to fairly distribute (Section 5.1), which requires a total of 2 # # 32- the interference due to garbage collection (Section 3.4). Re- bit words; (2) the alone wait time of each transaction waiting in the chip-level queues (Section 5.2), which requires a total call from Section 2.1 that for many SSDs, such as those that max transactions use NAND ash memory, garbage collection occurs at the of _ 32-bit words; (3) the average slowdown granularity of a block. A block that is selected for garbage of all high-intensity ows in each queue (Section 4.1, Step 3), which requires 2 × # ows × #chips 32-bit words; (4) the GC collection may still contain a small number of valid pages, i which must be moved to a new block before the selected block cost estimation data (i.e., Countwrite and Validi; Section 5.4), can be erased. For each valid page, this requires the SSD to which requires 2 × # ows 32 bit variables. For an SSD that (1) execute a read command on the memory chip containing has 64 ash chips and 2 GB of DRAM [61] (resulting in a very read conservative maximum of 262,144 8 kB transactions that the the old physical page (which takes Tmemory time), (2) two data DRAM can hold), and that supports a maximum of 128 con- transfers between the back end and the front end for GC read current ows [30,32,33,65,66,87,103,104,110,111], the DRAM and write operations (2Ttransfer ), and (3) execute a write com- overhead of FLIN is 1.13 MB, which is less than 0.06% of the mand on the memory chip containing the new physical page total DRAM capacity. write (Tmemory). Hence, for each page migration, we add the time Latency Impact. ReorderForFairness is the most time- required to perform these four steps. If all GC page migrations consuming function of FLIN, as it performs two passes over the from a block are nished, FLIN adds in the time required to queue each time. Using our evaluation platform (see Section 6) erase perform the erase operation on the block (Tmemory). to model a modern SSD controller with a 500 MHz multicore FLIN determines the number of GC page movements (GCM) processor [61], we nd that ReorderForFairness incurs a that should be executed ahead of a write transaction gener- worst-case and average latency overhead of 0.5% and 0.0007%, ated by ow f , based on the fraction of all garbage collection respectively, over a state-of-the-art out-of-order transaction operations that were were previously caused by the ow. The scheduling unit [37]. None of the FLIN functions, including key insight is to choose a GCM value that distributes a larger ReorderForFairness, execute on the critical path of trans- number of GC operations to a ow that is responsible for action processing. Hence, the maximum data throughput of causing more GC operations. FLIN estimates GCM by deter- FLIN is identical to the baseline. mining (1) the fraction of all writes that were generated by ow f , since write operations lead to garbage collection (see 6. Methodology Section 2.1); and (2) the fraction of valid pages that belong to Simulation Setup. For our model-based evaluations, we ow f in a block that is selected for garbage collection (which use MQSim [101], an open-source MQ-SSD simulator that
9 accurately models all of the major front-end and back-end Table 2: Characteristics of the evaluated I/O traces. components in a modern MQ-SSD. We have open-sourced our Read Average Avg. Inter Interference implementation of FLIN [1] to foster research into fairness in Trace Ratio Size (kB) Arrival (ms) Probability modern SSDs. Table 1 shows the conguration that we use to RD WR RD WR model a representative MQ-SSD system in our evaluations. In read write dev [69] 0.68 21.4 31.5 4.1 6.1 Low our evaluation, FLIN uses αc =32 MB/s, αc =256 kB/s, F = 0.6, and GC = 1 K. exch [68] 0.66 9.7 14.8 1.0 1.3 High thr FLIN n1 [7] 0.23 2.3 3.7 14.3 6.4 Low Table 1: Conguration of the simulated SSD. n2 [7] 0.82 2.3 2.9 11.1 10.8 Low msncfs [69] 0.74 8.7 12.6 6.2 1.0 Low Host interface: PCIe 3.0 (NVMe 1.2) msnfs [69] 0.67 10.7 11.2 0.9 0.4 High SSD Organization User capacity: 480 GB prn-0 [80] 0.11 22.8 9.7 34.2 117.2 Low 8 channels, 4 dies-per-channel prn-1 [80] 0.75 22.5 11.7 30 126.7 Low prxy-0 [80] 0.03 8.3 4.6 9.1 49.4 Low Flash Communication ONFI 3.1 (NV-DDR2) prxy-1 [80] 0.65 24.6 26.1 3.7 3.4 Low Interface Width: 8 bit, Rate: 333 MT/s rad-be [69] 0.18 106.2 11.7 3.6 13.5 Low 8 KiB page, 448 B metadata-per-page, 256 rad-ps [69] 0.10 10.3 8.2 54.2 21.5 Low Flash pages-per-block, 4096 blocks-per-plane, 2 src1-0 [80] 0.56 36.2 52.0 11.8 21.7 High Microarchitecture planes-per-die src1-1 [80] 0.95 35.8 14.7 3.2 213.9 High src1-2 [80] 0.25 19.1 32.5 56.5 405.6 Low Read latency: 75 µs, Program Flash Latencies [63] stg-0 [80] 0.15 24.9 9.2 4.6 350.3 Low latency: 1300 µs, Erase latency: 3.8 ms stg-1 [80] 0.64 59.5 7.9 7.4 746.2 Low tpcc [68] 0.65 8.1 9.4 0.1 0.1 High tpce [68] 0.92 8.0 12.6 0.1 0.1 High Workloads. We study a diverse set of storage traces col- wsrch [7] 0.99 15.1 8.6 3.0 1.2 Low lected from real enterprise and datacenter workloads, which are listed in Table 2 [7,68,69,80]. We categorize these work- loads as low-interference or high-interference, based on the baselines. For workload mixes comprised of 25%, 50%, 75%, average ash chip busy time, which captures the combined and 100% high-interference ows, FLIN improves the aver- e ect of all four types of interference. We nd that a work- age fairness by 1.8x/1.3x, 2.5x/1.6x, 5.6x/2.4x, and 54x/3.2x, load interferes highly with other workloads if it keeps all of respectively, compared to Sprinkler/Sprinkler+Fairness the ash chips busy for more than 8% of its total execution (Figure 10). For the same workload mixes, FLIN reduces the time. We form MQ-SSD workloads using randomly-selected average maximum slowdown of the concurrently-running combinations of four low- and high-interference traces. We ows by 24x/2.3x, 1400x/5.5x, 3231x/12.x, and 1597x/18x, classify each workload based on the fraction of storage traces respectively (Figure 11), and reduces the STDEV of the that are high-interference. slowdown of concurrently-running ows by an average Metrics. We use four key metrics in our evaluations. The of 137x/13x, 7504x/15x, 2784x/15x, and 2850x/21.5x (Fig- rst metric, F, is strict fairness [22, 70, 73, 74], which we cal- ure 12). We make four key observations from the three culate using Eq. 2. The second metric, SMax, is the maximum gures. First, Sprinkler’s fairness decreases signicantly slowdown [17, 49, 50, 109] among all concurrently-running as the fraction of high-interference ows in a workload in- ows. The third metric, STDEV, is the standard deviation of creases, and approaches zero in workloads with 100% high- the slowdowns of all concurrently-running ows. The fourth interference ows. Second, Sprinkler+Fairness improves metric, WS, measures the weighted speedup [93] of MQ-SSD’s fairness over Sprinkler due to its inclusion of fairness con- response time (RT), which represents the overall e ciency of trol, but Sprinkler+Fairness does not consider all sources the I/O scheduler for all concurrent ows. Weighted speedup of interference, and therefore has a much lower fairness than is calculated as: FLIN. Third, across all of our workloads, no ow has a max- X RT alone WS = i (11) imum slowdown greater than 80x under FLIN, even though shared there are several ows that have maximum slowdowns over i RTi alone 500x with Sprinkler and Sprinkler+Fairness. This indi- where RTi is the average response time of transactions cates that FLIN provides signicantly better fairness over both shared from ow i when the ow is run alone, and RTi is the baseline schedulers. Fourth, across all of our workloads, the average response time when the ow runs concurrently with STDEV of the slowdown is always lower than 10 with FLIN, other ows. while it ranges between 100 and 1000 for many ows under the baseline schedulers. This means that FLIN distributes 7. Evaluation the impact of interference more fairly across each ow in We compare FLIN to two state-of-the-art baselines. Our rst the workload. We conclude that FLIN’s slowdown manage- baseline (Sprinkler) uses three state-of-the-art scheduling ment scheme e ectively improves MQ-SSD fairness over both techniques: (1) Sprinkler for out-of-order transaction schedul- state-of-the-art baselines. ing in the TSU [37], (2) semi-preemptive GC execution [52], and (3) read prioritization and write/erase suspension [112]. Figure 13 compares the weighted speedup of FLIN Our second baseline (Sprinkler+Fairness) combines Sprin- against the two baselines. We observe from the g- kler with a state-of-the-art in-controller MQ-SSD fairness ure that for 25%, 50%, 75%, and 100% high-interference control mechanism [36]. workload mixes, FLIN improves the weighted speedup by 38%/21%, 74%/32%, 132%/41%, and 156%/76%, on average, over 7.1. Fairness and Performance of FLIN Sprinkler/Sprinkler+Fairness. The improvement is a re- Figures 10, 11, and 12 compare the fairness, maximum sult of FLIN’s fairness control mechanism, which removes slowdown, and STDEV, respectively, of FLIN against the two performance bottlenecks resulting from high unfairness. FLIN
10 Sprinkler Sprinkler+Fairness FLIN
1.0 1.0 1.0 1.0 1.0
0.8 0.8 0.8 0.8 0.8
0.6 0.6 0.6 0.6 0.6
0.4 0.4 0.4 0.4 0.4
0.2 0.2 0.2 0.2 0.2 Fairness
0.0 0.0 0.0 0.0 Mix1 Mix2 Mix3 Mix4 Mix5 Mix6 Mix7 Mix8 Mix9 Mix10AVG Mix1 Mix2 Mix3 Mix4 Mix5 Mix6 Mix7 Mix8 Mix9 Mix10AVG Mix1 Mix2 Mix3 Mix4 Mix5 Mix6 Mix7 Mix8 Mix9 Mix10AVG Mix1 Mix2 Mix3 Mix4 Mix5 Mix6 Mix7 Mix8 Mix9 Mix10AVG0.0 (Larger is Better) 25% High-Interference 50% High-Interference 75% High-Interference 100% High-Interference Figure 10: Fairness of FLIN vs. baseline schedulers.
100000 100000 100000 100000 100000
10000 10000 10000 10000 10000
1000 1000 1000 1000 1000
100 100 100 100 100
10 10 10 10 10
1 1 1 1 Mix1 Mix2 Mix3 Mix4 Mix5 Mix6 Mix7 Mix8 Mix9 Mix10AVG Mix1 Mix2 Mix3 Mix4 Mix5 Mix6 Mix7 Mix8 Mix9 Mix10AVG Mix1 Mix2 Mix3 Mix4 Mix5 Mix6 Mix7 Mix8 Mix9 Mix10AVG Mix1 Mix2 Mix3 Mix4 Mix5 Mix6 Mix7 Mix8 Mix9 Mix10AVG 1
(Smaller is Better) 25% High-Interference 50% High-Interference 75% High-Interference 100% High-Interference Maximum Slowdown Figure 11: Maximum slowdown of FLIN vs. baseline schedulers.
100000 100000 100000 100000 100000
10000 10000 10000 10000 10000
1000 1000 1000 1000 1000
100 100 100 100 100
10 10 10 10 10 STDEV 1 1 1 1 Mix1 Mix2 Mix3 Mix4 Mix5 Mix6 Mix7 Mix8 Mix9 Mix10AVG Mix1 Mix2 Mix3 Mix4 Mix5 Mix6 Mix7 Mix8 Mix9 Mix10AVG Mix1 Mix2 Mix3 Mix4 Mix5 Mix6 Mix7 Mix8 Mix9 Mix10AVG Mix1 Mix2 Mix3 Mix4 Mix5 Mix6 Mix7 Mix8 Mix9 Mix10AVG 1
(Smaller is Better) 25% High-Interference 50% High-Interference 75% High-Interference 100% High-Interference Figure 12: STDEV slowdown of FLIN vs. baseline schedulers.
4 4 4 4 4
3 3 3 3 3
2 2 2 2 2
1 1 1 1 1
0 0 0 0 Mix1 Mix2 Mix3 Mix4 Mix5 Mix6 Mix7 Mix8 Mix9 Mix10AVG Mix1 Mix2 Mix3 Mix4 Mix5 Mix6 Mix7 Mix8 Mix9 Mix10AVG Mix1 Mix2 Mix3 Mix4 Mix5 Mix6 Mix7 Mix8 Mix9 Mix10AVG Mix1 Mix2 Mix3 Mix4 Mix5 Mix6 Mix7 Mix8 Mix9 Mix10AVG 0 (Larger is Better) Weighted Speedup 25% High-Interference 50% High-Interference 75% High-Interference 100% High-Interference Figure 13: Weighted speedup of FLIN vs. baseline schedulers.
improves the performance of low-interference ows while Sprinkler Stage 1 Stage 3 FLIN throttling the performance of high-interference ows, which 1.0 4.0 104 0.8 3.0 improves weighted speedup by enabling low-intensity ows 3 0.6 10 Sprinkler 2 2.0 to make fast progress. In contrast, typically pri- 0.4 10
oritizes transactions from high-interference ows. This im- Fairness 0.2 10 1.0
0.0 Max Slowdown 1 0.0 proves the performance of high-interference ows, but penal- 25% 50% 75% 100% 25% 50% 75% 100% 25% 50% 75% 100% izes the low-interference ows and, thus, reduces the overall Fraction of High-Intensity Fraction of High-Intensity Weighted Speedup Fraction of High-Intensity weighted speedup by causing all ows to progress relatively Figure 14: E ect of Stages 1 and 3 of FLIN. slowly. Sprinkler+Fairness improves the performance of low-interference ows by controlling the throughput of high- of interference among concurrently-running ows, but each interference ows, but its weighted speedup remains low stage alone falls short of the improvements we observe when because the throughput control mechanism leaves many re- FLIN as a whole is used. Second, the fairness and perfor- sources idle in the SSD [89]. We conclude that FLIN’s fairness mance improvements of Stage 1 are much higher than those control mechanism signicantly improves weighted speedup, of Stage 3. This is because I/O intensity is the most dominant an important metric for overall MQ-SSD performance, over source of interference. Since Stage 1 is designed to mitigate state-of-the-art device-level schedulers. interference due to di erent I/O intensities, it delivers larger 7.2. E ect of Di erent FLIN Stages improvements than Stage 3. Third, Stage 3 reduces the max- imum slowdown by a greater amount than Stage 1. This is 1.0 We analyze the fairness, maximum slowdown, and because GC operations can signicantly increase the stall weighted speedup improvements when using only (1) the time of transactions, causing a large increase in maximum 0.8 rst stage of FLIN (Section 4.1), or (2) the third stage of FLIN slowdown, and Stage 3 manages such GC interference e ec- (Section 4.3), as these two stages are designed to reduce the tively. We conclude that Stages 1 and 3 of FLIN are e ective 0.6 four sources of interference listed in Section 3. Figure 14 at mitigating di erent sources of interference, and that FLIN shows the fairness (left), maximum slowdown (center), and makes e ective use of both stages to maximize fairness and 0.4 weighted speedup of Sprinkler, Stage 1 of FLIN only, Stage 3 performance improvements over state-of-the-art schedulers. of FLIN only, and all three stages of FLIN. For each category, 0.2 the gure shows the average value of each metric across all 7.3. Sensitivity to FLIN and MQ-SSD Parameters workloads in the category. We evaluate the sensitivity of fairness to four of FLIN’s 0.0 We make three observations from Figure 14. First, the indi- parameters, as shown in Figure 15: (1) the epoch length for de- vidual stages of FLIN improve both fairness and performance termining I/O intensity (Section 5.1), (2) the threshold for high read over Sprinkler, as each stage works to reduce some sources read intensity (αC ; Section 5.1), (3) the fairness threshold for
11 60x high-intensity transaction reordering (Fthr ; Section 4.1), and 50 10000 GC 40 40x 1000 (4) the GC management threshold ( FLIN ; Section 4.3). We 30 FLIN-NoCache FLIN-WithCache 100 sweep the values of each parameter individually. We make 20 20x four observations from the gure. First, for epoch length, 10 Fariness 10 Slowdown Slowdown 0 Improvement 25% 1 50% 75% 100% we observe from Figure 15a that epoch lengths of 10–100 ms Workload Category achieve the highest fairness, indicating that at these lengths, msnfs-3 msnfs-2 msnfs-1 msnfs-0 msnfstpce exchsrc-1-1 FLIN can best di erentiate between low-intensity and high- Figure 16: E ects of varying Figure 17: E ect of write read ow priorities. caching. intensity ows. Second, for αC , we observe from Figure 15b read 7.5. E ect of Write Caching that if αC is too small (e.g., below 32 MB/s), fairness de- creases, as low-intensity ows are incorrectly classied as Modern MQ-SSDs employ a front end DRAM write bu er high-intensity ows. Third, we observe in Figure 15c that in order to bridge the performance gap between the fast host for workloads where at least half of the ows are high inten- DRAM and the slow SSD memory chips. To investigate the sity, fairness is maximized for moderate values of F (e.g., e ects of write caching on fairness, we conduct a set of ex- thr periments assuming a 128 MB write cache per running ow Fthr = 0.6). This is because if Fthr is too low, FLIN does not prioritize the ow with the maximum slowdown very often (a total of 512 MB write cache). Figure 17 shows the average fairness improvement of FLIN with and without a write bu er. (see Section 4.1, Step 2b), which keeps fairness low. If Fthr is too high, then FLIN almost always prioritizes the ow that We observe that caching has only a minimal e ect on FLIN’s currently has the maximum slowdown, which causes the other fairness benets, for two reasons. First, write caching has no ows to stall for long periods of time, and increases their slow- immediate e ect on read transactions, so both I/O intensity down. Fourth, we nd that fairness is not highly sensitive and access pattern interference are not a ected when write caching is enabled. Second, high-interference ows typically to the GCFLIN parameter, though smaller values of GCFLIN provide slightly higher fairness because they cause FLIN to have high I/O intensity. When the write intensity of a ow defer GC operations for longer. is high, the write cache lls up and cannot cache any more requests. At this point, known as the write cli , any write 25% 50% 75% 100% requests that cannot be cached are directly issued to the back 1.0 1.0 (a) (b) 0.8 0.8 end [39], eliminating the benets of the write cache. We con- 0.6 0.6 clude that write caching has only a minimal e ect on the 0.4 0.4 performance of FLIN.
Fairness 0.2 Fairness 0.2 0.0 0.0 0.1 1 10 100 1000 8 16 32 64 256 α read 8. Related Work Epoch Length (ms) c (MB/s) 1.0 1.0 To our knowledge, this is the rst work to (1) thoroughly (c) (d) 0.8 0.8 analyze the sources of interference among concurrently- 0.6 0.6 0.4 0.4 running I/O ows in MQ-SSDs; and (2) propose a new device-
Fairness 0.2 Fairness 0.2 level I/O scheduler for modern SSDs that e ectively miti- 0.0 0.0 0.2 0.4 0.6 0.8 1 1K 10K 100K 1M 10M F GC gates interference among concurrently-running I/O ows thr FLIN to provide both high fairness and performance. Provid- Figure 15: Fairness with FLIN congurations. ing fairness and performance isolation among concurrently- We also evaluate the sensitivity of FLIN’s fairness and per- running I/O ows is an active area of research for modern formance benets to di erent MQ-SSD congurations. We SSDs [2, 28, 36, 45, 46, 84, 89, 94, 120], especially for mod- make three observations from these studies (not shown). First, ern MQ-SSD devices that support new host–interface pro- a larger page size decreases the number of transactions in tocols [36, 101], such as the NVMe protocol. the back end and, therefore, reduces the probability of in- Fair Disk Schedulers. Many works [3,9,34,86,95,113,116] terference. This reduces the benets that FLIN can deliver. propose OS-level disk schedulers to meet fairness and real- Second, newer ash memory technologies have higher ash time requirements in hard drives. These approaches are not read and write latencies [24,100], which results in an increase suitable for SSDs as they (1) essentially estimate I/O access la- in the busy time of the ash chips. This leads to a higher tencies based on the physical structure of rotating disk drives, probability of interference, and increases both the fairness which is di erent in SSDs; and (2) assume read/write latencies and performance benets of FLIN. Third, having a larger num- that are orders of magnitude longer than those of SSDs [47]. ber of memory chips reduces the interference probability, by More recent OS-level schedulers have evolved to better t distributing transactions over a larger number of chip-level the internal characteristics of SSDs [84, 89, 120]. FIOS [84] queues, and thus reduces FLIN’s ability to improve fairness. periodically assigns an equal number of time-slices to the running ows. The ows then trade these time-slices for 7.4. Evaluation of Support for Flow Priorities scheduling I/O requests based on the ash read/write ser- The priority-based queue arbitration stage of FLIN supports vice time costs. BCQ [120] adopts a similar approach with a ow-level priorities. We evaluate FLIN’s support for ow pri- higher accuracy in determining read/write costs. Both FIOS orities in a variety of scenarios and present a representative and BCQ su er from reduced responsiveness of the storage case study to highlight its e ectiveness. Figure 16 shows the device and reduced overall performance [89]. More precisely, slowdown of four msnfs ows concurrently running with a ow that consumes its time budget early would have to priority levels 0, 1, 2, and 3. With FLIN, we observe that wait for other ows to nish before its budget is replenished the highest-priority ow (the ow assigned priority level 3; in the next round, leading to a period of unresponsiveness msnfs-3 in Figure 16) has the lowest slowdown, as the transac- and resource idleness [89]. FlashFQ [89] is another OS-level tions of this ow are assigned more than half of the scheduling scheduler that eliminates such unresponsiveness. FlashFQ slots. The slowdown of each lower-priority version of msnfs estimates the progress of each running ow based on the increases proportionally with the ow’s priority. cost of ash read/write operations, and allows a ow to dis-
12 patch I/O requests as long as its progress is not much further High-Performance Device Schedulers. The vast major- ahead of other ows. If a ow’s progress is far ahead of other ity of device-level I/O schedulers [20, 37, 40, 77, 90] try to concurrently-running ows, FlashFQ throttles the ow that e ectively exploit SSD internal parallelism with out-of-order is far ahead to improve fairness. execution of ash transactions. These schedulers are designed FIOS, BCQ, and FlashFQ su er from two major shortcom- for performance, and are vulnerable to inter- ow interference. ings compared to FLIN. First, they do not have information Performance Evaluation of NVMe SSDs. A number of about the internal resources and mechanisms of an SSD, and prior works focus on the performance evaluation and imple- only speculate about resource availability and the behavior mentation issues of MQ-SSDs [5, 114, 115]. Xu et al. [114] of SSD management algorithms when making a scheduling analyze the performance implications of MQ-SSDs for mod- decision. Such speculation-based scheduling decisions can be ern database applications. Awad et al. [5] explore di erent suboptimal for mitigating GC and parallelism interference, as implementation aspects of NVMe via simulation, and show both types of interference are highly dependent on internal how these aspects a ect system performance. None of these SSD mechanisms. In contrast, FLIN has direct information works consider the fairness issues in MQ-SSDs. about the status of pending requests (in the I/O queue) and the internal SSD resources/mechanisms. Second, OS-level 9. Conclusion techniques have no direct control over the SSD’s internal mech- We propose FLIN, a lightweight transaction scheduler for anisms. Therefore, it is likely that there is some inter- ow modern multi-queue SSDs (MQ-SSDs), which provides fair- interference within the SSD, even when the OS-level scheduler ness among concurrently-running ows. FLIN uses a three- makes use of fairness control mechanisms. For example, the stage design to protect against all four major sources of TSU may execute GC activities due to a high-GC ow, which interference that exist in real MQ-SSDs, while enforcing can block the transactions of a low-GC ow. In contrast, FLIN application-level priorities that are assigned by the host. Our takes GC activities into account (e.g., Algorithm 2, which extensive evaluations show that FLIN e ectively improves works to minimize GC interference), and controls all major both fairness and system performance compared to state-of- sources of interference in an MQ-SSD. the-art device-level schedulers. FLIN is simple to implement While our goal is to design FLIN to exploit the many ad- within the SSD controller rmware, requires no additional vantages of performing scheduling and fairness control in hardware, and consumes less than 0.06% of the storage space the SSD instead of the OS, FLIN can also work in conjunc- available in the in-SSD DRAM. We conclude that FLIN is a tion with OS-level I/O resource control mechanisms to exploit promising scheduling mechanism that enables the design of the best of both worlds. For example, virtualized environ- fair and high-performance MQ-SSDs. ments require OS-level management to control I/O requests Acknowledgments across multiple VMs/containers [43]. One could envision a We thank the anonymous referees of ISCA 2018 and AS- system where the OS and FLIN work together to perform PLOS 2018. We thank SAFARI group members for feedback coordinated scheduling and fairness control. For example, the and the stimulating research environment. We thank our OS could make application-/VM-aware scheduling and fair- industrial partners, especially Alibaba, Google, Huawei, In- ness decisions, while FLIN performs resource-aware schedul- tel, Microsoft, and VMware, for their generous support. Lois ing/fairness decisions. We leave an exploration of coordinated Orosa was supported by FAPESP fellowship 2016/18929-4. OS/FLIN mechanisms to future work. References Fair Main Memory Scheduling. There is a large collec- [1] “MQSim GitHub Repository,” https://github.com/CMU-SAFARI/MQSim. tion of prior works that propose fair memory request sched- [2] S. Ahn et al., “Improving I/O Resource Sharing of Linux Cgroup for NVMe SSDs ulers for DRAM [4,18,19,44,49,50,71–75,81,96–99,105]. These on Multi-Core Systems,” in HotStorage, 2016. [3] W. G. Aref et al., “Scalable QoS-Aware Disk-Scheduling,” in IDEAS, 2002. schedulers cannot be directly used in SSDs, as DRAM devices [4] R. Ausavarungnirun et al., “Staged Memory Scheduling: Achieving High Perfor- (1) operate at a much smaller granularity than SSDs (64 B in mance and Scalability in Heterogeneous Systems,” in ISCA, 2012. DRAM vs. 8 kB in SSDs); (2) have symmetric read and write [5] A. Awad et al., “Non-Volatile Memory Host Controller Interface Performance Analysis in High-Performance I/O Systems,” in ISPASS, 2015. latencies; (3) do not perform garbage collection; (4) have row [6] J. Axboe, “Linux Block I/O—Present and Future,” in Ottawa Linux Symp., 2004. bu ers in banks that signicantly a ect performance; and [7] K. Bates and B. McNutt, “UMass Trace Repository.” (5) perform periodic refresh operations. [8] M. Bjørling et al., “Linux Block IO: Introducing Multi-Queue SSD Access on Multi- Core Systems,” in SYSTOR, 2013. [9] J. Bruno et al., “Disk Scheduling with Quality of Service Guarantees,” in ICMCS, SSD Performance Isolation. Performance isolation [15, 1999. 25, 28, 29, 46, 78, 91, 94] aims to provide predictable perfor- [10] Y. Cai et al., “Error Characterization, Mitigation, and Recovery in Flash-Memory- Based Solid-State Drives,” Proc. IEEE, 2017. mance by eliminating interference from di erent tenants that [11] Y. Cai et al., “Read Disturb Errors in MLC NAND Flash Memory: Characterization share an SSD. Inter-tenant interference can be reduced by and Mitigation,” in DSN, 2015. [12] Y. Cai et al., “Data Retention in MLC NAND Flash Memory: Characterization, (1) assigning di erent hardware resources to di erent ten- Optimization, and Recovery,” in HPCA, 2015. ants [28, 29, 46, 78, 94] (e.g., di erent tenants are mapped [13] Y. Cai et al., “Errors in Flash-Memory-Based Solid-State Drives: Analysis, Mitiga- to di erent channels/dies), or (2) using isolation-aware al- tion, and Recovery,” arXiv:1711.11427 [cs:AR], 2017. [14] Y. Cai et al., “Neighbor-Cell Assisted Error Correction for MLC NAND Flash Mem- gorithms for time sharing the hardware resources between ories,” in SIGMETRICS, 2014. tenants [15, 25, 91, 94] (e.g., credit-based algorithms). [15] D.-W. Chang et al., “VSSD: Performance Isolation in a Solid-State Drive,” TODAES, 2015. These techniques have three main di erences from FLIN. [16] L.-P. Chang et al., “Real-Time Garbage Collection for Flash-Memory Storage Sys- First, they do not provide fairness control. Instead, they re- tems of Real-Time Embedded Systems,” TECS, 2004. [17] R. Das et al., “Application-Aware Prioritization Mechanisms for On-Chip Net- duce the interference between ows without considering any works,” in MICRO, 2009. fairness metric (e.g., slowdown), and do not evaluate fairness [18] E. Ebrahimi et al., “Parallel Application Memory Scheduling,” in MICRO, 2011. [19] E. Ebrahimi et al., “Fairness via Source Throttling: A Congurable and High- quantitatively. Second, they are not designed to optimize Performance Fairness Substrate for Multi-Core Memory Systems,” in ASPLOS, the utilization of the MQ-SSD resources, which limits their 2010. [20] N. Elyasi et al., “Exploiting Intra-Request Slack to Improve SSD Performance,” in performance. Third, no existing approach considers all four ASPLOS, 2017. interference sources considered by FLIN. [21] G2M Research, “NVMe Market Forecast & Vendor Report Abstract,” 2017.
13 [22] R. Gabor et al., “Fairness and Throughput in Switch on Event Multithreading,” in [71] T. Moscibroda and O. Mutlu, “Distributed Order Scheduling and Its Application MICRO, 2006. to Multi-Core DRAM Controllers,” in PODC, 2008. [23] E. Gal and S. Toledo, “Algorithms and Data Structures for Flash Memories,” CSUR, [72] S. P. Muralidhara et al., “Reducing Memory Interference in Multicore Systems via 2005. Application-aware Memory Channel Partitioning,” in MICRO, 2011. [24] L. M. Grupp et al., “The Bleak Future of NAND Flash Memory,” in FAST, 2012. [73] O. Mutlu and T. Moscibroda, “Stall-Time Fair Memory Access Scheduling for Chip [25] A. Gulati et al., “PARDA: Proportional Allocation of Resources for Distributed Multiprocessors,” in MICRO, 2007. Storage Access,” in FAST, 2009. [74] O. Mutlu and T. Moscibroda, “Parallelism-Aware Batch Scheduling: Enhancing [26] J. Guo et al., “Parallelism and Garbage Collection Aware I/O Scheduler with Im- Both Performance and Fairness of Shared DRAM Systems,” in ISCA, 2008. proved SSD Performance,” in IPDPS, 2017. [75] O. Mutlu and L. Subramanian, “Research Problems and Opportunities in Memory [27] S. S. Hahn et al., “To Collect or Not to Collect: Just-in-Time Garbage Collection Systems,” SUPERFRI, 2015. for High-performance SSDs with Long Lifetimes,” in DAC, 2015. [76] J. Nagle, “On Packet Switches with Innite Storage,” TCOM, 1985. [28] J. Huang et al., “FlashBlox: Achieving Both Performance Isolation and Uniform [77] E. H. Nam et al., “Ozone (O3): An Out-of-order Flash Memory Controller Archi- Lifetime for Virtualized SSDs,” in FAST, 2017. tecture,” TC, 2011. [29] S.-M. Huang and L.-P. Chang, “Providing SLO Compliance on NVMe SSDs [78] M. Nanavati et al., “Decibel: Isolation and Sharing in Disaggregated Rack-Scale Through Parallelism Reservation,” TODAES, 2018. Storage,” in NSDI, 2017. [30] Huawei Technologies Co., Ltd., “Huawei ES3000 V3 NVMe SSD White Paper,” [79] D. Narayanan et al., “Write O -Loading: Practical Power Management for Enter- 2017. prise Storage,” ToS, 2008. [31] Intel Corp., “Intel Solid-State Drive DC P3600 Series, Product Specication,” 2014. [80] D. Narayanan et al., “Migrating Server Storage to SSDs: Analysis of Tradeo s,” in [32] Intel Corp., “Intel Solid-State Drive DC P3500 Series, Product Specication,” 2015. Eurosys, 2009. [33] Intel Corp., “Intel 3D NAND SSD DC P4500 Series, Product Brief,” 2018. [81] K. J. Nesbit et al., “Fair Queuing Memory Systems,” in MICRO, 2006. [34] S. Iyer and P. Druschel, “Anticipatory Scheduling: A Disk Scheduling Framework [82] D. M. Nguyen et al., “Solving the Multidimensional Assignment Problem by a to Overcome Deceptive Idleness in Synchronous I/O,” in SOSP, 2001. Cross-Entropy Method,” JCO, 2014. [35] W. Jin et al., “Interposed Proportional Sharing for a Storage Service Utility,” in [83] NVM Express Workgroup, “NVM Express Specication, Revision 1.2,” 2014. SIGMETRICS, 2004. [84] S. Park and K. Shen, “FIOS: A Fair, E cient Flash I/O Scheduler,” in FAST, 2012. [36] B. Jun and D. Shin, “Workload-aware Budget Compensation Scheduling for [85] W. P. Pierskalla, “The Multidimensional Assignment Problem,” OR, 1968. NVMe Solid State Drives,” in NVMSA, 2015. [86] P. E. Rocha and L. C. E. Bona, “A QoS Aware Non-Work-Conserving Disk Sched- [37] M. Jung and M. T. Kandemir, “Sprinkler: Maximizing Resource Utilization in uler,” in MSST, 2012. Many-chip Solid State Disks,” in HPCA, 2014. [87] Samsung Electronics Co., Ltd., “Samsung SSD 960 PRO M.2, Data Sheet,” 2017. [38] M. Jung et al., “HIOS: A Host Interface I/O Scheduler for Solid State Disks,” in [88] SATA-IO, “Serial ATA Revision 3.3,” http://www.sata-io.org, 2016. ISCA, 2014. [39] M. Jung and M. Kandemir, “Revisiting Widely Held SSD Expectations and Re- [89] K. Shen and S. Park, “FlashFQ: A Fair Queueing I/O Scheduler for Flash-Based thinking System-Level Implications,” in SIGMETRICS, 2013. SSDs,” in ATC, 2013. [90] J.-Y. Shin et al., “Exploiting Internal Parallelism of Flash-based SSDs,” CAL, 2010. [40] M. Jung et al., “PAQ: Physically Addressed Queuing for Resource Con ict Avoid- ance in Solid State Disk,” in ISCA, 2012. [91] D. Shue et al., “Performance Isolation and Fairness for Multi-Tenant Cloud Stor- [41] J.-U. Kang et al., “The Multi-streamed Solid-state Drive,” in HotStorage, 2014. age,” in OSDI, 2012. [42] S. Kavalanekar et al., “Characterization of Storage Workload Traces from Produc- [92] SK Hynix Inc., “F26 32Gb MLC NAND Flash Memory TSOP Legacy,” 2011. tion Windows Servers,” in IISWC, 2008. [93] A. Snavely and D. M. Tullsen, “Symbiotic Jobscheduling for a Simultaneous Mul- [43] H.-J. Kim et al., “NVMeDirect: A User-space I/O Framework for Application- tithreaded Processor,” in ASPLOS, 2000. specic Optimization on NVMe SSD,” in HotStorage, 2016. [94] X. Song et al., “Architecting Flash-Based Solid-State Drive for High-Performance [44] H. Kim et al., “Bounding Memory Interference Delay in COTS-Based Multi-Core I/O Virtualization,” CAL, 2014. Systems,” in RTAS, 2014. [95] M. J. Stanovich et al., “Throttling On-Disk Schedulers to Meet Soft-Real-Time Requirements,” in RTAS, 2008. [45] J. Kim et al., “I/O Scheduling Schemes for Better I/O Proportionality on Flash- Based SSDs,” in MASCOTS, 2016. [96] L. Subramanian et al., “The Blacklisting Memory Scheduler: Achieving High Per- [46] J. Kim et al., “Towards SLO Complying SSDs Through OPS Isolation,” in FAST, formance and Fairness at Low Cost,” in ICCD, 2014. 2015. [97] L. Subramanian et al., “BLISS: Balancing Performance, Fairness and Complexity [47] J. Kim et al., “Disk Schedulers for Solid State Drivers,” in EMSOFT, 2009. in Memory Access Scheduling,” TPDS, 2016. [48] T. Y. Kim et al., “Improving Performance by Bridging the Semantic Gap Between [98] L. Subramanian et al., “MISE: Providing Performance Predictability and Improv- Multi-queue SSD and I/O Virtualization Framework,” in MSST, 2015. ing Fairness in Shared Main Memory Systems,” in HPCA, 2013. [49] Y. Kim et al., “ATLAS: A Scalable and High-Performance Scheduling Algorithm [99] L. Subramanian et al., “The Application Slowdown Model: Quantifying and Con- for Multiple Memory Controllers,” in HPCA, 2010. trolling the Impact of Inter-Application Interference at Shared Caches and Main [50] Y. Kim et al., “Thread Cluster Memory Scheduling: Exploiting Di erences in Memory,” in MICRO, 2015. Memory Access Behavior,” in MICRO, 2010. [100] K. Suzuki and S. Swanson, “A Survey of Trends in Non-Volatile Memory Tech- [51] J. Lee et al., “Preemptible I/O Scheduling of Garbage Collection for Solid State nologies: 2000-2014,” in IMW, 2015. Drives,” TCAD, 2013. [101] A. Tavakkol et al., “MQSim: A Framework for Enabling Realistic Studies of Mod- [52] J. Lee et al., “A Semi-preemptive Garbage Collector for Solid State Drives,” in ern Multi-Queue SSD Devices,” in FAST, 2018. ISPASS, 2011. [102] A. Tavakkol et al., “Performance Evaluation of Dynamic Page Allocation Strate- [53] Y. Li et al., “Stochastic Modeling of Large-Scale Solid-State Storage Systems: Anal- gies in SSDs,” ToMPECS, 2016. ysis, Design Tradeo s and Optimization,” in SIGMETRICS, 2013. [103] Toshiba Corp., “OCZ RD400/400A Series, Product Brief,” 2016. [54] M. Lin and S. Chen, “E cient and Intelligent Garbage Collection Policy for NAND [104] Toshiba Corp., “PX04PMC Series, Data Sheet,” 2016. Flash-based Consumer Electronics,” TCE, 2013. [105] H. Usui et al., “DASH: Deadline-Aware High-Performance Memory Scheduler for [55] Linux Kernel Organization, Inc., “Block IO Priorities,” https://www.kernel.org/ Heterogeneous Systems with Hardware Accelerators,” TACO, 2016. doc/Documentation/block/ioprio.txt. [106] P. Valente and F. Checconi, “High Throughput Disk Scheduling with Fair Band- [56] Linux Kernel Organization, Inc., “CFQ (Complete Fairness Queueing),” https:// width Distribution,” TC, 2010. www.kernel.org/doc/Documentation/block/cfq-iosched.txt. [107] B. Van Houdt, “A Mean Field Model for a Class of Garbage Collection Algorithms [57] Y. Luo et al., “WARM: Improving NAND Flash Memory Lifetime With Write- in Flash-based Solid State Drives,” in SIGMETRICS, 2013. Hotness Aware Retention Management,” in MSST, 2015. [108] B. Van Houdt, “On the Necessity of Hot and Cold Data Identication to Reduce [58] Y. Luo et al., “Enabling Accurate and Practical Online Flash Channel Modeling the Write Amplication in Flash-based SSDs,” Perform. Eval., 2014. for Modern MLC NAND Flash Memory,” JSAC, 2016. [109] H. Vandierendonck and A. Seznec, “Fairness Metrics for Multi-Threaded Proces- [59] Y. Luo et al., “HeatWatch: Improving 3D NAND Flash Memory Device Reliability sors,” CAL, 2011. by Exploiting Self-Recovery and Temperature Awareness,” in HPCA, 2018. [110] Western Digital Technologies, Inc., “Skyhawk & Skyhawk Ultra NVMe PCIe SSD, [60] Marvell Technology Group, Ltd., “Marvell 88NV9145 PCIe-NAND controller,” Data Sheet,” 2017. 2012. [111] Western Digital Technologies, Inc., “Ultrastar SN200 Series, Data Sheet,” 2017. [61] Marvell Technology Group, Ltd., “Marvell 88SS1093 Flash Memory Controller,” [112] G. Wu and X. He, “Reducing SSD Read Latency via NAND Flash Program and 2017. Erase Suspension,” in FAST, 2012. [62] Micron Technology, Inc., “64Gb, 128Gb, 256Gb, 512Gb Asyn- [113] J. C. Wu et al., “Hierarchical Disk Sharing for Multimedia Systems,” in NOSSDAV, chronous/Synchronous NAND,” 2009. 2005. [63] Micron Technology, Inc., “NAND Flash Memory MLC MT29F256G08CKCAB [114] Q. Xu et al., “Performance Characterization of Hyper-scale Applications on Datasheet,” 2014. NVMe SSDs,” in SIGMETRICS, 2015. [64] Micron Technology, Inc., “128Gb, 256Gb, 512Gb Async/Sync Enterprise NAND,” [115] Q. Xu et al., “Performance Analysis of NVMe SSDs and their Implication on Real 2016. World Databases,” in SYSTOR, 2015. [65] Micron Technology, Inc., “9100 U.2 and HHHL NVMe PCIe SSDs,” 2016. [116] Y. Xu and S. Jiang, “A Scheduling Framework That Makes Any Disk Schedulers [66] Micron Technology, Inc., “Micron 9200 NVMe SSDs,” 2016. Non-Work-Conserving Solely Based on Request Characteristics,” in FAST, 2011. [67] Micron Technology, Inc., “MLC 128Gb to 2Tb Enterprise Async/Sync NAND,” [117] S. Yan et al., “Tiny-Tail Flash: Near-Perfect Elimination of Garbage Collection 2016. Tail Latencies in NAND SSDs,” in FAST, 2017. [68] Microsoft Corp., “Microsoft Enterprise Traces,” http://iotta.snia.org/traces/130. [118] S. Yang et al., “Split-Level I/O Scheduling,” in SOSP, 2015. [69] Microsoft Corp., “Microsoft Production Server Traces,” http://iotta.snia.org/ [119] J. Zhang et al., “Synthesizing Representative I/O Workloads for TPC-H,” in HPCA, traces/158. 2004. [70] T. Moscibroda and O. Mutlu, “Memory Performance Attacks: Denial of Memory [120] Q. Zhang et al., “An E cient, QoS-Aware I/O Scheduler for Solid State Drive,” in Service in Multi-Core Systems,” in USENIX Security, 2007. HPCC EUC, 2013.
14