DOCSLIB.ORG

Enhancing the Performance of Openldap Directory Server with Multiple Caching

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Enhancing the Performance of Openldap Directory Server with Multiple Caching Enhancing the Performance of OpenLDAP Directory Server with Multiple Caching Jong Hyuk Choi and Hubertus Franke Kurt Zeilenga IBM Thomas J. Watson Research Center The OpenLDAP Foundation P.O. Box 218, Yorktown Heights, NY 10598 [email protected] jongchoi,frankeh¡ @us.ibm.com Keywords—Directory, LDAP, Caching, Linux, Profiling. This paper presents our efforts to enhance OpenLDAP [2], Abstract— Directory is a specialized data store optimized for ef- an open-source directory software suite. More specifically, this ficient information retrieval which has standard information model, paper focuses on the performance and reliability of slapd, the naming scheme, and access protocol for interoperability over net- standalone LDAP server of OpenLDAP. We participated in the work. It stores critical data such as user, resource, and policy infor- mation in the enterprise computing environment. This paper presents development of back-bdb, a transactional backend of OpenL- a performance driven design of a transactional backend of the OpenL- DAP, which directly utilizes the underlying Berkeley DB with- DAP open-source directory server which provides improved reliabil- out the general backend API layer used in the existing OpenL- ity with high performance. Based on a detailed system-level profiling DAP backend, back-ldbm. Back-bdb provides a higher level of of OpenLDAP on the Linux OS, we identify major performance bot- reliability and concurrency. The transaction support of back- tlenecks. After investigating the characteristics of each bottleneck, we propose a set of caching techniques in order to eliminate them: bdb makes directory recoverable from temporary failures and directory entry cache, search candidate cache, IDL (ID List) cache, disasters, while the page level locking enables concurrent ac- IDL stack slab cache, and BER (Basic Encoding Rule) transfer cache. cess to directory by the directory server and various admin- The performance evaluation with a directory workload convinces that istrative tools at the same time. However, because the initial these caches, when combined, yields a 126% throughput increase and design of back-bdb did not exhibit the expected performance, a 59% latency reduction with a reasonable level of storage overhead. we analyzed its performance through a detailed system-level profiling. Based on the bottleneck identification and analysis, we propose five distinct caches for OpenLDAP back-bdb: en- I. INTRODUCTION try cache, candidate cache, IDL (ID List) cache, slab cache for A directory provides a logically centralized view of informa- the IDL stack, and BER (Basic Encoding Rule) [3] cache for tion in a distributed environment, enabling different platforms transfer contents. The combined use of these caches yields a to access a shared, consistent information base. By sharing performance improvement of 126%. This paper analyzes the critical information such as user, resource, and policy data, in- efficacy and the performance impact of these caches in detail. teroperability among heterogeneous systems and services can The next section will introduce LDAP and the OpenLDAP be significantly enhanced. As IT services become available to open-source project. Section III will describe the architecture customers in a dynamic, on-demand basis, it becomes vital to of OpenLDAP directory server, slapd, focusing on the design provide standard ways of information access and sharing. of the search operation in back-bdb. Section IV will introduce LDAP (Lightweight Directory Access Protocol) is a stan- five caching approaches proposed in this paper. Section V will dard access protocol for OSI X.500 [1] conforming directo- describe the experimental setup used throughout the paper. In ries. Because LDAP can run on top of TCP/IP instead of the Section VI, the design and performance analysis of the entry OSI protocol stack, LDAP was first used to alleviate the client cache for back-bdb will be described. After presenting the pro- side protocol overhead. LDAP clients connect to an LDAP filing mechanism used to identify performance bottlenecks in gateway that forwards requests and responses to and from an Section VII, the following three sections present the design and X.500 directory. Further in the direction of being lightweight, performance analysis of the four caches : the candidate cache it has become commonplace to use standalone LDAP direc- in Section VIII, the IDL cache and the IDL stack slab cache tory servers that store directory data directly in them without in Section IX, and the BER cache in Section X. Section XI the need of separate X.500 directories. concludes the paper. Because directory searches constitute the majority of direc- tory operations, it is particularly important to provide a high II. LDAP AND OPENLDAP performance directory search in terms of latency and through- put. Low latency is essential to low delay IT services that rely LDAP (Lightweight Directory Access Protocol) [4] is a on the directory. High throughput is essential because one di- standard directory access protocol of the Internet to access di- rectory server should be able to process a large number of re- rectories having the X.500 [1] naming and data models [5]. quests from multiple directory clients simultaneously. It is also LDAP defines an access protocol over TCP/IP that is a well important to provide a highly reliable directory service since defined subset of the X.500 DAP (Directory Access Protocol) critical data are usually stored in enterprise directories. to enable lightweight implementations. Its protocol syntax is ISBN: 1-56555-269-5 737 SPECTS ©03 defined in ASN.1 (Abstract Syntax Notation One) [6]. LDAP rectory server (slapd), a replication daemon (slurpd), client provides ten directory operations : search, compare, add, APIs (C, C++, Perl ...), and various client and server side tools. delete, modify, modifyDN, bind, unbind, abandon, and ex- Slapd is a multi-threaded directory server that can be easily tended. The exchanged protocol messages between the server configured to support various types of backends. and the client are encoded by using BER of ASN.1. LDAP The OpenLDAP directory software suite is currently being uses a restricted form of BER to reduce the complexity of the deployed as the default directory software in most Linux distri- BER encoding/decoding process [4]. The attribute values are butions including RedHat and SuSE. OpenLDAP is also being in a string format (LDAPString) in LDAP. widely used in many enterprise IT environments. LDAP uses a subset of the X.500 naming and data models and organizes directory entries hierarchically in a DIT (Direc- III. LDAP SEARCH OPERATION tory Information Tree). A DIT can be composed of multiple subtrees residing on different LDAP servers. A referral mech- This section will introduce the search operation and the ar- anism is provided to allow clients to chase a link across servers chitecture of OpenLDAP slapd directory server. when they encounter such boundaries. On the other hand, an LDAP server may host a forest consisting of multiple DITs. A. LDAP Search Request An entry is identified by a unique name called RDN (Rel- The following is the definition of the search request [4]: ative Distinguished Name) among its siblings under the com- mon superior entry and by a unique name called DN (Distin- SearchRequest ::= SEQUENCE { baseObject LDAPDN, guished Name) within the entire DIT. One or more attribute scope ENUMERATED { values of an entry form RDN of the entry. DN can be formed baseObject (0), by concatenating RDN and DN of its immediate superior. singleLevel (1), wholeSubtree (2) }, An entry consists of a set of attributes permitted by the en- derefAliases ENUMERATED { try’s object class defined in the directory schema. and system neverDerefAliases (0), derefInSearching (1), and user schema definitions. An attribute consists of an at- derefFindingBaseObj (2), tribute type followed by one or more attribute values. The at- derefAlways (3) }, tribute type designates the name and OID (object identifier) of sizeLimit INTEGER (0 .. maxInt), timeLimit INTEGER (0 .. maxInt), the attribute and the syntax, the matching rules, and the cardi- typesOnly BOOLEAN, nality of the attribute values [7]. The object class designates filter Filter, entry’s name, OID, description, and its superclass as well as attributes AttributeDescriptionList } the required and allowed attributes. The object class hierarchy baseObject defines the DN of the base object entry, the represents the class hierarchy of entries, whereas DIT repre- reference point relative to which the search is performed. If sents the hierarchy of directory entry objects. An object class scope is baseObject, only the baseObject is searched; if inherits attributes from its superclass. singleLevel, all direct subordinate entries of the base object OpenLDAP [2] is an open-source directory software suite are searched; if wholeSubtree, the entire subtree rooted at conforming to the LDAPv3 protocol [4] and supporting vari- the base object is searched. derefAliases indicates whether ous platforms including Linux, FreeBSD, Apple Mac OS/X, to dereference an aliases encountered during the search. Pos- Sun Solaris, Microsoft Windows. Currently, it sports a rich sible options are 1) always-dereference, 2) never-dereference, set of features [8] : LDAPv3 over both IPv4 and IPv6, SASL 3) dereference only upon positioning the base object, and 4) (Simple Authentication and Security Layer) support, TLS dereference except for the base object positioning. size- (Transport Layer
Load more

Recommended publications
  • Open Directory & Openldap
    Open Directory & OpenLDAP David M. O’Rourke Engineering Manager Overview • Background on Apple’s Open Directory Technology (8 minutes) – What is it – What is Directory Services • How has Apple integrated OpenLDAP (20 minutes or less) – what has Apple added to OpenLDAP? • Questions and Answers (remaining time) Open Directory • Open Directory is a technology name – Covers both client access technologies and server technologies – Integrates and promotes industry standard technologies • Open Directory is built into Mac OS X & Mac OS X Server – Been there since 10.0 • Open Sourced as part of Darwin – http://developer.apple.com/darwin/projects/ opendirectory/ What Is Directory Services • Abstraction API for read/write access to system configuration and management data – Users, groups, mount records and others – Authentication abstraction Mac OS X Software Directory Services NetInfo LDAP BSD Files Other… Directory Services in 10.3 • Includes – LDAPv3 (read/write), Native Active Directory, NetInfo, NIS, BSD/etc files – Service Discovery: Rendezvous, SMB, AppleTalk, and SLP – LDAPv3 client support replication fail over • Documented Access API and plug-in API – SDK posted – Sample code, sample plug-in, notes – Directory Services Headers are installed in – /System/Library/Frameworks/DirectoryService.framework – Command line tool for directory editing ‘dscl’ 10.3 Usage of Directory Services • Login Window uses Directory Services for all user authentication – Managed Desktop • All Security Framework authentication uses Directory Services • Legacy Unix
    [Show full text]
  • The Design and Implementation of Multiprocessor Support for an Industrial Operating System Kernel
    Blekinge Institute of Technology Research Report 2005:06 The Design and Implementation of Multiprocessor Support for an Industrial Operating System Kernel Simon Kågström Håkan Grahn Lars Lundberg Department of Systems and Software School of Engineering Blekinge Institute of Technology The Design and Implementation of Multiprocessor Support for an Industrial Operating System Kernel Simon Kågström, Håkan Grahn, and Lars Lundberg Department of Systems and Software Engineering School of Engineering Blekinge Institute of Technology P.O. Box 520, SE-372 25 Ronneby, Sweden {ska, hgr, llu}@bth.se Abstract The ongoing transition from uniprocessor to multiprocessor computers requires support from the op- erating system kernel. Although many general-purpose multiprocessor operating systems exist, there is a large number of specialized operating systems which require porting in order to work on multipro- cessors. In this paper we describe the multiprocessor port of a cluster operating system kernel from a producer of industrial systems. Our initial implementation uses a giant locking scheme that serializes kernel execution. We also employed a method in which CPU-local variables are placed in a special sec- tion mapped to per-CPU physical memory pages. The giant lock and CPU-local section allowed us to implement an initial working version with only minor changes to the original code, although the giant lock and kernel-bound applications limit the performance of our multiprocessor port. Finally, we also discuss experiences from the implementation. 1 Introduction A current trend in the computer industry is the transition from uniprocessors to various kinds of multipro- cessors, also for desktop and embeddedsystems. Apart from traditional SMP systems, many manufacturers are now presenting chip multiprocessors or simultaneous multithreaded CPUs [9, 15, 16] which allow more efficient use of chip area.
    [Show full text]
  • The Lightning Memory-Mapped Database
    The Lightning Memory-Mapped Database Howard Chu CTO, Symas Corp. [email protected] Chief Architect, OpenLDAP [email protected] 2015-09-21 OpenLDAP Project ● Open source code project ● Founded 1998 ● Three core team members ● A dozen or so contributors ● Feature releases every 12-18 months ● Maintenance releases as needed 2 A Word About Symas ● Founded 1999 ● Founders from Enterprise Software world – platinum Technology (Locus Computing) – IBM ● Howard joined OpenLDAP in 1999 – One of the Core Team members – Appointed Chief Architect January 2007 ● No debt, no VC investments: self-funded 3 Intro ● Howard Chu – Founder and CTO Symas Corp. – Developing Free/Open Source software since 1980s ● GNU compiler toolchain, e.g. "gmake -j", etc. ● Many other projects... – Worked for NASA/JPL, wrote software for Space Shuttle, etc. 4 Topics (1) Background (2) Features (3) Design Approach (4) Internals (5) Special Features (6) Results 5 (1) Background ● API inspired by Berkeley DB (BDB) – OpenLDAP has used BDB extensively since 1999 – Deep experience with pros and cons of BDB design and implementation – Omits BDB features that were found to be of no benefit ● e.g. extensible hashing – Avoids BDB characteristics that were problematic ● e.g. cache tuning, complex locking, transaction logs, recovery 6 (2) Features LMDB At A Glance ● Key/Value store using B+trees ● Fully transactional, ACID compliant ● MVCC, readers never block ● Uses memory-mapped files, needs no tuning ● Crash-proof, no recovery needed after restart ● Highly optimized, extremely compact – under 40KB object code, fits in CPU L1 I$ ● Runs on most modern OSs – Linux, Android, *BSD, MacOSX, iOS, Solaris, Windows, etc..
    [Show full text]
  • Overview and Frequently Asked Questions
    Overview and Frequently Asked Questions OVERVIEW Oracle Buys Open Source Software Company Sleepycat - Expands Embedded Database Product Line with Berkeley DB Oracle has expanded its embedded Oracle’s portfolio of embedded database technologies provide database offerings through the customers with a range of technical options to meet their diverse needs, with the backing of a global support organization. Oracle acquisition of Sleepycat Software, Inc., and Sleepycat’s complementary products, expertise and resources a privately held supplier of open source will help accelerate innovation and provide customers with richer, database software for developers of more complete solutions. Oracle currently intends to continue embedded applications. Sleepycat’s dual license model and plans to continue to invest in the Berkeley DB products and support offerings, and to utilize Oracle’s global channel reach to service even more customers. With the proliferation of new applications, devices, appliances, Web Services and highly available Software as a Service (SaaS), In conjunction with the expansion of available embedded there is significant and growing demand for embedded databases. database options, Oracle has formed a global initiative focused In addition to Oracle Database Lite Edition for mobile devices on marketing, selling and servicing the Oracle Lite, TimesTen and Oracle TimesTen In-Memory Database for highly dynamic, and Berkeley DB products. This will include support procedures standards-compliant applications, Oracle now adds Berkeley DB and product release cycles tuned to the rapid-response needs database products for developers of tightly embedded, highly of embedded systems developers, and packaging and pricing customized applications. Berkeley DB is added to the Oracle flexibility that adjusts to the needs of Oracle’s partners.
    [Show full text]
  • Mac OS X Server Administrator's Guide
    034-9285.S4AdminPDF 6/27/02 2:07 PM Page 1 Mac OS X Server Administrator’s Guide K Apple Computer, Inc. © 2002 Apple Computer, Inc. All rights reserved. Under the copyright laws, this publication may not be copied, in whole or in part, without the written consent of Apple. The Apple logo is a trademark of Apple Computer, Inc., registered in the U.S. and other countries. Use of the “keyboard” Apple logo (Option-Shift-K) for commercial purposes without the prior written consent of Apple may constitute trademark infringement and unfair competition in violation of federal and state laws. Apple, the Apple logo, AppleScript, AppleShare, AppleTalk, ColorSync, FireWire, Keychain, Mac, Macintosh, Power Macintosh, QuickTime, Sherlock, and WebObjects are trademarks of Apple Computer, Inc., registered in the U.S. and other countries. AirPort, Extensions Manager, Finder, iMac, and Power Mac are trademarks of Apple Computer, Inc. Adobe and PostScript are trademarks of Adobe Systems Incorporated. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. and other countries. Netscape Navigator is a trademark of Netscape Communications Corporation. RealAudio is a trademark of Progressive Networks, Inc. © 1995–2001 The Apache Group. All rights reserved. UNIX is a registered trademark in the United States and other countries, licensed exclusively through X/Open Company, Ltd. 062-9285/7-26-02 LL9285.Book Page 3 Tuesday, June 25, 2002 3:59 PM Contents Preface How to Use This Guide 39 What’s Included
    [Show full text]
  • Filesystem Performance on Freebsd
    Filesystem Performance on FreeBSD Kris Kennaway [email protected] BSDCan 2006, Ottawa, May 12 Introduction ● Filesystem performance has many aspects ● No single metric for quantifying it ● I will focus on aspects that are relevant for my workloads (concurrent package building) ● The main relevant filesystem workloads seem to be – Concurrent tarball extraction – Recursive filesystem traversals ● Aim: determine relative performance of FreeBSD 4.x, 5.x and 6.x on these workloads – Overall performance and SMP scaling – Evaluate results of multi-year kernel locking strategy as it relates to these workloads Outline ● SMP architectural differences between 4/5/6.x ● Test methodology ● Hardware used ● Parallel tarball extraction test – Disk array and memory disk ● Scaling beyond 4 CPUs ● Recursive filesystem traversal test ● Conclusions & future work SMP Architectural Overview ● FreeBSD 4.x; rudimentary SMP support – Giant kernel lock restricts kernel access to one process at a time – SPL model; interrupts may still be processed in parallel ● FreeBSD 5.x; aim towards greater scalability – Giant-locked to begin with; then finer-grained locking pushdown ● FreeBSD 5.3; VM Giant-free ● FreeBSD 5.4; network stack Giant-free (mostly) ● Many other subsystems/drivers also locked – Interrupts as kernel threads; compete for common locks (if any) with everything else ● FreeBSD 6.x; – Consolidation; further pushdown; payoff! – VFS subsystem, UFS filesystem Giant-free FreeBSD versions ● FreeBSD 4.11-STABLE (11/2005) – Needed for amr driver fixes after 4.11-RELEASE ● FreeBSD 5.4-STABLE (11/05) – No patches needed ● FreeBSD 6.0-STABLE (11/05) – patches: ● Locking reworked in amr driver by Scott Long for better performance ● All relevant changes merged into FreeBSD 6.1 – A kernel panic was encountered at very high I/O loads ● Also fixed in 6.1 Test aims and Methodology ● Want to measure – overall performance difference between FreeBSD branches under varying (concurrent process I/O) loads – scaling to multiple CPUs ● Avoid saturating hardware resources (e.g.
    [Show full text]
  • USENIX October Proof 3
    FOR THE PAST COUPLE OF MONTHS, I have become absorbed in operating sys- RIK FARROW tems. My quest began with some consult- ing regarding security features of the cur- rent Linux kernel, including SELinux, then plunged even deeper during the HotOS workshop. I do not pretend to be a kernel hacker, but I am very interested in what musings goes on with the design and implementa- Rik Farrow provides UNIX and Internet security con- tion of operating system software. sulting and training. He is the author of UNIX System Security and System Administrator’s Guide to System V and editor of the SAGE Short Topics in System Like some others, I had wondered what had hap- Administration series. pened to FreeBSD 4’s stellar performance when Free- [email protected] BSD 5 appeared. Instead of getting faster, FreeBSD was slower. If I had bothered digging deeper, I would have learned that this was the result of far-reaching changes in the FreeBSD kernel. Michael W. Lucas ex- plains these changes in his article, which in turn is based on a talk given by Robert Watson about modify- ing the kernel to support SMP (symmetric multipro- cessing). And perhaps by the time you read this col- umn, FreeBSD 6 will have appeared, ready to utilize the new multiprocessor cores that are popping up. Lucas explains just why the transition from single- threaded to multi-threaded kernel takes so long and is so hard to do right. I first understood the importance of the Big Giant Lock when I was reviewing an early multiprocessing server that used SPARC processors.
    [Show full text]
  • Brave Gnu World
    LINUXCOVERCOMMUNITY USERSTORY SchlagwortSchlagwortBrave GNU sollte Worldsollte hier hier stehen stehen Schlagwort sollte hier stehen COVER STORY The Monthly GNU Column BRAVE GNU WORLD This column looks into projects and current affairs in the world of free software from the perspective of the GNU Project and the FSF. In this issue, I’ll focus on Comspari and the EU decision on software patents. BY GEORG C.F. GREVE n the past, the German government the Kroupware project in 2002. The pub- principles back to front. For example, has often caused a (positive) stir due lic tender by the BSI was aimed to pro- many other solutions are based on the Ito its activities with free software. duce a groupware solution that would principle that the server is the king of The Federal Ministry of the Economy support strong cryptography and inte- the hill and the users have to bow down (BMWi) sponsored Gnupg [1], for exam- grate seamlessly into a heterogeneous and worship it. The Kolab server does ple. Add to this a number of accompany- environment. things differently. The program makes ing activities, such as a report by the The government office gave a consor- the user king, and gives the user power Bavarian Accounts Office, the KBST let- tium comprising Erfrakon, Klarälvdalens to do things. The software simply helps ter, and the highly acclaimed migration Datakonsult [2], and Intevation [3] the coordinate things as a central intermedi- guide by the Federal Ministry of the Inte- task of developing the software. Inter- ary. rior (BMI). nally, the design and the software was Kolab 1 combined so-called partial referred to as Kolab 1.
    [Show full text]
  • How Freebsd Boots: a Soft-Core MIPS Perspective Brooks Davis, Robert Norton, Jonathan Woodruff, Robert N
    How FreeBSD Boots: a soft-core MIPS perspective Brooks Davis, Robert Norton, Jonathan Woodruff, Robert N. M. Watson Abstract We have implemented an FPGA soft-core, multithreaded, 64-bit MIPS R4000-style CPU called BERI to support research on the hardware/software interface. We have ported FreeBSD to this platform including support for mul- tithreaded and soon multicore CPUs. This paper describes the process by which a BERI system boots from CPU startup through the boot loaders, hand off to the kernel, and en- abling secondary CPU threads. Historically, the process of booting FreeBSD has been documented from a user perspec- tive or at a fairly high level. This paper aims to improve the documentation of the low level boot process for developers aiming to port FreeBSD to new targets. 1. Introduction Figure 1: BERIpad with application launcher From its modest origins as a fork of 386BSD targeting Intel i386 class CPUs, FreeBSD has been ported to a range of architectures including DEC Alpha1, AMD x86_64 (aka The rest of this paper narrates the boot process with a amd64), ARM, Intel IA64, MIPS, PC98, PowerPC, and special focus on the places customization was required for Sparc64. While the x86 and Alpha are fairly homogeneous BERI. We begin by describing the BERI platform (Section targets with mechanics for detecting and adapting to spe- 2), and then in detail documents the kernel architecture- cific board and peripheral configurations, embedded systems specific boot process for FreeBSD on BERI: boot loader platforms like ARM, MIPS, and PowerPC are much more (Section 3) and kernel boot process (Section 4).
    [Show full text]
  • Real-Time Audio Servers on BSD Unix Derivatives
    Juha Erkkilä Real-Time Audio Servers on BSD Unix Derivatives Master's Thesis in Information Technology June 17, 2005 University of Jyväskylä Department of Mathematical Information Technology Jyväskylä Author: Juha Erkkilä Contact information: [email protected].fi Title: Real-Time Audio Servers on BSD Unix Derivatives Työn nimi: Reaaliaikaiset äänipalvelinsovellukset BSD Unix -johdannaisjärjestelmissä Project: Master's Thesis in Information Technology Page count: 146 Abstract: This paper covers real-time and interprocess communication features of 4.4BSD Unix derived operating systems, and especially their applicability for real- time audio servers. The research ground of bringing real-time properties to tradi- tional Unix operating systems (such as 4.4BSD) is covered. Included are some design ideas used in BSD-variants, such as using multithreaded kernels, and schedulers that can provide real-time guarantees to processes. Factors affecting the design of real- time audio servers are considered, especially the suitability of various interprocess communication facilities as mechanisms to pass audio data between applications. To test these mechanisms on a real operating system, an audio server and a client utilizing these techniques is written and tested on an OpenBSD operating system. The performance of the audio server and OpenBSD is analyzed, with attempts to identify some bottlenecks of real-time operation in the OpenBSD system. Suomenkielinen tiivistelmä: Tämä tutkielma kattaa reaaliaikaisuus- ja prosessien väliset kommunikaatio-ominaisuudet, keskittyen 4.4BSD Unix -johdannaisiin käyt- töjärjestelmiin, ja erityisesti siihen kuinka hyvin nämä soveltuvat reaaliaikaisille äänipalvelinsovelluksille. Tutkimusalueeseen sisältyy reaaliaikaisuusominaisuuk- sien tuominen perinteisiin Unix-käyttöjärjestelmiin (kuten 4.4BSD:hen). Mukana on suunnitteluideoita, joita on käytetty joissakin BSD-varianteissa, kuten säikeis- tetyt kernelit, ja skedulerit, jotka voivat tarjota reaaliaikaisuustakeita prosesseille.
    [Show full text]
  • Evaluating Locking Schemes in the Freebsd Kernel
    Evaluating Locking Schemes in the FreeBSD Kernel james francis toy iv Advisor: David Hemmendinger 20100321 Abstract This project is concerned with locking strategies in the FreeBSD unix kernel. Locks prevent race conditions in a multi-programming environ- ment. At present the FreeBSD kernel employs a GIANT_LOCK to pre- vent race conditions by locking access to the entire kernel. This method may inhibit concurrency; however, it is a simple and easy way to ensure correctness. The goal of this project is to see if fine-grained locking (FGL), an alternative to the GIANT_LOCK, promotes concurrency and through- put by locking only critical shared resources. FGL was implemented for two kernel subsystems, and we measured the time that a thread was wait- ing to acquire a lock; FGL showed zero wait time and the GIANT_LOCK showed non-zero wait time. 1 An Introduction to Locking Locking and synchronization were developed to prevent processes from interfer- ing with one another when having access to shared resources. This is important because in a multi-threaded environment these situations often arise since it is more efficient to execute programs in parallel. This contention for shared data is more commonly described as a “race condition” where multiple pro- cesses are racing to a shared resource. These race conditions can be resolved by mutual exclusion (ensures only one process is using a shared resource at a time) which is behavior provided by the locking mechanisms currently employed by the FreeBSD kernel. The purpose of this project is to analyze the current locking scheme in the FreeBSD kernel; attempt to implement a more efficient 1 locking method, and compare the initial method versus the proposed method in this project.
    [Show full text]
  • Matthew Dillon Dragonfly BSD Project 23 October 2003 Dragonfly Overview
    Matthew Dillon DragonFly BSD Project 23 October 2003 DragonFly Overview ●FreeBSD 4.x and 5.x directions ●Differentiating DragonFly, the basis for a project fork ● A different, more maintainable user threading API (syscall messaging) ● A different, more maintainable approach to MP design ● CPU Isolation by design using IPI messaging rather then by accident w/Mutexes ● Light Weight Kernel Threading with fewer hacks ●Project Goals ● Maintaining stability, producing production-capable releases ● A more consistent and more easily maintained message-based framework ● UP, MP, SSI Scaleability ● Userland VFS Development ● Machine-verified Package Management ●This Presentation ● Threading And Messaging ● Our approach to the Big Giant Lock problem. Why not mutexes? ● Our approach to achieving a Single System Image (SSI) DragonFly Threading and Messaging Model Matthew Dillon DragonFly BSD Project 23 October 2003 Light Weight Kernel Threading and User Processes LWKT SUBSYSTEM USERLAND PROCESS SCHEDULER CPU #1 PROCESS RUNQ SCHEDULER THREAD LWKT SCHEDULER THREAD PROCESS1 (FIXED PRIORITY) PROCESS2 THREAD PROCESS2 CURPROC ON CPU1 PROCESS3 CPU #2 THREAD PROCESS4 LWKT SCHEDULER PROCESS5 THREAD PROCESS5 (FIXED PRIORITY) CURPROC ON CPU2 THREAD PROCESS6 IPI Messaging ●ABSTRACTION PROMOTES CPU ISOLATION ●ASYNCHRONOUS IPI MESSAGING AVOIDS MUTEX OPS ●SIMPLE CRITICAL SECTIONS FOR LOCAL ACCESS ●MANY IPI OPS CAN BE PASSIVE / CONTRAST W/ RCU CPU #1 CPU #2 SCHEDULER SCHEDULER ASYNC THREAD1 SCHEDULE IPI SCHEDULE THREAD2 THREAD2 MESSAGE THREAD2 ALLOCATOR ALLOCATE MEMORY
    [Show full text]