Enhancing the Performance of Openldap Directory Server with Multiple Caching
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Open Directory & Openldap
Open Directory & OpenLDAP David M. O’Rourke Engineering Manager Overview • Background on Apple’s Open Directory Technology (8 minutes) – What is it – What is Directory Services • How has Apple integrated OpenLDAP (20 minutes or less) – what has Apple added to OpenLDAP? • Questions and Answers (remaining time) Open Directory • Open Directory is a technology name – Covers both client access technologies and server technologies – Integrates and promotes industry standard technologies • Open Directory is built into Mac OS X & Mac OS X Server – Been there since 10.0 • Open Sourced as part of Darwin – http://developer.apple.com/darwin/projects/ opendirectory/ What Is Directory Services • Abstraction API for read/write access to system configuration and management data – Users, groups, mount records and others – Authentication abstraction Mac OS X Software Directory Services NetInfo LDAP BSD Files Other… Directory Services in 10.3 • Includes – LDAPv3 (read/write), Native Active Directory, NetInfo, NIS, BSD/etc files – Service Discovery: Rendezvous, SMB, AppleTalk, and SLP – LDAPv3 client support replication fail over • Documented Access API and plug-in API – SDK posted – Sample code, sample plug-in, notes – Directory Services Headers are installed in – /System/Library/Frameworks/DirectoryService.framework – Command line tool for directory editing ‘dscl’ 10.3 Usage of Directory Services • Login Window uses Directory Services for all user authentication – Managed Desktop • All Security Framework authentication uses Directory Services • Legacy Unix -
The Design and Implementation of Multiprocessor Support for an Industrial Operating System Kernel
Blekinge Institute of Technology Research Report 2005:06 The Design and Implementation of Multiprocessor Support for an Industrial Operating System Kernel Simon Kågström Håkan Grahn Lars Lundberg Department of Systems and Software School of Engineering Blekinge Institute of Technology The Design and Implementation of Multiprocessor Support for an Industrial Operating System Kernel Simon Kågström, Håkan Grahn, and Lars Lundberg Department of Systems and Software Engineering School of Engineering Blekinge Institute of Technology P.O. Box 520, SE-372 25 Ronneby, Sweden {ska, hgr, llu}@bth.se Abstract The ongoing transition from uniprocessor to multiprocessor computers requires support from the op- erating system kernel. Although many general-purpose multiprocessor operating systems exist, there is a large number of specialized operating systems which require porting in order to work on multipro- cessors. In this paper we describe the multiprocessor port of a cluster operating system kernel from a producer of industrial systems. Our initial implementation uses a giant locking scheme that serializes kernel execution. We also employed a method in which CPU-local variables are placed in a special sec- tion mapped to per-CPU physical memory pages. The giant lock and CPU-local section allowed us to implement an initial working version with only minor changes to the original code, although the giant lock and kernel-bound applications limit the performance of our multiprocessor port. Finally, we also discuss experiences from the implementation. 1 Introduction A current trend in the computer industry is the transition from uniprocessors to various kinds of multipro- cessors, also for desktop and embeddedsystems. Apart from traditional SMP systems, many manufacturers are now presenting chip multiprocessors or simultaneous multithreaded CPUs [9, 15, 16] which allow more efficient use of chip area. -
The Lightning Memory-Mapped Database
The Lightning Memory-Mapped Database Howard Chu CTO, Symas Corp. [email protected] Chief Architect, OpenLDAP [email protected] 2015-09-21 OpenLDAP Project ● Open source code project ● Founded 1998 ● Three core team members ● A dozen or so contributors ● Feature releases every 12-18 months ● Maintenance releases as needed 2 A Word About Symas ● Founded 1999 ● Founders from Enterprise Software world – platinum Technology (Locus Computing) – IBM ● Howard joined OpenLDAP in 1999 – One of the Core Team members – Appointed Chief Architect January 2007 ● No debt, no VC investments: self-funded 3 Intro ● Howard Chu – Founder and CTO Symas Corp. – Developing Free/Open Source software since 1980s ● GNU compiler toolchain, e.g. "gmake -j", etc. ● Many other projects... – Worked for NASA/JPL, wrote software for Space Shuttle, etc. 4 Topics (1) Background (2) Features (3) Design Approach (4) Internals (5) Special Features (6) Results 5 (1) Background ● API inspired by Berkeley DB (BDB) – OpenLDAP has used BDB extensively since 1999 – Deep experience with pros and cons of BDB design and implementation – Omits BDB features that were found to be of no benefit ● e.g. extensible hashing – Avoids BDB characteristics that were problematic ● e.g. cache tuning, complex locking, transaction logs, recovery 6 (2) Features LMDB At A Glance ● Key/Value store using B+trees ● Fully transactional, ACID compliant ● MVCC, readers never block ● Uses memory-mapped files, needs no tuning ● Crash-proof, no recovery needed after restart ● Highly optimized, extremely compact – under 40KB object code, fits in CPU L1 I$ ● Runs on most modern OSs – Linux, Android, *BSD, MacOSX, iOS, Solaris, Windows, etc.. -
Overview and Frequently Asked Questions
Overview and Frequently Asked Questions OVERVIEW Oracle Buys Open Source Software Company Sleepycat - Expands Embedded Database Product Line with Berkeley DB Oracle has expanded its embedded Oracle’s portfolio of embedded database technologies provide database offerings through the customers with a range of technical options to meet their diverse needs, with the backing of a global support organization. Oracle acquisition of Sleepycat Software, Inc., and Sleepycat’s complementary products, expertise and resources a privately held supplier of open source will help accelerate innovation and provide customers with richer, database software for developers of more complete solutions. Oracle currently intends to continue embedded applications. Sleepycat’s dual license model and plans to continue to invest in the Berkeley DB products and support offerings, and to utilize Oracle’s global channel reach to service even more customers. With the proliferation of new applications, devices, appliances, Web Services and highly available Software as a Service (SaaS), In conjunction with the expansion of available embedded there is significant and growing demand for embedded databases. database options, Oracle has formed a global initiative focused In addition to Oracle Database Lite Edition for mobile devices on marketing, selling and servicing the Oracle Lite, TimesTen and Oracle TimesTen In-Memory Database for highly dynamic, and Berkeley DB products. This will include support procedures standards-compliant applications, Oracle now adds Berkeley DB and product release cycles tuned to the rapid-response needs database products for developers of tightly embedded, highly of embedded systems developers, and packaging and pricing customized applications. Berkeley DB is added to the Oracle flexibility that adjusts to the needs of Oracle’s partners. -
Mac OS X Server Administrator's Guide
034-9285.S4AdminPDF 6/27/02 2:07 PM Page 1 Mac OS X Server Administrator’s Guide K Apple Computer, Inc. © 2002 Apple Computer, Inc. All rights reserved. Under the copyright laws, this publication may not be copied, in whole or in part, without the written consent of Apple. The Apple logo is a trademark of Apple Computer, Inc., registered in the U.S. and other countries. Use of the “keyboard” Apple logo (Option-Shift-K) for commercial purposes without the prior written consent of Apple may constitute trademark infringement and unfair competition in violation of federal and state laws. Apple, the Apple logo, AppleScript, AppleShare, AppleTalk, ColorSync, FireWire, Keychain, Mac, Macintosh, Power Macintosh, QuickTime, Sherlock, and WebObjects are trademarks of Apple Computer, Inc., registered in the U.S. and other countries. AirPort, Extensions Manager, Finder, iMac, and Power Mac are trademarks of Apple Computer, Inc. Adobe and PostScript are trademarks of Adobe Systems Incorporated. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. and other countries. Netscape Navigator is a trademark of Netscape Communications Corporation. RealAudio is a trademark of Progressive Networks, Inc. © 1995–2001 The Apache Group. All rights reserved. UNIX is a registered trademark in the United States and other countries, licensed exclusively through X/Open Company, Ltd. 062-9285/7-26-02 LL9285.Book Page 3 Tuesday, June 25, 2002 3:59 PM Contents Preface How to Use This Guide 39 What’s Included -
Filesystem Performance on Freebsd
Filesystem Performance on FreeBSD Kris Kennaway [email protected] BSDCan 2006, Ottawa, May 12 Introduction ● Filesystem performance has many aspects ● No single metric for quantifying it ● I will focus on aspects that are relevant for my workloads (concurrent package building) ● The main relevant filesystem workloads seem to be – Concurrent tarball extraction – Recursive filesystem traversals ● Aim: determine relative performance of FreeBSD 4.x, 5.x and 6.x on these workloads – Overall performance and SMP scaling – Evaluate results of multi-year kernel locking strategy as it relates to these workloads Outline ● SMP architectural differences between 4/5/6.x ● Test methodology ● Hardware used ● Parallel tarball extraction test – Disk array and memory disk ● Scaling beyond 4 CPUs ● Recursive filesystem traversal test ● Conclusions & future work SMP Architectural Overview ● FreeBSD 4.x; rudimentary SMP support – Giant kernel lock restricts kernel access to one process at a time – SPL model; interrupts may still be processed in parallel ● FreeBSD 5.x; aim towards greater scalability – Giant-locked to begin with; then finer-grained locking pushdown ● FreeBSD 5.3; VM Giant-free ● FreeBSD 5.4; network stack Giant-free (mostly) ● Many other subsystems/drivers also locked – Interrupts as kernel threads; compete for common locks (if any) with everything else ● FreeBSD 6.x; – Consolidation; further pushdown; payoff! – VFS subsystem, UFS filesystem Giant-free FreeBSD versions ● FreeBSD 4.11-STABLE (11/2005) – Needed for amr driver fixes after 4.11-RELEASE ● FreeBSD 5.4-STABLE (11/05) – No patches needed ● FreeBSD 6.0-STABLE (11/05) – patches: ● Locking reworked in amr driver by Scott Long for better performance ● All relevant changes merged into FreeBSD 6.1 – A kernel panic was encountered at very high I/O loads ● Also fixed in 6.1 Test aims and Methodology ● Want to measure – overall performance difference between FreeBSD branches under varying (concurrent process I/O) loads – scaling to multiple CPUs ● Avoid saturating hardware resources (e.g. -
USENIX October Proof 3
FOR THE PAST COUPLE OF MONTHS, I have become absorbed in operating sys- RIK FARROW tems. My quest began with some consult- ing regarding security features of the cur- rent Linux kernel, including SELinux, then plunged even deeper during the HotOS workshop. I do not pretend to be a kernel hacker, but I am very interested in what musings goes on with the design and implementa- Rik Farrow provides UNIX and Internet security con- tion of operating system software. sulting and training. He is the author of UNIX System Security and System Administrator’s Guide to System V and editor of the SAGE Short Topics in System Like some others, I had wondered what had hap- Administration series. pened to FreeBSD 4’s stellar performance when Free- [email protected] BSD 5 appeared. Instead of getting faster, FreeBSD was slower. If I had bothered digging deeper, I would have learned that this was the result of far-reaching changes in the FreeBSD kernel. Michael W. Lucas ex- plains these changes in his article, which in turn is based on a talk given by Robert Watson about modify- ing the kernel to support SMP (symmetric multipro- cessing). And perhaps by the time you read this col- umn, FreeBSD 6 will have appeared, ready to utilize the new multiprocessor cores that are popping up. Lucas explains just why the transition from single- threaded to multi-threaded kernel takes so long and is so hard to do right. I first understood the importance of the Big Giant Lock when I was reviewing an early multiprocessing server that used SPARC processors. -
Brave Gnu World
LINUXCOVERCOMMUNITY USERSTORY SchlagwortSchlagwortBrave GNU sollte Worldsollte hier hier stehen stehen Schlagwort sollte hier stehen COVER STORY The Monthly GNU Column BRAVE GNU WORLD This column looks into projects and current affairs in the world of free software from the perspective of the GNU Project and the FSF. In this issue, I’ll focus on Comspari and the EU decision on software patents. BY GEORG C.F. GREVE n the past, the German government the Kroupware project in 2002. The pub- principles back to front. For example, has often caused a (positive) stir due lic tender by the BSI was aimed to pro- many other solutions are based on the Ito its activities with free software. duce a groupware solution that would principle that the server is the king of The Federal Ministry of the Economy support strong cryptography and inte- the hill and the users have to bow down (BMWi) sponsored Gnupg [1], for exam- grate seamlessly into a heterogeneous and worship it. The Kolab server does ple. Add to this a number of accompany- environment. things differently. The program makes ing activities, such as a report by the The government office gave a consor- the user king, and gives the user power Bavarian Accounts Office, the KBST let- tium comprising Erfrakon, Klarälvdalens to do things. The software simply helps ter, and the highly acclaimed migration Datakonsult [2], and Intevation [3] the coordinate things as a central intermedi- guide by the Federal Ministry of the Inte- task of developing the software. Inter- ary. rior (BMI). nally, the design and the software was Kolab 1 combined so-called partial referred to as Kolab 1. -
How Freebsd Boots: a Soft-Core MIPS Perspective Brooks Davis, Robert Norton, Jonathan Woodruff, Robert N
How FreeBSD Boots: a soft-core MIPS perspective Brooks Davis, Robert Norton, Jonathan Woodruff, Robert N. M. Watson Abstract We have implemented an FPGA soft-core, multithreaded, 64-bit MIPS R4000-style CPU called BERI to support research on the hardware/software interface. We have ported FreeBSD to this platform including support for mul- tithreaded and soon multicore CPUs. This paper describes the process by which a BERI system boots from CPU startup through the boot loaders, hand off to the kernel, and en- abling secondary CPU threads. Historically, the process of booting FreeBSD has been documented from a user perspec- tive or at a fairly high level. This paper aims to improve the documentation of the low level boot process for developers aiming to port FreeBSD to new targets. 1. Introduction Figure 1: BERIpad with application launcher From its modest origins as a fork of 386BSD targeting Intel i386 class CPUs, FreeBSD has been ported to a range of architectures including DEC Alpha1, AMD x86_64 (aka The rest of this paper narrates the boot process with a amd64), ARM, Intel IA64, MIPS, PC98, PowerPC, and special focus on the places customization was required for Sparc64. While the x86 and Alpha are fairly homogeneous BERI. We begin by describing the BERI platform (Section targets with mechanics for detecting and adapting to spe- 2), and then in detail documents the kernel architecture- cific board and peripheral configurations, embedded systems specific boot process for FreeBSD on BERI: boot loader platforms like ARM, MIPS, and PowerPC are much more (Section 3) and kernel boot process (Section 4). -
Real-Time Audio Servers on BSD Unix Derivatives
Juha Erkkilä Real-Time Audio Servers on BSD Unix Derivatives Master's Thesis in Information Technology June 17, 2005 University of Jyväskylä Department of Mathematical Information Technology Jyväskylä Author: Juha Erkkilä Contact information: [email protected].fi Title: Real-Time Audio Servers on BSD Unix Derivatives Työn nimi: Reaaliaikaiset äänipalvelinsovellukset BSD Unix -johdannaisjärjestelmissä Project: Master's Thesis in Information Technology Page count: 146 Abstract: This paper covers real-time and interprocess communication features of 4.4BSD Unix derived operating systems, and especially their applicability for real- time audio servers. The research ground of bringing real-time properties to tradi- tional Unix operating systems (such as 4.4BSD) is covered. Included are some design ideas used in BSD-variants, such as using multithreaded kernels, and schedulers that can provide real-time guarantees to processes. Factors affecting the design of real- time audio servers are considered, especially the suitability of various interprocess communication facilities as mechanisms to pass audio data between applications. To test these mechanisms on a real operating system, an audio server and a client utilizing these techniques is written and tested on an OpenBSD operating system. The performance of the audio server and OpenBSD is analyzed, with attempts to identify some bottlenecks of real-time operation in the OpenBSD system. Suomenkielinen tiivistelmä: Tämä tutkielma kattaa reaaliaikaisuus- ja prosessien väliset kommunikaatio-ominaisuudet, keskittyen 4.4BSD Unix -johdannaisiin käyt- töjärjestelmiin, ja erityisesti siihen kuinka hyvin nämä soveltuvat reaaliaikaisille äänipalvelinsovelluksille. Tutkimusalueeseen sisältyy reaaliaikaisuusominaisuuk- sien tuominen perinteisiin Unix-käyttöjärjestelmiin (kuten 4.4BSD:hen). Mukana on suunnitteluideoita, joita on käytetty joissakin BSD-varianteissa, kuten säikeis- tetyt kernelit, ja skedulerit, jotka voivat tarjota reaaliaikaisuustakeita prosesseille. -
Evaluating Locking Schemes in the Freebsd Kernel
Evaluating Locking Schemes in the FreeBSD Kernel james francis toy iv Advisor: David Hemmendinger 20100321 Abstract This project is concerned with locking strategies in the FreeBSD unix kernel. Locks prevent race conditions in a multi-programming environ- ment. At present the FreeBSD kernel employs a GIANT_LOCK to pre- vent race conditions by locking access to the entire kernel. This method may inhibit concurrency; however, it is a simple and easy way to ensure correctness. The goal of this project is to see if fine-grained locking (FGL), an alternative to the GIANT_LOCK, promotes concurrency and through- put by locking only critical shared resources. FGL was implemented for two kernel subsystems, and we measured the time that a thread was wait- ing to acquire a lock; FGL showed zero wait time and the GIANT_LOCK showed non-zero wait time. 1 An Introduction to Locking Locking and synchronization were developed to prevent processes from interfer- ing with one another when having access to shared resources. This is important because in a multi-threaded environment these situations often arise since it is more efficient to execute programs in parallel. This contention for shared data is more commonly described as a “race condition” where multiple pro- cesses are racing to a shared resource. These race conditions can be resolved by mutual exclusion (ensures only one process is using a shared resource at a time) which is behavior provided by the locking mechanisms currently employed by the FreeBSD kernel. The purpose of this project is to analyze the current locking scheme in the FreeBSD kernel; attempt to implement a more efficient 1 locking method, and compare the initial method versus the proposed method in this project. -
Matthew Dillon Dragonfly BSD Project 23 October 2003 Dragonfly Overview
Matthew Dillon DragonFly BSD Project 23 October 2003 DragonFly Overview ●FreeBSD 4.x and 5.x directions ●Differentiating DragonFly, the basis for a project fork ● A different, more maintainable user threading API (syscall messaging) ● A different, more maintainable approach to MP design ● CPU Isolation by design using IPI messaging rather then by accident w/Mutexes ● Light Weight Kernel Threading with fewer hacks ●Project Goals ● Maintaining stability, producing production-capable releases ● A more consistent and more easily maintained message-based framework ● UP, MP, SSI Scaleability ● Userland VFS Development ● Machine-verified Package Management ●This Presentation ● Threading And Messaging ● Our approach to the Big Giant Lock problem. Why not mutexes? ● Our approach to achieving a Single System Image (SSI) DragonFly Threading and Messaging Model Matthew Dillon DragonFly BSD Project 23 October 2003 Light Weight Kernel Threading and User Processes LWKT SUBSYSTEM USERLAND PROCESS SCHEDULER CPU #1 PROCESS RUNQ SCHEDULER THREAD LWKT SCHEDULER THREAD PROCESS1 (FIXED PRIORITY) PROCESS2 THREAD PROCESS2 CURPROC ON CPU1 PROCESS3 CPU #2 THREAD PROCESS4 LWKT SCHEDULER PROCESS5 THREAD PROCESS5 (FIXED PRIORITY) CURPROC ON CPU2 THREAD PROCESS6 IPI Messaging ●ABSTRACTION PROMOTES CPU ISOLATION ●ASYNCHRONOUS IPI MESSAGING AVOIDS MUTEX OPS ●SIMPLE CRITICAL SECTIONS FOR LOCAL ACCESS ●MANY IPI OPS CAN BE PASSIVE / CONTRAST W/ RCU CPU #1 CPU #2 SCHEDULER SCHEDULER ASYNC THREAD1 SCHEDULE IPI SCHEDULE THREAD2 THREAD2 MESSAGE THREAD2 ALLOCATOR ALLOCATE MEMORY