DOCSLIB.ORG

The Puppet Book

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
The Puppet Book Puppet Docs Welcome to the Puppet documentation site. An downloadable version of this guide may be found on our downloads page. Getting Started New users should begin here. • An Introduction to Puppet • Supported Platforms • Installing Puppet – from packages, source, or gems • Configuring Puppet – includes server setup & testing • Frequently Asked Questions Components Learn more about major working parts of the Puppet system. • puppetmasterd, puppetd, puppet, & ralsh – components of the system • Security Infrastructure / Securing Puppet – how PKI works Extended Knowledge Once you’ve learned the basics, go here next. • Puppet Language Tutorial – all the language details • Puppet Modules – modules make it easy to organize and share content • Puppet File Serving – Serving files with Puppet • Style Guide – Puppet community conventions • Best Practices – use Puppet effectively • Tips & Tricks • Troubleshooting Puppet Dashboard (Web GUI) Puppet from a graphical perspective. • Installing Dashboard • Using Dashboard Advanced Topics Power user features. • Templating – template out config files using ERB Puppet Docs 1 • Virtual Resources • Exported Resources – share data between hosts • Environments – separate dev, stage, & production • Reporting – learn what your nodes are up to • External Nodes – specify what your machines do using external data sources • Scaling Puppet – general tips & tricks • Scaling With Passenger – for Puppet 0.24.6 and later • Scaling With Mongrel – for older versions of Puppet Resource Types Documentation on what Puppet can manage out of the box. • Types By Category – documentation on resources managed out of the box • Alphabetical Listing Extending Puppet Adapt Puppet to your specific requirements through extensible code. • Writing Custom Facts • Writing Custom Types & Providers • Complete Resource Example – more information on custom types & providers • Provider Development – more about providers • Writing Custom Functions • Plugins In Modules – where to put plugins, how to sync to clients • REST API – reference of api accessible resources Development Information Learn how to work with Puppet core. • Running Puppet from Source – preview the leading edge • Development Life Cycle – learn how to contribute code Auto-generated Docs In addition to the formatted and annotated documentation above, auto-generated documentation is available for current and previous Puppet versions. Here are links to the latest versions: • Metaparameters – these are usable on all types • Types – all default types • Configuration – all configuration file settings • Functions – all built in functions • Report – available reports Advanced Topics 2 Other Resources • Puppet Wiki & Bug Tracker • Puppet Patterns (Recipes) Help Improve This Document This document belongs to the community and is licensed under the Creative Commons. You can help improve it! To contribute ideas, problems, or suggestions, simply use the Contribute link. If you would like to submit your own content, the process is easy. You can fork the project on github, make changes, and send us a pull request. See the README files in the project for more information. Other Resources 3 Introduction to Puppet Why Puppet As system administrators acquire more and more systems to manage, automation of mundane tasks is increasingly important. Rather than develop in-house scripts, it is desirable to share a system that everyone can use, and invest in tools that can be used regardless of one’s employer. Certainly doing things manually doesn’t scale. Puppet has been developed to help the sysadmin community move to building and sharing mature tools that avoid the duplication of everyone solving the same problem. It does so in two ways: • It provides a powerful framework to simplify the majority of the technical tasks that sysadmins need to perform • The sysadmin work is written as code in Puppet’s custom language which is shareable just like any other code. This means that your work as a sysadmin can get done much faster, because you can have Puppet handle most or all of the details, and you can download code from other sysadmins to help you get done even faster. The majority of Puppet implementations use at least one or two modules developed by someone else, and there are already hundreds of modules developed and shared by the community. Learning Recommendations We’re glad you want to learn Puppet. You’re free to browse around the documentation as you like, though we generally recommend trying out Puppet locally first (without the daemon and client/server setup), so you can understand the basic concepts. From there, move on to centrally managed server infrastructure. Ralsh is also a great way to get your feet wet exploring the Puppet model, after you have read some of the basic information – you can quickly see how the declarative model works for simple things like users, services, and file permissions. Once you’ve learned the basics, make sure you understand classes and modules, then move on to the advanced sections and read more about the features that are useful to you. Learning all at once is definitely not required. If you find something confusing, use the feedback tab to let us know. System Components Puppet is typically (but not always) used in a client/server formation, with all of your clients talking to one or more central servers. Each client contacts the server periodically (every half hour, by default), downloads the latest configuration, and makes sure it is in sync with that configuration. Once done, the client can send a report back to the server indicating if anything needed to change. This diagram shows the data flow in a regular Puppet implementation: Puppet’s functionality is built as a stack of separate layers, each responsible for a fixed aspect of the system, with tight controls on how information passes between layers: Introduction to Puppet 4 See also Configuring Puppet. For more information about components (puppetmasterd, puppetd, puppet, and so on), see the Tools section. Features of the System Idempotency One big difference between Puppet and most other tools is that Puppet configurations are idempotent, meaning they can safely be run multiple times. Once you develop your configuration, your machines will apply the configuration often – by default, every 30 minutes – and Puppet will only make any changes to the system if the system state does not match the configured state. If you tell the system to operate in no-op (“aka dry-run”), mode, using the --noop argument to one of the Puppet tools, puppet will guarantee that no work happens on your system. Similarly, if any changes do happen when running without that flag, puppet will ensure those changes are logged. Because of this, you can use Puppet to manage a machine throughout its lifecycle – from initial installation, to ongoing upgrades, and finally to end-of-life, where you move services elsewhere. Unlike system install tools like Sun’s Jumpstart or Red Hat’s Kickstart, Puppet configurations can keep machines up to date for years, rather than just building them correctly only the first time and then neccessitating a rebuild. Puppet users usually do just enough with their host install tools to boostrap Puppet, then they use Puppet to do everything else. Cross Platform Puppet’s Resource Abstraction Layer (RAL) allows you to focus on the parts of the system you care about, ignoring implementation details like command names, arguments, and file formats – your tools should treat all users the same, whether the user is stored in NetInfo or /etc/passwd. We call these system entities resources. Ralsh, listed in the Tools section is a fun way to try out the RAL before you get too deep into Puppet language. Model & Graph Based Resource Types The concept of each resource (like service, file, user, group, etc) is modelled as a “type”. Puppet decouples the definition from how that implementation is fulfilled on a particular operating system, for instance, a Linux user versus an OS X user can be talked about in the same way but are implemented differently inside of Puppet. See Type Guides for a list of managed types and information about how to use them. Providers Providers are the fulfillment of a resource. For instance, for the package type, both ‘yum’ and ‘apt’ are valid ways to manage packages. Sometimes more than one provider will be available on a particular platform, though each platform always has a default provider. There are currently 17 providers for the package type. System Components 5 Modifying the System Puppet resource providers are what are responsible for directly managing the bits on disk. You do not directly modify a system from Puppet language – you use the language to specify a resource, which then modifies the system. This way puppet language behaves exactly the same way in a centrally managed server setup as it does locally without a server. Rather than tacking a couple of lines onto the end of your fstab, you use the mount type to create a new resource that knows how to modify the fstab, or NetInfo, or wherever mount information is kept. Resources have attributes called ‘properties’ which change the way a resource is managed. For instance, users have an attribute that specicies whether the home directory should be created. ‘Metaparams’ are another special kind of attribute, those exist on all resources. This include things like the log level for the resource, whether the resource should be in noop mode so it never modifies the system, and the relationships between resources. Resource Relationships Puppet has a system of modelling relationships between resources – what resources should be evaluated before or after one another. They also are used to determine whether a resource needs to respond to changes in another resource (such as if a service needs to restart if the configuration file for the service has changed). This ordering reduces unneccessary commands, such as avoiding restarting a service if the configuration has not changed. Because the system is graph based, it’s actually possible to generate a diagram (from Puppet) of the relationships between all of your resources.
Load more

Recommended publications
  • DIRECTOR's REPORT September 21, 2017
    DIRECTOR’S REPORT September 21, 2017 SUMMER PROGRAMMING The 2017 Summer Reading Club (SRC), Read Up! Rise Up! by Design, utilized key aspects of the design thinking methodology in the development of the SRC program curriculum. Design thinking, as it relates to program development, seeks to identify creative solutions to problems by utilizing solution-based strategies. In an ideal setting these creative strategies ultimately result in a constructive resolution to an identified problem or challenge. The design thinking methodology is used in a variety of disciplines i.e. urban planning, web development, education etc. Programming content focused on S.T.R.E.A.M (Science, Technology, Reading, Writing, Engineering, Arts and Math) related subjects. Throughout the summer program participants participated in variety of enrichment activities that promoted creative thinking, problem solving, reading, writing and other forms of creative expression. Summer Reading Club registration began May 15th, 2017 with the contest and associated programming continuing for 9 weeks (June 5th – August 5th). 10,156 students registered for this year’s SRC with 5,286 participants completing. The 2017 completion rate continued its upward trend with 52% of all participants completing the program. The Cleveland Public Library received generous financial and in-kind support from the Friends of the Cleveland Public Library Foundation, The Cleveland Museum of Art, The City of Cleveland, Cleveland Fire Department, Cleveland Metropolitan School District, United Way of Greater Cleveland, Greater Cleveland Food Bank, KPMG, Mitchell’s Ice Cream, McDonalds, and Georgio’s Pizza. The Library was also the recipient of multiple book grants that enabled children to receive free books for participating in the program.
    [Show full text]
  • ROADS and BRIDGES: the UNSEEN LABOR BEHIND OUR DIGITAL INFRASTRUCTURE Preface
    Roads and Bridges:The Unseen Labor Behind Our Digital Infrastructure WRITTEN BY Nadia Eghbal 2 Open up your phone. Your social media, your news, your medical records, your bank: they are all using free and public code. Contents 3 Table of Contents 4 Preface 58 Challenges Facing Digital Infrastructure 5 Foreword 59 Open source’s complicated relationship with money 8 Executive Summary 66 Why digital infrastructure support 11 Introduction problems are accelerating 77 The hidden costs of ignoring infrastructure 18 History and Background of Digital Infrastructure 89 Sustaining Digital Infrastructure 19 How software gets built 90 Business models for digital infrastructure 23 How not charging for software transformed society 97 Finding a sponsor or donor for an infrastructure project 29 A brief history of free and public software and the people who made it 106 Why is it so hard to fund these projects? 109 Institutional efforts to support digital infrastructure 37 How The Current System Works 38 What is digital infrastructure, and how 124 Opportunities Ahead does it get built? 125 Developing effective support strategies 46 How are digital infrastructure projects managed and supported? 127 Priming the landscape 136 The crossroads we face 53 Why do people keep contributing to these projects, when they’re not getting paid for it? 139 Appendix 140 Glossary 142 Acknowledgements ROADS AND BRIDGES: THE UNSEEN LABOR BEHIND OUR DIGITAL INFRASTRUCTURE Preface Our modern society—everything from hospitals to stock markets to newspapers to social media—runs on software. But take a closer look, and you’ll find that the tools we use to build software are buckling under demand.
    [Show full text]
  • FOSDEM 2017 Schedule
    FOSDEM 2017 - Saturday 2017-02-04 (1/9) Janson K.1.105 (La H.2215 (Ferrer) H.1301 (Cornil) H.1302 (Depage) H.1308 (Rolin) H.1309 (Van Rijn) H.2111 H.2213 H.2214 H.3227 H.3228 Fontaine)… 09:30 Welcome to FOSDEM 2017 09:45 10:00 Kubernetes on the road to GIFEE 10:15 10:30 Welcome to the Legal Python Winding Itself MySQL & Friends Opening Intro to Graph … Around Datacubes Devroom databases Free/open source Portability of containers software and drones Optimizing MySQL across diverse HPC 10:45 without SQL or touching resources with my.cnf Singularity Welcome! 11:00 Software Heritage The Veripeditus AR Let's talk about The State of OpenJDK MSS - Software for The birth of HPC Cuba Game Framework hardware: The POWER Make your Corporate planning research Applying profilers to of open. CLA easy to use, aircraft missions MySQL Using graph databases please! 11:15 in popular open source CMSs 11:30 Jockeying the Jigsaw The power of duck Instrumenting plugins Optimized and Mixed License FOSS typing and linear for Performance reproducible HPC Projects algrebra Schema Software deployment 11:45 Incremental Graph Queries with 12:00 CloudABI LoRaWAN for exploring Open J9 - The Next Free It's time for datetime Reproducible HPC openCypher the Internet of Things Java VM sysbench 1.0: teaching Software Installation on an old dog new tricks Cray Systems with EasyBuild 12:15 Making License 12:30 Compliance Easy: Step Diagnosing Issues in Webpush notifications Putting Your Jobs Under Twitter Streaming by Open Source Step. Java Apps using for Kinto Introducing gh-ost the Microscope using Graph with Gephi Thermostat and OGRT Byteman.
    [Show full text]
  • Automating Configuration N49(PDF)
    Automating Network Configuration Brent Chapman Netomata, Inc. [email protected] www.netomata.com NANOG 49 — 13 June 2010 Copyright © 2010, Netomata, Inc. All Rights Reserved. Introduction Who I am What I'm here to talk about 2 Copyright © 2010, Netomata, Inc. All Rights Reserved. Why automate network configuration? Because automated networks are More reliable Easier to maintain Easier to scale 3 Copyright © 2010, Netomata, Inc. All Rights Reserved. For example... Imagine you're managing a moderately complex web site Multiple real and virtual hosts Several "environments" (production, testing, development, etc.) Separate VLAN for each environment 4 Copyright © 2010, Netomata, Inc. All Rights Reserved. For example... What networking devices & services need to be managed? Routers Switches Load Balancers Firewalls Real-time status monitoring (i.e., Nagios) Long-term usage monitoring (i.e., MRTG) 5 Copyright © 2010, Netomata, Inc. All Rights Reserved. For example... How to add new virtual host to existing load balancer pool? Set up host itself, using Puppet or cfengine or whatever Add host to VLAN defs on switches Add host to ACLs on routers Add host to pool on load balancers Add host to NAT and ACLs on firewalls Add host to real-time monitoring (i.e., Nagios) Add host to usage monitoring (i.e., MRTG) 6 Copyright © 2010, Netomata, Inc. All Rights Reserved. For example... What's the problem with doing all that by hand? You have to remember how to manage all those very different devices (and you probably don't do it very often) It takes a lot of time Every step is a chance to make a mistake You might get distracted, and never finish Over time, these small mistakes add up, leading to inconsistent networks that are unreliable and difficult to troubleshoot 7 Copyright © 2010, Netomata, Inc.
    [Show full text]
  • Puppet Dashboard 1.2 Manual
    Puppet Dashboard Manual (Generated on July 01, 2013, from git revision 46784ac1656bd7b57fcfb51d0865ec7ff65533d9) Puppet Dashboard 1.2 Manual This is the manual for Puppet Dashboard 1.2. Overview Puppet Dashboard is a web interface for Puppet. It can view and analyze Puppet reports, assign Puppet classes and parameters to nodes, and view inventory data and backed-up file contents. Chapters Installing Dashboard Upgrading Dashboard Configuring Dashboard Maintaining Dashboard Using Dashboard Rake API Installing Puppet Dashboard This is a chapter of the Puppet Dashboard 1.2 manual. NAVIGATION Installing Dashboard Upgrading Dashboard Configuring Dashboard Maintaining Dashboard Using Dashboard Rake API Overview Puppet Dashboard is a Ruby on Rails web app that interfaces with Puppet. It will run on most modern Unix-like OSes (including Mac OS X and most Linux distributions), requires a certain amount of supporting infrastructure, and can be deployed and served in a variety of ways. Dashboardʼs web interface supports the following browsers: Chrome (current versions) Firefox 3.5 and higher Puppet Dashboard Manual • Puppet Dashboard 1.2 Manual 2/27 Safari 4 and higher Internet Explorer 8 and higher Installing, in Summary In outline, the steps to get Dashboard running are: Installing the external dependencies Installing the Dashboard code Configuring Dashboard Creating and configuring a MySQL database Testing that Dashboard is working Configuring Puppet Starting the delayed job worker processes Running Dashboard in a production-quality server After completing these tasks, Dashboardʼs main functionality will be on-line and working smoothly. You can then configure Dashboard further and enable optional features If you are trying to upgrade Puppet Dashboard instead of installing it from scratch, see the chapter of this manual on upgrading instead of reading further in this chapter.
    [Show full text]
  • Third-Party Software for Engage Products APPLICATIONS Red Hat
    Third-Party Software for Engage Products APPLICATIONS Red Hat Enterprise Linux General Information Source Code Status Not modified by Vocera URL https://www.redhat.com/en/about/licenses-us Supplemental END USER LICENSE AGREEMENT (November 2010) License Text RED HAT® ENTERPRISE LINUX® AND RED HAT APPLICATIONS PLEASE READ THIS END USER LICENSE AGREEMENT CAREFULLY BEFORE USING SOFTWARE FROM RED HAT. BY USING RED HAT SOFTWARE, YOU SIGNIFY YOUR ASSENT TO AND ACCEPTANCE OF THIS END USER LICENSE AGREEMENT AND ACKNOWLEDGE YOU HAVE READ AND UNDERSTAND THE TERMS. AN INDIVIDUAL ACTING ON BEHALF OF AN ENTITY REPRESENTS THAT HE OR SHE HAS THE AUTHORITY TO ENTER INTO THIS END USER LICENSE AGREEMENT ON BEHALF OF THAT ENTITY. IF YOU DO NOT ACCEPT THE TERMS OF THIS AGREEMENT, THEN YOU MUST NOT USE THE RED HAT SOFTWARE. THIS END USER LICENSE AGREEMENT DOES NOT PROVIDE ANY RIGHTS TO RED HAT SERVICES SUCH AS SOFTWARE MAINTENANCE, UPGRADES OR SUPPORT. PLEASE REVIEW YOUR SERVICE OR SUBSCRIPTION AGREEMENT(S) THAT YOU MAY HAVE WITH RED HAT OR OTHER AUTHORIZED RED HAT SERVICE PROVIDERS REGARDING SERVICES AND ASSOCIATED PAYMENTS. This end user license agreement (“EULA”) governs the use of any of the versions of Red Hat Enterprise Linux, certain other Red Hat software applications that include or refer to this license, and any related updates, source code, appearance, structure and organization (the “Programs”), regardless of the delivery mechanism. 1. License Grant. Subject to the following terms, Red Hat, Inc. (“Red Hat”) grants to you a perpetual, worldwide license to the Programs (most of which include multiple software components) pursuant to the GNU General Public License v.2.
    [Show full text]
  • Interfacing Apache HTTP Server 2.4 with External Applications
    Interfacing Apache HTTP Server 2.4 with External Applications Jeff Trawick Interfacing Apache HTTP Server 2.4 with External Applications Jeff Trawick November 6, 2012 Who am I? Interfacing Apache HTTP Server 2.4 with External Applications Met Unix (in the form of Xenix) in 1985 Jeff Trawick Joined IBM in 1990 to work on network software for mainframes Moved to a different organization in 2000 to work on Apache httpd Later spent about 4 years at Sun/Oracle Got tired of being tired of being an employee of too-huge corporation so formed my own too-small company Currently working part-time, coding on other projects, and taking classes Overview Interfacing Apache HTTP Server 2.4 with External Applications Jeff Trawick Huge problem space, so simplify Perspective: \General purpose" web servers, not minimal application containers which implement HTTP \Applications:" Code that runs dynamically on the server during request processing to process input and generate output Possible web server interactions Interfacing Apache HTTP Server 2.4 with External Applications Jeff Trawick Native code plugin modules (uhh, assuming server is native code) Non-native code + language interpreter inside server (Lua, Perl, etc.) Arbitrary processes on the other side of a standard wire protocol like HTTP (proxy), CGI, FastCGI, etc. (Java and \all of the above") or private protocol Some hybrid such as mod fcgid mod fcgid as example hybrid Interfacing Apache HTTP Server 2.4 with External Applications Jeff Trawick Supports applications which implement a standard wire protocol, no restriction on implementation mechanism Has extensive support for managing the application[+interpreter] processes so that the management of the application processes is well-integrated with the web server Contrast with mod proxy fcgi (pure FastCGI, no process management) or mod php (no processes/threads other than those of web server).
    [Show full text]
  • Puppet Offers a Free, Reliable and Cross Flavor Option for Remote Enterprise Computer Management
    This material is based on work supported by the National Science Foundation under Grant No. 0802551 Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author (s) and do not necessarily reflect the views of the National Science Foundation C4L8S1 System administrators are constantly challenged when managing large enterprise systems using Linux-based operating systems. Administrators need to know a variety of command line differentiations, dependency variations, and support options to support the various computers systems in use. Puppet offers a free, reliable and cross flavor option for remote enterprise computer management. This lesson will introduce you to the Puppet AdministrativeU the tool and provide you with a basic overview on how to use Puppet. Lab activities will provide you with hands-on experience with the Puppet application and assignments and discussion activities will increase your learning on this subject. Understanding Puppet is important because of its ability to manage enterprise systems. Students hoping to become Linux Administrators must gain mastery of enterprise management tools like Puppet to improve efficiency and productivity. C4L8S2 You should know what will be expected of you when you complete this lesson. These expectations are presented as objectives. Objectives are short statements of expectations that tell you what you must be able to do, perform, learn, or adjust after reviewing the lesson. Lesson Objective: U the Given five computers that need to be configured,
    [Show full text]
  • Build Tools & Package Manager
    Module Checklist Build Tools & Package Manager By Techworld with Nana Video Overview ★ Introduction to Build and Package Manager Tools ★ Install Build Tools ★ Windows Installation Help - Part 1 ★ Windows Installation Help - Part 2 ★ MacOS/Unix Installation Help ★ Build Artifact ★ Build Tools for Development (Managing Dependencies) ★ Run/Start the application ★ Build JavaScript applications ★ Other Programming Languages ★ Publish Artifact ★ Build Tools & Docker ★ Build Tools & DevOps Demo Infos Java Gradle Project https://gitlab.com/nanuchi/java-app Java Maven Project https://gitlab.com/nanuchi/java-maven-app React-Node Project https://github.com/bbachi/react-nodejs-example Check your progress... 1/3 Introduction to Build & Package Manager Tools ❏ Watched video Install Build Tools ❏ Watched video ❏ Go to Windows or MacOS/Unix Installation Help Useful Links: ● Maven: https://maven.apache.org/install.html ● Node.js: https://nodejs.org/en/download/ - npm is distributed with Node.js Windows Installation Help - Part 1 + 2 ❏ Watched video ❏ Installed everything Useful Links: ● Java on Windows: https://www3.ntu.edu.sg/home/ehchua/programming/howto/JDK_Howto.html MacOS/Unix Installation Help ❏ Watched video ❏ Installed everything Useful Links: ● Homebrew Package Manager: https://brew.sh/ ● Java on Mac/Linux: Install using your OS package manager (e.g. brew) ● Symbolic Link Cmd I use in the demo: https://mkyong.com/java/how-to-install-java-on-mac-osx/ Check your progress… 2/3 Build Artifact ❏ Watched video ❏ Demo executed ❏ Built Java Gradle Project ❏ Built Java Maven Project Build Tools for Development ❏ Watched video ❏ Demo executed - add a new dependency to project Run the application ❏ Watched video ❏ Demo executed Build JavaScript applications ❏ Watched video ❏ Demo executed Other Programming Languages ❏ Watched video Publish artifact ❏ Watched video Check your progress… 3/3 Build Tools & Docker ❏ Watched video Build Tools & DevOps ❏ Watched video More Resources..
    [Show full text]
  • Configuration File Manipulation with Configuration Management Tools
    Configuration File Manipulation with Configuration Management Tools Student Paper for 185.307 Seminar aus Programmiersprachen, SS2016 Bernhard Denner, 0626746 Email: [email protected] Abstract—Manipulating and defining content of configuration popular CM-Tool in these days and offers a huge variety of files is one of the central tasks of configuration management configuration manipulation methods. But we will also take a tools. This is important to ensure a desired and correct behavior look at other CM-Tools to explore their view of configuration of computer systems. However, often considered very simple, this file handling. task can turn out to be very challenging. Configuration manage- ment tools provide different strategies for content manipulation In Section II an introduction will be given about CM- of files, hence an administrator should think twice to choose the Tools, examining which ones are established in the community best strategy for a specific task. This paper will explore different and how they work. Before we dive into file manipulation approaches of automatic configuration file manipulation with the strategies, Section III will give some introduction in configu- help of configuration management tools. We will take a close look at the configuration management tool Puppet, as is offers various ration stores. Section IV will describe different configuration different approaches to tackle content manipulation. We will also manipulation strategies of Puppet, whereas Section V will explore similarities and differences of approaches by comparing look at approaches of other CM-Tools. Section VI will show it with other configuration management tools. This should aid some scientific papers about CM-Tools.
    [Show full text]
  • Confex: a Framework for Automating Text-Based Software Configuration
    1 ConfEx: A Framework for Automating Text-based Software Configuration Analysis in the Cloud Ozan Tuncer, Anthony Byrne, Nilton Bila, Sastry Duri, Canturk Isci, and Ayse K. Coskun Abstract—Modern cloud services have complex architectures, often comprising many software components, and depend on hundreds of configurations parameters to function correctly, securely, and with high performance. Due to the prevalence of open-source software, developers can easily deploy services using third-party software without mastering the configurations of that software. As a result, configuration errors (i.e., misconfigurations) are among the leading causes of service disruptions and outages. While existing cloud automation tools ease the process of service deployment and management, support for detecting misconfigurations in the cloud has not been addressed thoroughly, likely due to the lack of frameworks suitable for consistent parsing of unstandardized configuration files. This paper introduces ConfEx, a framework that enables discovery and extraction of text-based software configurations in the cloud. ConfEx uses a novel vocabulary-based technique to identify configuration files in cloud system instances with unlabeled content. To extract the information in these files, ConfEx leverages existing configuration parsers and post-processes the extracted data for analysis. We show that ConfEx achieves over 99% precision and 100% recall in identifying configuration files on 7805 popular Docker Hub images. Using two applied examples, we demonstrate that ConfEx also enables detecting misconfigurations in the cloud via existing tools that are designed for configurations represented as key-value pairs, revealing 184 errors in public Docker Hub images. Index Terms—Software configuration, cloud, misconfiguration diagnosis. F 1 INTRODUCTION LOUD applications are designed in a highly config- Existing failure avoidance and mitigation mechanisms in C urable way to ensure high levels of reusability and the cloud (e.g., redundancy or recovery) are insufficient to portability.
    [Show full text]
  • Questions for Mongrel
    www.YoYoBrain.com - Accelerators for Memory and Learning Questions for Mongrel Category: Introduction - (16 questions) Mongrel is described in what way in the "A web application container for Ruby on Mongrel pdf available from O Reilly Rails" Mongrel is compared with what web servers production performance: Fast CGI or SCGI in the Rails world in terms of production performance and development Development: WEBrick simplicity/speed Creator of Mongrel Zed A Shawwww.zedshaw.com Mongrel is developed on what mixture of Ruby and C programming/scripting languages Documentation for Mongrel mongrel.rubyforge.org/docs/index.html The creators of Mongrel describe it how? a fast HTTP library and server for Ruby that is intended for hosting Ruby web applications of any kind using plain HTTP rather than FastCGI or SCGI. It is framework agnostic Three key technologies that are used for A custom HTTP 1.1 parser (based on RFC Mongrel's internals standard, written using Ragel in C and Java as a Rby extension) Simple server that uses the parser and URIClassifier to process requests, find the right handlers, then pass the results to the handler for processing Handlers are responsible for using HttpRequet and HttpResponse objects to "do their thing and then return results" Component of Mongrel responsible for Handlers dealing with HttpRequest and HttpResponse How does Mongrel support threading one thread per request, but it will start closing connections when it gets "overloaded"while Mongrel is processing HTTP requests and sending responses it uses Ruby's threading system What platforms that already work with Camping and Og+Nitro Mongrel are throught to be "thread-safe" Have not been heavily tested Is Ruby on Rails thread safe? no How does Mongrel handle Rails" " Ruby on Rails is not thread safe so there is a synchronized block around the calls to Dispatcher.dispatch.
    [Show full text]