DOCSLIB.ORG

Foreman and Puppet @ University of Bonn

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Foreman and Puppet @ University of Bonn FOREMAN AND PUPPET @ UNIVERSITY OF BONN OLIVER FREYERMUTH, PETER WIENEMANN PHYSICS INSTITUTE @ U BONN ● The Physics Institute at the University of Bonn has – ~240 members – ~1500 registered devices in IPAM – Biggest particle accelerator run by a German university (164.4 m circumference) with two „large“ experiments (O(50) collaborators) ● IT group of Physics Institute – 2 „full time“ persons with teaching duties – 1 technician – 5 part-time persons (~0.2 FTE each) – Plan to merge with IT group of Nuclear/Hadron Physics Institute (HISKP) O. Freyermuth and P. Wienemann: Foreman and Puppet @ University of Bonn 2 MORE NUMBERS ● Run approximately 400 centrally managed Linux nodes at Physics Institute – Desktops (Ubuntu 18.04) → ~150 nodes – HPC/HTC cluster nodes (SL6/CentOS 7) → ~150 nodes – Servers → ~100 nodes with ~40 different services ● O(1000) self-managed (sometimes ~unmanaged) devices (laptops, lab PCs, printers, cameras, oscilloscopes, H.323 devices, TVs, tablets, PDUs, ...) ● Use central university computing centre offerings for standard services like mail system, web content management system, IPAM, ticket system, network services, OwnCloud with Onlyoffice, etc. O. Freyermuth and P. Wienemann: Foreman and Puppet @ University of Bonn 3 HISTORY ● Using Puppet since ~2009 – Completely self-written manifests, no modules, poor maintainability – Only used for desktops and HPC/HTC cluster nodes ● Refactored everything in ~2014: Switch to „roles and profiles“ paradigm – Started using publicly available third-party modules – Still self-written PXE/kickstart/preseed machinery ● Started Foreman tests in 2015 Migration of production systems to Foreman started in 2017 – Full puppetization of all nodes/services – Discarded self-written deployment machinery → replaced by Foreman tools O. Freyermuth and P. Wienemann: Foreman and Puppet @ University of Bonn 4 LAYOUT Network A Network B Foreman Proxy Foreman Proxy (Puppet Master, TFTP) (Puppet Master, TFTP) Network C Network D Foreman Server (Puppet-Master, CA) Foreman Proxy Foreman Proxy (Puppet Master, TFTP) (Puppet Master, TFTP) Git Repository O. Freyermuth and P. Wienemann: Foreman and Puppet @ University of Bonn 5 PRESENT WORKFLOW ● Most of the presently used 86 Puppet modules are third-party code. Forked on Github and collected on https://github.com/unibonn. Patches always sent upstream. ● Discarded „roles“ → Foreman hostgroups/config groups ● „profiles“ kept on private, on premise git hosting platform – Contains site/host specific configuration, augmented by parameters (see below) ● Module management done with r10k, synced to all Puppet masters ● Foreman Puppet class filter: Only „profiles“ visible (to avoid clutter) ● No hiera → Foreman global and smart class parameters ● No mcollective O. Freyermuth and P. Wienemann: Foreman and Puppet @ University of Bonn 6 QUESTIONS ● What is your Puppet module design „philosophy“? ● How do you manage Puppet code? ● How do you handle parameters? ● Which Foreman plugins do you use/do you consider useful? O. Freyermuth and P. Wienemann: Foreman and Puppet @ University of Bonn 7 Thanks for your attention! Oliver Freyermuth: [email protected] Peter Wienemann: [email protected].
Load more

Recommended publications
  • Red Hat Satellite 6.7 Provisioning Guide
    Red Hat Satellite 6.7 Provisioning Guide A guide to provisioning physical and virtual hosts on Red Hat Satellite Servers. Last Updated: 2021-05-14 Red Hat Satellite 6.7 Provisioning Guide A guide to provisioning physical and virtual hosts on Red Hat Satellite Servers. Red Hat Satellite Documentation Team [email protected] Legal Notice Copyright © 2021 Red Hat, Inc. The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/ . In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version. Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law. Red Hat, Red Hat Enterprise Linux, the Shadowman logo, the Red Hat logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries. Linux ® is the registered trademark of Linus Torvalds in the United States and other countries. Java ® is a registered trademark of Oracle and/or its affiliates. XFS ® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries. MySQL ® is a registered trademark of MySQL AB in the United States, the European Union and other countries.
    [Show full text]
  • Types of LMS Deployment & Common Features
    Types of LMS Deployment & Common Features Daisyane Barreto, Amy Rottmann, & Salena Rabidoux In cooking, you have many different types of tools you can use to make a meal. For example, you have mixers, measuring cups, cutting boards, and so many other tools. Each tool is designed to meet the needs of the cooker and serve a specific purpose in the cooking process. An LMS is not different in that sense. There are several types of LMSs in the market being used for educational as well as training purposes, and one of the most difficult choices for organizations and institutions to make is deciding the type of LMS to select based on deployment or license (Pappas, 2014). Of course, many other factors (e.g., pricing, support, additional features, and others) must be considered and weighed in this decision. Still, the cost combined with financial and technical requirements are still the main focus when selecting an LMS (Croitoru & Dinu, 2016). Indeed, it is important to analyze and examine the type of LMS deployment because of its influence in the overall cost as well as other relevant requirements that can inform the decision about an LMS. There are two main types of LMS deployment you need to be familiar with: (1) proprietary, and (2) open-source. Within those two solutions, there are two other distinct categories to be considered: (a) Software as a Service (SaaS) or cloud-based system and (b) Installed LMS. In order to determine what type of LMS is beneficial to an organization or institution, a clear understanding of the advantages and disadvantages of each solution must be reviewed.
    [Show full text]
  • Foreman Basics Be Smart
    Foreman Basics Be smart. Think open source. Foreman - Basics Lifecycle management of physical and virtual machines made easy! Agenda Introduction to Foreman Architecture Setup Provisioning Configuration Monitoring Advanced features Introduction to Foreman What's it all about? Facts Project started in 2009 Licensed under the GPLv3 Development pushed by Red Hat Very active & helpful community Overview Tool for provisioning of VMs & bare metal Provides config management & monitoring integration Rails & JavaScript application Exposes a web interface, REST API & CLI Ecosystem Foreman Smart Proxy (foreman-proxy) Katello Tons of plugins Strong suite Very flexible Offers tons of features Active development & open community Modular setup, start small then expand Strong suite Can serve as a source of truth (CMDB) Can be used as an ENC Proper ACL implementation Enterprise Support available (Red Hat Satellite 6) Weak spots Somewhat steep learning curve Can be quite tricky to debug an issue API has room for improvement Offers sometimes too many possible ways to implement a task Architecture Overview of the different components Bird's-eye view Foreman Heart of the whole stack Stores all resources & information Rails stack, use Passenger + nginx / Apache to run it Stores most data in a DB (SQLite, MySQL or PostgreSQL) Local or LDAP users for authentication Smart Proxy Small autonomous HTTP application Exposes a REST API to provide different services Allows Foreman to control components in isolated networks Also called foreman-proxy Smart Proxy DHCP DNS TFTP BMC / IPMI Puppet / Salt / Chef / Ansible Realm / FreeIPA Smart Proxy - DHCP Takes care of reserving the required IPs Provides IP auto-assignment Supports ISC DHCP, MS DHCP & libvirt More providers can be installed or developed (e.g.
    [Show full text]
  • Before Peer Production: Infrastructure Gaps and the Architecture of Openness in Synthetic Biology
    BEFORE PEER PRODUCTION: INFRASTRUCTURE GAPS AND THE ARCHITECTURE OF OPENNESS IN SYNTHETIC BIOLOGY David Singh Grewal* In memoriam Mark Fischer (1950 – 2015)µ * Professor of Law, Yale Law School, and Director, BioBricks Foundation. The views expressed here are my own, and do not reflect the views of any of the organizations or entities with which I am or have been affiliated. Among those to whom I owe thanks, I am especially grateful to Drew Endy, President of the BioBricks Foundation (BBF) who worked to educate me about synthetic biology, and later invited me to join the board of directors of the BBF, and to the late Mark Fischer, who was the lead drafter (along with Drew Endy, Lee Crews, and myself) of the BioBrick™ Public Agreement, and to whose memory this Article is dedicated. I am also grateful to other past and present directors and staff of the BBF, including Richard Johnson, Linda Kahl, Tom Knight, Thane Krier, Nathalie Kuldell, Holly Million, Jack Newman, Randy Rettberg, and Pamela Silver. I have benefitted from conversations about synthetic biology and open source theory with a number of other scientists, including Rob Carlson, George Church, Jason Kelly, Manu Prakash, Zach Serber, Reshma Shetty, Christina Smolke, Ron Weiss and with a number of lawyers, scholars, and open source advocates including Yochai Benkler, James Boyle, Daniela Cammack, Paul Cammack, Paul Goldstein, Hank Greely, Janet Hope, Margot Kaminski, Mark Lemley, Stephen Maurer, Eben Moglen, Lisa Ouellette, Jedediah Purdy, Arti Rai, Pamela Samuelson, Jason Schultz, and Andrew Torrance, and especially Talli Somekh, who first introduced me to synthetic biology and noted the connections between my work in network theory and new developments in biotechnology.
    [Show full text]
  • An Overview of Security in the Freebsd Kernel 131 Dr
    AsiaBSDCon 2014 Proceedings March 13-16, 2014 Tokyo, Japan Copyright c 2014 BSD Research. All rights reserved. Unauthorized republication is prohibited. Published in Japan, March 2014 INDEX P1A: Bold, fast optimizing linker for BSD — Luba Tang P1B: Visualizing Unix: Graphing bhyve, ZFS and PF with Graphite 007 Michael Dexter P2A: LLVM in the FreeBSD Toolchain 013 David Chisnall P2B: NPF - progress and perspective 021 Mindaugas Rasiukevicius K1: OpenZFS: a Community of Open Source ZFS Developers 027 Matthew Ahrens K2: Bambi Meets Godzilla: They Elope 033 Eric Allman P3A: Snapshots, Replication, and Boot-Environments—How new ZFS utilities are changing FreeBSD & PC-BSD 045 Kris Moore P3B: Netmap as a core networking technology 055 Luigi Rizzo, Giuseppe Lettieri, and Michio Honda P4A: ZFS for the Masses: Management Tools Provided by the PC-BSD and FreeNAS Projects 065 Dru Lavigne P4B: OpenBGPD turns 10 years - Design, Implementation, Lessons learned 077 Henning Brauer P5A: Introduction to FreeNAS development 083 John Hixson P5B: VXLAN and Cloud-based networking with OpenBSD 091 Reyk Floeter INDEX P6A: Nested Paging in bhyve 097 Neel Natu and Peter Grehan P6B: Developing CPE Routers based on NetBSD: Fifteen Years of SEIL 107 Masanobu SAITOH and Hiroki SUENAGA P7A: Deploying FreeBSD systems with Foreman and mfsBSD 115 Martin Matuška P7B: Implementation and Modification for CPE Routers: Filter Rule Optimization, IPsec Interface and Ethernet Switch 119 Masanobu SAITOH and Hiroki SUENAGA K3: Modifying the FreeBSD kernel Netflix streaming servers — Scott Long K4: An Overview of Security in the FreeBSD Kernel 131 Dr. Marshall Kirk McKusick P8A: Transparent Superpages for FreeBSD on ARM 151 Zbigniew Bodek P8B: Carve your NetBSD 165 Pierre Pronchery and Guillaume Lasmayous P9A: How FreeBSD Boots: a soft-core MIPS perspective 179 Brooks Davis, Robert Norton, Jonathan Woodruff, and Robert N.
    [Show full text]
  • Foreman / Katello Administration
    "Charting the Course ... ... to Your Success!" Foreman / Katello Administration Course Summary Description Foreman and Katello together are the extremely popular enterprise infrastructure management solution. Foreman and Katello allow complete lifecycle management of Linux servers from a single management console. The main features include the ability to centrally provision new servers whether bare metal, private cloud/virtualization platforms, or in the public cloud. After the servers are provisioned, Foreman supports centralized configuration management of the servers using Puppet, Ansible, Salt or Chef. Foreman's Katello plugin allows for versioned content repositories to move through defined life cycles such as DEV, QA, and PROD as well as errata management. This four-day course provides comprehensive hands on coverage all the of the major features starting from best practices in installation and initial configuration, managing subscriptions, products and repositories, content views, lifecycles, activation keys, system registration of existing system, bare metal and virtualization provisioning including discovery, provisioning templates, configuration management with puppet, IdM integration, smart proxies/capsule servers, virt-who, and subscription management. Foreman and Katello are rather complex and students who take this class will feel comfortable deploying, configuring using best practices and managing it day to day. Topics Overview and Installation Provisioning New Systems Initial Configuration Virtualization Integration Registering Existing Hosts Deploying a Smart Proxy, aka Capsule Puppet Integration Server Identity Management Integration Audience This course is designed for those wanting to learn Foreman and Katello and understand the complete lifecycle management of Linux servers from a single management console. Prerequisites An intermediate knowledge of the Linux operating system is required.
    [Show full text]
  • Er the Microscope
    I 3in> m zn wmrr r~ — H C ZHHtfl <U OH rco O2Z2 00 »!fl 09 X30 -j M i -j r -J 1-1 Vol. 421 Friday, February 14,1997 > TO er the microscope With the help of magnification, what was once purely scientific is now artistic in a new exhibit By Steve Janas Whether depicting a microchip or a The Packet Group virus, each picture explodes with col- or, with the exception of the gray As a piece of expressionist fanta- first-place prize winner, which makes sy, the picture's not bad. From lay- up for its deficiency with rich tex- ered shades of gray, there rises a line tures. The shapes are varied and dy- of rigid cones that give the distinct namic: There are delicate crystalline impression of ramparts. You think, structures and amorphous blobs. perhaps, of Elsinore, moody Prince Hamlet's home, although re-con- The images are stunning, but not ceived for the post-industrial era. A many people know about them, Ms. reasonably successful feat of the Albright says. She says she happened imagination, you might conclude. upon news of last year's competition while browsing through an obscure Except that there's nothing imag- technical journal geared toward mi- inary about it. This is an un-doctored, croscopic photography. She decided un-retouched photograph of an actual right away that it would be a perfect object you would be able to see with fit for the museum. your own eyes, if they were capable "It's a little beyond what we nor- of magnifying objects 80 times.
    [Show full text]
  • Integrating Ovirt, Foreman and Katello to Empower Your Data-Center Utilization
    Integrating oVirt, Foreman And Katello To Empower Your Data-Center Utilization Yaniv Bronhaim Senior Software Engineer, Maintainer @ RHEV Red Hat IL, Raanana August 2015, CloudOpen Europe CloudOpen 2015 Agenda ● Introducing Open-Source solutions ● oVirt, Foreman and Katello ● New third-party integration in oVirt 3.6 ● Using Foreman for Bare-Metal Provisioning ● Using Katello for displaying Errata (update) content information for hosts, Vms and Engine ● oVirt 3.6 Upgrade Manager ● oVirt 4.0 plans CloudOpen 2015 Large scale, centralized management for server and desktop virtualization Provide an open source alternative to vCenter/vSphere Focus on KVM for best integration/performance CloudOpen 2015 Focus #1 : Performance And Scalability Maintain a large amount of resources Data Centers Linux Hypervisors Network Storage Guests \ VMs CloudOpen 2015 Focus #2 : User Intuitive Different type of users and requirements CloudOpen 2015 Focus #3 : Secure and Recoverable It hurts to lose your VM in the cloud CloudOpen 2015 Data Centers Linux Hypervisors Network Storage Guests \ VMs CloudOpen 2015 oVirt Engine Redhat Enterprise Virtualization http://www.ovirt.org/Quick_Start_Guide CloudOpen 2015 CloudOpen 2015 Host Deploy The Basic Of Utilizing The Hypervisor Engine Client (Web Portal) CloudOpen 2015 Deploy Host Flow Engine Client (web portals) CloudOpen 2015 New Host Deploy (Configure) CloudOpen 2015 Host Deployment The process of preparing an operating system environment suitable to host virtual machines and to be managed by the ovirt back-end ● VDSM
    [Show full text]
  • How to Keep Track of Puppet with Foreman
    How to keep track of puppet with Foreman Glen Ogilve - [email protected] What does the do? Provisioning "Provision on bare-metal & public or private clouds all from one place." It does with a mix of: hardware via network boot and automated install virtual by connecting to the hypervisors AMI customisation Network For network installation, it signals the target server to network boot, by communicating with the BMC or hypervisor, and creates the appropraite network boot PXE setup via the foreman proxy to install the OS. The contents of the network boot file are built from templates and customised before starting the installation. It relies on a thrid party source for the installation media, and has templates available for Jumpstart, Kickstart, Preseed, AutoYast. Image based (AMI) For cloud based compute resources, new systems are provisioned using images, rather than running an installation. These are then finished by Foreman logging into the instance via ssh. Supported platforms EC2 Google compute engine Libvirt OpenStack Nova oVirt / RHEV Rackspace VMware Physical hardware via BMC Inputs When creating a system via foreman, it requires a number of details about the new system which then are used in the templates and finish scripts. This includes things like, disk size, cpus, memory, networks, puppet classes, host groups, variables, owner, availbility zone, vpc, os, architecture or AMI image Managed systems If you have provisioning turned on for a system, in some cases, you will have access to the system console, and the ability to delete/rebuild the machine from the foreman console. Turning off provisioning For basic usage of foreman with puppet, as I am describing in this talk, the provisioning component can be disabled.
    [Show full text]
  • 2015 Undergraduate Academic Catalog
    2014 – 2015 Undergraduate Catalog BETHUNE-COOKMAN UNIVERSITY 640 Dr. Mary McLeod Bethune Blvd. Daytona Beach, Florida 32114-3099 Telephone: 386-481-2000 Office of Admissions: 386-481-2600 FAX Number: 386-481-2601 www.cookman.edu 1 2014 - 2015 UNDERGRADUATE CATALOG Bethune-Cookman University is accredited by the Commission on Colleges of the Southern Association of Colleges and Schools to award the Bachelor of Arts, Bachelor of Science and Master of Science degrees. Contact the Commission on Colleges at 1866 Southern Lane, Decatur, Georgia 30033-4097 or call 404-679-4500 for questions about the accreditation of Bethune-Cookman University. Also accredited by Commission on Collegiate Nursing Education (BSN Program) National League for Nursing Accrediting Commission, Inc. 61 Broadway, 33rd Floor, New York, NY, telephone: 212-363-5555 Accreditation Commission for Programs in Hospitality Administration National Collegiate Athletic Association Florida State Board of Nursing (approved) Florida State Department of Education for Special Programs National Council for Accreditation of Teacher Education University Senate of the United Methodist Church Affiliated with American Association of Colleges of Nursing American Association of Collegiate Registrars and Admissions Officers American Association of Colleges for Teacher Education American Assembly of Collegiate Schools of Business American Association of Higher Education American Council on Education American Library Association American Management Association Association of American Colleges and
    [Show full text]
  • Learn Ruby on Rails a Tutorial by Daniel Kehoe · 1.C16 (Prerelease) · 18 November 2013 2 Contents
    1 learn-rails.com Learn Ruby on Rails A tutorial by Daniel Kehoe · 1.c16 (prerelease) · 18 November 2013 2 Contents 1. About the Book ........................................... 3 29. Credits and Comments......................... 247 2. Introduction ................................................. 4 3. Concepts........................................................ 9 4. Get Help When You Need It................. 15 5. Plan Your Product.................................... 19 6. Manage Your Project............................... 26 7. Accounts You May Need ....................... 28 8. Get Started.................................................. 31 9. Create the Application............................ 39 10. The Parking Structure............................. 47 11. Time Travel with Git ............................... 51 12. Gems ............................................................ 61 13. Configure.................................................... 71 14. Static Pages and Routing........................ 76 15. Request and Response ............................ 79 16. Dynamic Home Page .............................. 88 17. Troubleshoot.............................................. 98 18. Just Enough Ruby .................................. 110 19. Layout and Views .................................. 132 20. Front-End Framework .......................... 155 21. Add Pages................................................. 176 22. Contact Form ........................................... 181 23. Spreadsheet Connection......................
    [Show full text]
  • Presentation
    Linux Clusters Institute: Cluster Stack Basics Derek Weitzel Software Engineer Holland Computing Center, University of Nebraska Open Science Grid 1 A Bunch of Computers • Users can login to any node • User file systems aren’t shared between nodes • Work is run wherever you can find space • Nodes maintained individually 2 What’s wrong with a bunch of nodes? • Competition for resources • Size and type of problem is limited • Nodes get out of sync • Problems for users • Difficulty in management 3 Cluster Approach • Shared file systems • Job management • Nodes dedicated to compute • Consistent environment • Interconnect 4 What’s right about the cluster approach? • Easier to use • Maximize efficiency • Can do bigger and better problems • Nodes can be used cooperatively 5 The Types of Nodes • Login • Users login here • Compiling • Editing • Submitting and Monitoring jobs • Compute • Users might login here • Run jobs as directed by the scheduler • Support • Users don’t login here • Do all the other stuff 6 Technology Choices • I’m going to be listing a lot of different technologies • Reasons to choose a technology: 1. Best for a job 2. Knowledge of the technology 3. Mindshare – How big is the community? How easy is it to “StackOverflow” a problem? 4. Institutional momentum – Is the institution already using it? 7 Cluster Design • Each cluster is different • Different OS, different tools… • Even very different design decisions. Some clusters are ‘stateless’, with no local disk. • Every reboot is a re-install. • There’s no easy-button that says:
    [Show full text]