DOCSLIB.ORG

Automating Arcgis Deployments Using Chef

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

Automating ArcGIS Deployments Using Chef Cherry Lin ([email protected]) Bill Major ([email protected]) David Cordes ([email protected]) Slides: http://bit.ly/2uiEG0p Overview • What, Why, When? • Esri’s Chef Cookbooks • Easy Cooking Demo • Best Practices • Case Study: Esri What, Why, When What is Automation and Why • The linking of disparate systems and software in such a way that they become self-acting or self-regulating. • Reliable, repeatable and testable deployments and upgrades - DEV → TEST → PROD - Moving between Data Centers - Faster Disaster Recovery • Quick configuration with disposable machines and rapid auto-scaling • Automate and become more efficient at administration - Focus more on GIS tasks - Prepare for larger scale and clouds - Ready-to-use tools with minimum requirements to start. Operate Software Org Operation (HA, DR) Administration Content Software Configuration Management Software Analytics Installation GIS Infrastructure Automation Software Operation (HA, DR) Software Configuration Software Installation Software Setups, Enterprise Builder, Chef, Cloud formation, Cloud Builder GIS Workflow Automation Org Administration Content Management Analytics Python API for ArcGIS, WebGISDR, Admin API, ArcPy, R-bridge for ArcGIS GIS Infrastructure Automation Software Operation (HA, DR) Software Configuration Software Installation Software Setups, Enterprise Builder, Chef, Cloud formation, Cloud Builder ArcGIS Enterprise Software Setups = + + + ArcGIS Portal ArcGIS ArcGIS ArcGIS Web Adaptor for ArcGIS Server Data Store Enterprise ArcGIS Enterprise Software Setups • No need to learn 3-party technology. • Deploy exact the way you want • Other Esri software • Time consuming • More manual steps • Manual configurations following different docs • One time task • For customized deployments • Not recommended for highly available and scalable deployments ArcGIS Enterprise Builder ArcGIS Portal Web Adaptor for ArcGIS ArcGIS Hosting Web Adaptor Server ArcGIS Data Store (relational + tile cache) ArcGIS Enterprise Builder • All-in-one deployment • One GUI setup, 5 or 6 parameters • Easy to use • A lot less configurations • One deployment pattern for one machine quick start • Not for a system that needs quickly redeployed • Not a DR deployment? ArcGISIn the Cloud Enterprise Software Setups • AWS - Esri AMIs - Esri CloudFormation Templates - Python and Powershell scripts •Microsoft Azure - Esri Images - Cloud Builder •Highly recommend using our Images and tools - Customization based on our Images and tools - Details: AWS and Azure sessions ArcGISEsri ArcGIS Enterprise Chef Cookbooks Software Setups • IT standard automation framework • Automate all that you can manually do • Deploy exact the way you want. You have full control. •Other Esri software •Require small up-front investments •Reliable, repeatable and testable - Documented deployments - High Availability and Disaster Recovery deployments - Move between Data Centers - Potential scale-ups - Automate upgrades GIS Infrastructure Automation ArcGIS Enterprise Software Setups Enterprise Software Chef Builder Setups ArcGIS Cookbooks Quick start Reliable Easy to use Repeatable Typical deployment pattern Testable Simple deployment Documented deployment Limited control Full control Not for repeatable deployments Esri’s Chef Cookbooks What is Chef • Chef - One of the most popular IT automation platforms - Supported on Windows and Linux - Provide tools to take on configuration files • Chef Components - Cookbook - Attribute - Chef-Server vs. Client • How it works - Install ready-to-use cookbooks and define the configuration - Run chef with the configuration file ArcGIS Cookbook Recipe • Recipe - Attributes - Action • Property Settings: Attributes • Run-list Define You Own Attributes Define You Own Run List Steps to get started with a single machine deployment with Chef • Check out Esri Github Chef Cookbooks website – v.3.1 just released - https://github.com/Esri/arcgis-cookbook/wiki/Get-Started-With-ArcGIS-Cookbook • Install the Chef Client (v. 12.5.1 or later) • Copy ArcGIS Cookbooks to server and unzip to c:\chef - https://github.com/Esri/arcgis-cookbook/releases/tag/v3.1.0 • Stage Setups and License Files • Edit JSON configuration file to meet your own needs - c:\chef\roles\webgis-windows.json • Run chef-solo from command prompt ‘As Administrator’ - chef-solo -j C:\chef\roles\webgis-windows.json Easy Cooking Demo Supported ArcGIS Software • ArcGIS Enterprise -Portal for ArcGIS -ArcGIS Server - Enabled Roles: Hosting, Image Server, GeoAnalytics, GeoEvent Server -ArcGIS Data Store - General, spatialtemporal -ArcGIS Web Adaptor • And other Esri Software: - ArcGIS Pro - ArcGIS Insight - ArcGIS Desktop - ArcGIS License Manager Support ArcGIS Enterprise Deployment Patterns Best Practices ArcGISDistributed Enterprise Deployment Software Setups • Running Chef on different machines in order from a workstation machine - Different property files - Runs in order as required by ArcGIS Enterprise •Chef Server - Central repository - Assign roles to every machine - Run the recipes/updates accordingly - Setting up -By yourself or -Hosted Solution ArcGISUpgrade toEnterprise 10.5 or 10.5.1 Software Setups • Required properties -New Version # -New Setups -New License File •Recommend to use Chef deploy before upgrade -Easier to get the properties right ArcGISWork with Enterprise Other Automation Software Tools Setups • Esri ArcGIS Chef Cookbooks is ArcGIS deployment tool • ArcGIS Infrastructure part of your automation • Work with GIS workflow automation tools -Python API for ArcGIS -WebGISDR tool • Other software deployment automation tools • Whole system infrastructure automation -High Availability and Disaster Recovery deployment -Moving to different data centers ArcGISChef Deployment Enterprise Options Software Setups ArcGISDisconnected Enterprise Environment Software Considerations Setups • Make sure you have proper Esri license files staged for use • Software setups in a common location for access • Setting up your own Chef Server • Having Chef Client installation staged for installation ArcGISTricks and Enterprise Tips Software Setups • Consider creating a base image for some environment - Have some processes done, e.g, setups - Have some components baked in, e.g. base map data, security updates, etc. •Chef Client version 12 •Set password in an environment variable •“-l debug” for debug •Attributes value of True or False, no “”. e.g. “configure_autostart”: true •ArcGIS Cookbooks support enabling hosting server role and GeoAnalytics Server role. •Support security configuration for ArcGIS Server. ArcGISKey Points Enterprise about ArcGIS Software Chef Automation Setups • Automate Your Manual Work • Doesn’t require programming skills • Need plan ahead -Design -Configuration -Testing •Repeatable -Success in production environment -Less downtime in production -Faster recovery Case Study : Esri ArcGIS Enterprise : New Software Daily Many software changes Daily testing prevents each day. accumulation complexity. Test Automation : Not just automated functional testing Configuration automation • Software installed and configured many times per build • Many platforms • Many software components • Many architectures • Limited resources Hundreds of Scripts • Every tester had their own configuration script - Python - .Net - Java - Perl - Batch files - Etc • Wasted effort • Hard to maintain ArcGIS Cookbooks to the Rescue Solution: Use our cookbooks • Tested • Reliable • Maintained Challenge #1: But I already have a script…. “Not invented here” • Resistance to moving away from existing scripts. • Time eventually removed this barrier from most of our test teams. Challenge #2: But I don’t have time to learn a new technology... Training • In groups • One-on-one • Troubleshooting alias Outcome: Successful adoption, focus on the functionality Has taken time but we spend • Less time maintaining and troubleshooting scripts. • More time focused on functional testing. Multi-Machine Configuration Step 1: Prepare the image Multi-Machine Architecture Command Machine Tomcat + Web Adaptor test-datastore.dev.geocloud.com Hosting Portal Server Registered Data Store test-portal.dev.geocloud.com test-server.dev.geocloud.com test-datastore.dev.geocloud.com Baked an Image in Three Easy Steps Install Chef Copy Cookbook System Requirements Multiple Machine Automation Step 2: Set the Parameters One Deployment - Many Machines - One Set of Parameters Portal Server Data Store Web Server Parameters Multiple Machine Automation Step 3: Create Role-Specific Runlists Run Lists - What is Run On Each Machine Portal Server Data Store Web Server Parameters Run Lists - What is Run On Each Machine Portal Server Data Store Web Server Parameters Run Lists - What is Run On Each Machine Portal Server Data Store Web Server Parameters Run Lists - What is Run On Each Machine Portal Server Data Store Web Server Parameters Run Lists - Federation - Back to Portal Portal Server Data Store Web Server Parameters Multiple Machine Automation Step 4: Create Shell Script (Batch File) Script Writing for Multiple Machines Command Machine ● Scripts stored on command machine ● Copied to test machines at deploy time ● Run with ssh Multiple Machine Automation Step 5: Run Deployment Script Run: Multi-Machine Configuration in 30 minutes Q&A and Slides Slides: http://bit.ly/2uiEG0p .
Recommended publications
  • Alexander, Kleymenov

    Alexander, Kleymenov

    Alexander, Kleymenov Key Skills ▪ Ruby ▪ JavaScript ▪ C/C++ ▪ SQL ▪ PL/SQL ▪ XML ▪ UML ▪ Ruby on Rails ▪ EventMachine ▪ Sinatra ▪ JQuery ▪ ExtJS ▪ Databases: Oracle (9i,10g), MySQL, PostgreSQL, MS SQL ▪ noSQL: CouchDB, MongoDB ▪ Messaging: RabbitMQ ▪ Platforms: Linux, Solaris, MacOS X, Windows ▪ Testing: RSpec ▪ TDD, BDD ▪ SOA, OLAP, Data Mining ▪ Agile, Scrum Experience May 2017 – June 2018 Digitalkasten Internet GmbH (Germany, Berlin) Lead Developer B2B & B2C SaaS: Development from the scratch. Ruby, Ruby on Rails, Golang, Elasticsearch, Ruby, Ruby on Rails, Golang, Elasticsearch, Postgresql, Javascript, AngularJS 2 / Angular 5, Ionic 2 & 3, Apache Cordova, RabbitMQ, OpenStack January 2017 – April 2017 (project work) Stellenticket Gmbh (Germany, Berlin) Lead developer Application prototype development with Ruby, Ruby on Rails, Javascript, Backbone.js, Postgresql. September 2016 – December 2016 Part-time work & studying German in Goethe-Institut e.V. (Germany, Berlin) Freelancer & Student Full-stack developer and German A1. May 2016 – September 2016 Tridion Assets Management Gmbh (Germany, Berlin) Team Lead Development team managing. Develop and implement architecture of application HRLab (application for HRs). Software development trainings for team. Planning of software development and life cycle. Ruby, Ruby on Rails, Javascript, Backbone.js, Postgresql, PL/pgSQL, Golang, Redis, Salesforce API November 2015 – May 2016 (Germany, Berlin) Ecratum Gmbh Ruby, Ruby on Rails developer ERP/CRM - Application development with: Ruby 2, RoR4, PostgreSQL, Redis/Elastic, EventMachine, MessageBus, Puma, AWS/EC2, etc. April 2014 — November 2015 (Russia, Moscow - Australia, Melbourne - Munich, Germany - Berlin, Germany) Freelance/DHARMA Dev. Ruby, Ruby on Rails developer notarikon.net Application development with: Ruby 2, RoR 4, PostgreSQL, MongoDB, Javascript (CoffeeScript), AJAX, jQuery, Websockets, Redis + own project: http://featmeat.com – complex service for health control: trainings tracking and data providing to medical adviser.
  • Test Kitchen, Inspec Docker - Fastest Way to Run Stuff (Virtual Virtual Machines) DOCKER

    Test Kitchen, Inspec Docker - Fastest Way to Run Stuff (Virtual Virtual Machines) DOCKER

    TEST DRIVEN INFRASTRUCTURE COMPLIANCE AS CODE by Joaquín Menchaca À PROPOS DE MOI ABOUT ME ROCKET LAWYER SENIOR DEVOPS BUILD/RELEASE ENGINEER aka Linux Ninja Pants Automation Engineering Mutant actual photo https://slides.com/devopsstu dio/lisa18_tdi AGENDA 1. Setup 2. Context 3. Chef + InSpec 4. Ansible + InSpec 5. Bonus SETUP THE CODE https://github.com/darkn3rd/lisa18_t est_driven_infra CODE git clone \ https://github.com/darkn3rd/lisa18_test_driven_infra cd lisa18_test_driven_infra # Using Virtual Workstation vagrant up vagrant ssh cd lisa18_test_driven_infra # Using Host (MacOS X or Linux) #### Install Requirements # Using Host (Windows) #### Install Rrequiremnts #### Warning: Might not work, had success w/ Chef THE HOST SETUP must be able to run Docker or Vagrant MINIMUM REQUIRED ChefDK - bundles ruby, test kitchen, inspec Docker - fastest way to run stuff (virtual virtual machines) DOCKER Easiest Path is Docker Desktop DOCKER PACKAGE MANAGERS choco install docker-for-windows https://chocolatey.org/ brew cask install docker https://brew.sh/ DIRECT DOWNLOAD https://www.docker.com/products/docker-desktop DOCKER Debian Package on Ubuntu sudo apt-get update -qq sudo apt-get install -y apt-transport-https \ curl ca-certificates software-properties-common DOCKER_REPO="https://download.docker.com/linux/ubuntu" curl -fsSL ${DOCKER_REPO}/gpg | \ sudo apt-key add - sudo add-apt-repository \ "deb [arch=amd64] ${DOCKER_REPO} \ $(lsb_release -cs) \ stable" sudo apt-get update -qq sudo apt-get install -y docker-ce sudo usermod -aG docker $USER
  • Summer Camps

    Summer Camps

    SUMMER | 2014 RECREATION CAMPS ADULT CLASSES SENIOR SERVICES CITY RECREATION PROGRAMS & SERVICES Special Events Table of Contents Featured Events 2 Lafayette’s Annual Restaurant Walk Summer Camp Calendar 3 Tour Lafayette’s finest dining establishments and sample the Summer Camps 4-19 signature cuisine of each restaurant Preschool/Youth Classes 20-23 Tuesday, May 20, 2014 • 5:30–9:00pm Adult Classes 24-29 Wine Reception • Restaurant Tastings • Coffee and Dessert Musical entertainment at selected locations Adults 55+ Classes 30-33 Raffle Prizes • Auctions General Information 34 $45 per person Registration Form 35 www.lafayettechamber.org Summer Shows Back Cover Benefits Lafayette Community Foundation and the programs and services of the Lafayette Chamber of Commerce Lafayette Fall Ball 2014 Big League Fundamentals for Little League Players—All the Training, without the Travel! June 6, 13, 20 & 27 5/6 Yrs 7/8 Yrs 9/10 Yrs 3:30 to 4:30 4:30 to 5:30 5:30 to 6:30 Music Begins at 6:30pm Rock SESSION/DAYS DATES I: Mon/Wed 9/8 to 10/1 Visit www.LafayetteChamber.org II: Mon/Wed 10/6 to 10/29 The Plaza for full details 11/12 Yrs 3:30 to 5:30 SESSION/DAYS DATES Fridays in the Lafayette Plaza I: Thurs 9/11 to 10/2 II: Thurs 10/9 to 10/30 FEE MIN/MAX LOCATION $203 15/25 Buckeye Fields Lafayette Boys Basketball or Girls Volleyball Boy’s Basketball for Boys entering Grades 1–8 this fall & Girl’s Volleyball for Girls entering Grades 4–8 this fall New This Year: 1.
  • UNIVERSITY of CALIFORNIA, SAN DIEGO Toward Understanding And

    UNIVERSITY of CALIFORNIA, SAN DIEGO Toward Understanding And

    UNIVERSITY OF CALIFORNIA, SAN DIEGO Toward Understanding and Dealing with Failures in Cloud-Scale Systems A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Computer Science by Peng Huang Committee in charge: Professor Yuanyuan Zhou, Chair Professor Tara Javidi Professor Ranjit Jhala Professor George Porter Professor Stefan Savage 2016 Copyright Peng Huang, 2016 All rights reserved. The Dissertation of Peng Huang is approved and is acceptable in quality and form for publication on microfilm and electronically: Chair University of California, San Diego 2016 iii DEDICATION To my parents, brother and fiancée for their unconditional love and support. iv EPIGRAPH Quis custodiet ipsos custodes? (But who can watch the watchmen?) Juvenal Anything that can go wrong, will go wrong. Murphy’s law Those who fail to learn from the mistakes are doomed to repeat them. George Santayana In the middle of the night, [...] He would awaken and find himeself wondering if one of the machines had stopped working for some new, unknown reason. Or he would wake up thinking about the latest failure, the one whose cause they’d been looking for a whole week and sitll hadn’t found. The bogeyman—la machine—was there in his bedroom. Tracy Kidder, The Soul of a New Machine v TABLE OF CONTENTS SignaturePage...................................... .................. iii Dedication ......................................... .................. iv Epigraph........................................... .................. v TableofContents
  • The Apple Ecosystem

    The Apple Ecosystem

    APPENDIX A The Apple Ecosystem There are a lot of applications used to manage Apple devices in one way or another. Additionally, here’s a list of tools, sorted alphabetically per category in order to remain vendor agnostic. Antivirus Solutions for scanning Macs for viruses and other malware. • AVG: Basic antivirus and spyware detection and remediation. • Avast: Centralized antivirus with a cloud console for tracking incidents and device status. • Avira: Antivirus and a browser extension. Avira Connect allows you to view device status online. • BitDefender: Antivirus and malware managed from a central console. • CarbonBlack: Antivirus and Application Control. • Cylance: Ransomware, advanced threats, fileless malware, and malicious documents in addition to standard antivirus. • Kaspersky: Antivirus with a centralized cloud dashboard to track device status. © Charles Edge and Rich Trouton 2020 707 C. Edge and R. Trouton, Apple Device Management, https://doi.org/10.1007/978-1-4842-5388-5 APPENDIX A THe AppLe ECOSYSteM • Malware Bytes: Antivirus and malware managed from a central console. • McAfee Endpoint Security: Antivirus and advanced threat management with a centralized server to track devices. • Sophos: Antivirus and malware managed from a central console. • Symantec Mobile Device Management: Antivirus and malware managed from a central console. • Trend Micro Endpoint Security: Application whitelisting, antivirus, and ransomware protection in a centralized console. • Wandera: Malicious hot-spot monitoring, jailbreak detection, web gateway for mobile threat detection that integrates with common MDM solutions. Automation Tools Scripty tools used to automate management on the Mac • AutoCasperNBI: Automates the creation of NetBoot Images (read: NBI’s) for use with Casper Imaging. • AutoDMG: Takes a macOS installer (10.10 or newer) and builds a system image suitable for deployment with Imagr, DeployStudio, LANrev, Jamf Pro, and other asr or Apple Systems Restore-based imaging tools.
  • Idiomatic and Reproducible Software Builds Using Containers for Reliable Computing

    Idiomatic and Reproducible Software Builds Using Containers for Reliable Computing

    Master’s Thesis Idiomatic and Reproducible Software Builds using Containers for Reliable Computing Jonas Weber April 18, 2016 arXiv:1702.02999v1 [cs.SE] 9 Feb 2017 Albert-Ludwigs-Universität Freiburg Faculty of Engineering Department of Computer Science Bioinformatics Eingereichte Masterarbeit gemäß den Bestimmungen der Prüfungsordnung der Albert-Ludwidgs-Universität Freiburg für den Studiengang Master of Science (M.Sc.) Informatik vom 19. August 2005. Bearbeitungszeitraum 12. Januar 2016 - 12. Juli 2016 Gutachter Prof. Dr. Rolf Backofen Head of the Group Chair for Bioinformatics Zweitgutachter Prof. Dr. Christoph Scholl Director Chair of Operating Systems Betreuer Dr. Björn Grüning Abstract Containers as the unit of application delivery are the ‘next big thing’ in the software development world. They enable developers to create an executable image containing an application bundled with all its dependencies which a user can run inside a controlled environment with virtualized resources. Complex workflows for business-critical applications and research environments require a high degree of reproducibility which can be accomplished using uniquely identified images as units of computation. It will be shown in this thesis that the most widely used approaches to create an image from pre-existing software or from source code lack the ability to provide idiomaticity in their use of the technology as well as proper reproducibility safe-guards. In the first part, existing approaches are formalized and discussed and a new approach is introduced. The approaches are then evaluated using a suite of three different examples. This thesis provides a framework for formalizing operations involving a layered file system, containers and images, and a novel approach to the creation of images using utility containers and layer donning fulfilling the idiomaticity and reproducibility criteria.
  • Perfect Programmers

    Perfect Programmers

    TYTUŁHOW DOKUMENTU WE WORK AsdasdaAsdasda asdasdad asdasdasd TABLE OF CONTENTS Collaboration in a nutshell................................................................................................................................. 3 First contact.......................................................................................................................................................... 3 Prepare for development.................................................................................................................................... 4 Development......................................................................................................................................................... 5 Development teams............................................................................................................................................. 5 Agreements........................................................................................................................................................... 6 Estimates............................................................................................................................................................... 6 FA ......................................................................................................................................................................... 6 Do I have to pa" for bugs?.................................................................................................................................
  • Software Engineer (Python)

    Software Engineer (Python)

    Software Engineer (Python) Your Mission As a developer at Criteo you will be working to improve the reliability and performance of our services. You will be part of a team of 5 developers and work side by side with our SRE, Network, Hosting and development teams to design and build the next generation of asset management and provisioning tools. Through this unique component we want to provide a portal and an API to manage all our servers. Offer cutting-edge infrastructure services to internal customers : on-demand resource allocation, automatic configuration, self-registration to service discovery and inventory. Either by developing new components or features, or by integrating open sources technologies (Ansible, Chef, Capistrano, Nagios, Centreon, Hadoop & more). Develop open source projects. Because we are working at the forefront of technology, we are dealing with problems that few have faced. We’re big users of open source, and we’d like to give back to the community. Minimum Skills and Qualifications We’re looking for talented Python developers but moreover we’re looking for people who can adapt very quickly and with an open mind. Our engineers chose and use the best tool for the job. You love algorithms and infrastructures, work on personal projects, are curious and stay up-to-date with technologies evolution. You are a problem solver, a fixer, and a creative technologist. We believe coding is a talent and a passion, not just a skill. You need to be a great team worker and a great communicator. Previous experience of agile methodologies (we love Scrum !) Bonus JavaScript, AngularJS Ruby For more information about Criteo’s R&D, please visit http://www.criteolabs.com .
  • Kids College (KIDS) 1

    Kids College (KIDS) 1

    Kids College (KIDS) 1 KIDS 108B KIDS COLLEGE (KIDS) A Touch of Spanish Ages 8-12 KIDS 098 This class offers an introductory program that makes Spanish acquisition Let's Learn Spanish Camp a natural, personalized, enjoyable and rewarding experience. Students will Ages 5-7 learn to read and write basic Spanish and will be introduced to speaking This camp will introduce children to Spanish as a communication skills. first level Spanish. (Fee-Based) They will learn pronunciation, greetings, numbers, music, customs and KIDS 113 songs! (Fee-Based) English Rocks KIDS 099 Ages 5-7 Let's Learn Spanish Camp This course is a fun way to learn English. Class will include colors, shapes, Age 8-12 numbers, phonics, alphabet, spelling, songs, poems, and simple stories. This camp will introduce students to Spanish as a communication skill. (Fee-Based) They will learn pronunciation, greetings, numbers, music, customs and KIDS 115 songs! (Fee-Based) Fun With Phonics KIDS 101 Grades 1-2 Kindergarten Readiness and Review For children just learning to read or having difficulty reading. Sounds are Ages 4-6 taught in a fun but learning atmosphere through stories and writing. (Fee- For children entering kindergarten within the next year or students needing Based) review of current kindergarten skills. Have your child be at the top of KIDS 117 his class and join us for fun and learning. Children will be exposed to Tutorial: Reading and Comprehension math, reading readiness and other school experiences. Each session uses Grades 3-4 different work. (Fee-Based) Students will learn to read more effectively, enhance their reading KIDS 102 comprehension, interpretation and vocabulary skills.
  • Evaluating Web Development Frameworks: Django, Ruby on Rails and Cakephp

    Evaluating Web Development Frameworks: Django, Ruby on Rails and Cakephp

    Evaluating web development frameworks: Django, Ruby on Rails and CakePHP Julia Plekhanova Temple University © September 2009 Institute for Business and Information Technology Fox School of Business Temple University The IBIT Report © 2009 Institute for Business and Information Technology, Bruce Fadem Fox School of Business, Temple University, Philadelphia, PA Editor-in-chief 19122, USA. All rights reserved. ISSN 1938-1271. Retired VP and CIO, Wyeth The IBIT Report is a publication for the members of the Fox Munir Mandviwalla School’s Institute for Business and Information Technology. IBIT reports are written for industry and based on rigorous Editor academic research and vendor neutral analysis. For additional Associate Professor and Executive Director reports, please visit our website at http://ibit.temple.edu. Fox School of Business, Temple University No part of this publication may be reproduced, stored in a Laurel Miller retrieval system or transmitted in any form or by any means, Managing Editor electronic, mechanical, photocopying, recording, scanning Director, Fox School of Business, Temple University or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without the prior written permission of the Publisher. Requests to the Publisher Board of editors for permission should be addressed to Institute for Business and Information Technology, Fox School of Business, Temple Andrea Anania University, 1810 N. 13th Street, Philadelphia, PA 19122, Retired VP and CIO, CIGNA USA, 215.204.5642, or [email protected]. Jonathan A. Brassington Disclaimer: The conclusions and statements of this report Founding Partner and CEO are solely the work of the authors. They do not represent LiquidHub Inc.
  • Chef@GSI Revisited

    Chef@GSI Revisited

    Chef@GSI revisited Christopher Huhn, Dennis Klein GSI Darmstadt [email protected], [email protected] Hepix Spring 2016, Desy Zeuthen Hepix Vancouver 2011 „Evolution of Configuration Management at GSI“ Hepix Spring 2016, Desy Zeuthen 2 21/04/2016 Status 2011 • All Linux nodes managed with Cfengine 2 • Admin team: 3 + 2 • Only basic setup, service config out-of-band (complex shell scripts) • Scalability? Future requirements? • number of nodes • number of services • overall complexity of the infrastructure • number of co-admins • Devops! Chef testbed on the then new batchfarm (Gridengine) Hepix Spring 2016, Desy Zeuthen 3 21/04/2016 Wishlist 2011 • Ad-hoc (push) operations • Software engineering magic • Extensible configuration language, API, • Knowledge Management abstraction level higher than 'editfiles' • Why has node X role Y? • Completely create configuration for a • Where does feature X come from? complex service inside the CMS • Automated Documentation • Using cluster inventory information • Automatic quality assurance and • Automatically assign roles to emerging nodes tests: • • By hostname, subnet, static assignment Coverage, Unit tests, Integration tests, Continuous Integration • Admin roles and multi-tenancy, partitioning • API, Interfaces with other systems • No global root for every admin (Issue tracker, Hardware DB, etc.) Hepix Spring 2016, Desy Zeuthen 4 21/04/2016 Hepix Spring 2016, Desy Zeuthen 5 21/04/2016 Chef status 2016 • Chef is serious business • Old and new competitors: • In use at Facebook, Yahoo, • Puppet Etsy, … • Cfengine
  • What's New with Chef 11

    What's New with Chef 11

    What’s New with Chef 11 Nathen Harvey @nathenharvey What's New Since Chef 0.10? • Windows support • why-run • Librarian • output formatters • Spiceweasel • Test Kitchen • Foodcritic • Solaris support • Cookbooks in separate • Chef Developer Summit(2!) repositories • Berkshelf Private Chef • • ChefConf Food Fight podcast • • Fauxhai knife plugins for • docs.opscode.com openstack, hp, azure, gce • Hangouts for reviews • 0.10 renumbered to Chef 10 • nyan-cat output formatter • full-stack client • What is NOT in Chef 11 • CouchDB • Ruby-based chef-server-api • depsolver, gecode, treetop • Merb • OpenID support in Web UI • Migration tooling (coming soon) http://trainweb.org/carl/Pullman150/IMG_3380.jpg What’s NEW in Chef 11 • PostgreSQL • erchef • nginx • Rails • bookshelf • omnibus-chef server What’s NEW in Chef 11 • chef-apply • partial search • users with key pairs • partials in templates • knife-essentials Erchef + SQL RDBMS nginx erchef bookshelf chef-webui RabbitMQ Chef Server chef-expander solr filesystem (lucene) cookbook PostgreSQL store filesystem search index erchef chef_wm Erchef chef_objects chef_authn chef_index chef_db bookshelf solr RabbitMQ PostgreSQL Erchef • Erlang • Concurrent, fault tolerant • Distributed systems • Complete Chef Server API rewrite Let's see some graphs! CPU Usage on Chef Server • Erchef + Ruby Chef • 12 mo old • 3 CPU aggregate CouchDB Uptime - cron'd restarts Database system CPU Usage CouchDB MySQL Database system Memory usage CouchDB MySQL Database system Load Average CouchDB MySQL Of course, we all know