Fabric Management using Open Source Tools

Jason A. Smith RHIC & ATLAS Computing Facility Brookhaven National Lab

Monday, October 18, 2010 Jason A. Smith, RACF @ BNL CHEP 2010, Taipei What is it?

Basic Components: ● Provisioning – Cobbler/Koan ● Asset Management – FusionInventory & GLPI (with OSCInventory-NG) ● Configuration Management – Puppet ● Management – Will be added later on Monday, October 18, 2010 Jason A. Smith, RACF @ BNL CHEP 2010, Taipei Provisioning

● Cobbler/Koan – Written by RedHat & integrated into – Supports: RHEL, SL, CentOS & /Ubuntu – Powerful Cheetah templating language and config/code reuse with “Snippets” – CLI or WebUI (most actions) ● Single ks template used to install most systems – Specify OS version & arch, network (MAC, IP, etc) & template metadata to install base OS, including inventory agent & puppet client

Monday, October 18, 2010 Jason A. Smith, RACF @ BNL CHEP 2010, Taipei Monday, October 18, 2010 Jason A. Smith, RACF @ BNL CHEP 2010, Taipei Configuration Management

● Config mgmt tool with svn for revision control – change management (who changed what, when?) – Svn branches to separate test & production ● Fully automated configuration after provisioning new systems and can even reconfigure an existing system for a new purpose. ● Centralized management and control of all OS & application level configuration.

Monday, October 18, 2010 Jason A. Smith, RACF @ BNL CHEP 2010, Taipei Why do we need it?

● Most systems are configured manually or with custom scripts, and everyone does it differently. – Varied levels shared knowledge & methodology ● Want to know how something is configured? – Document?, may say how something should be, but have to manually log in and look around. ● Mini personal admin domains. – Problematic when primary admin is out. ● Wastes a lot of time and duplication of work. ● No official change management system. Monday, October 18, 2010 Jason A. Smith, RACF @ BNL CHEP 2010, Taipei Considered Several Tools

● Cfengine, puppet, chef, etch, bcfg2, AutomateIt ● Puppet was selected for many reasons: – Simple & powerful DSL (Domain-Specific Lang) – Explicitly declared dependency graphing model

● Provides better deterministic state convergence – Central config catalog & dependency resolution

● Better security, conflict resolution & logic analysis – Web dashboard, GraphViz config visualization – Long history, stable codebase, large user base – Free OpenSource (optional commercial support) Monday, October 18, 2010 Jason A. Smith, RACF @ BNL CHEP 2010, Taipei Why Not Something Else?

● Cfengine: copy/paste methodology, long history resulted in non-uniform config language, minimal reporting except in commercial product ● Chef: ruby-only DSL, procedural ordering only ● Etch: one developer, small user community ● AutomateIt: ruby-only DSL, no central server (must use NFS or SVN to distribute config) ● Bcfg2: Combined XML config (good for computers, bad for humans), python scripts & plain text config files, small user base Monday, October 18, 2010 Jason A. Smith, RACF @ BNL CHEP 2010, Taipei Asset Management

● FusionInventory & GLPI plugin – Collects server inventory & SNMP data from network devices, like switches. ● OCSInventory-NG – Relays data from assets to GLPI

● Won't be necessary in future versions. ● GLPI with OCS Import & Custom Field plugins – Enterprise quality IRM (Info Resource Manager) – Nearly 100 plugins to support additional features

Monday, October 18, 2010 Jason A. Smith, RACF @ BNL CHEP 2010, Taipei Monday, October 18, 2010 Jason A. Smith, RACF @ BNL CHEP 2010, Taipei Puppet & GLPI

● Write well organized modules & classes for the OS and Software configuration ● Use puppet's external node classifier option – Configure puppet classes in GLPI CF, then our external node classifier script queries GLPI's MySQL DB to tell puppet which class manifests to apply to each server.

● YAML output (JSON in future?) – Cobbler and puppet's own dashboard can also be used as external node classifiers.

Monday, October 18, 2010 Jason A. Smith, RACF @ BNL CHEP 2010, Taipei Future

● Add RackTables to provide better physical rack location display (lacking in GLPI) – Data synced from GLPI ● Add Switches and other SNMP into GLPI ● Finish Puppet manifests & setup dashboard ● Integrate Nagios (Puppet's exported resources) ● Integrate Virtualization Management – RHEV, ConVirt, oVirt, openQRM, etc.

Monday, October 18, 2010 Jason A. Smith, RACF @ BNL CHEP 2010, Taipei References & Links

● Cobbler: https://fedorahosted.org/cobbler/ ● Puppet: http://www.puppetlabs.com/ ● FusionInventory: http://fusioninventory.org/ ● GLPI: http://www.glpi-project.org/ ● OCSInventory: http://www.ocsinventory-ng.org/ ● RackTables: http://racktables.org/

Monday, October 18, 2010 Jason A. Smith, RACF @ BNL CHEP 2010, Taipei