Saltstack-Formulas Documentation Release Master
Total Page:16
File Type:pdf, Size:1020Kb
SaltStack-Formulas Documentation Release master salt-formulas contributors Nov 12, 2018 Contents 1 Overview 1 2 Contents 3 2.1 Project Introduction...........................................3 2.2 Development Documentation...................................... 488 2.3 Installation and Operations Manual................................... 525 3 Indices and tables 549 i ii CHAPTER 1 Overview This project provides scalable and reliable IT automation using SaltStack for installing and operating wide variety of services and resources. Project provides standards to define service models and processes with ability to reuse these components in varying contexts. 1 SaltStack-Formulas Documentation, Release master 2 Chapter 1. Overview CHAPTER 2 Contents 2.1 Project Introduction Here you will find documentation relevant to architecture and goals of the project. Existing formula ecosystem and undelying metadata standards. 2.1.1 Overview Chapter 1. Overview Home SaltStack-Formulas Project Introduction Project Objectives • Collateral Goodies Project provides standards to define service models and processes with ability to reuse these components in varying contexts. Metadata model shared accross all services let us explore underlying relationships that ease the management of infrastructures acoross whole life-span. The project has little different objectives compare to official salt-formulas. The general orientation of project may be similar to the official salt formulas but the major differences lie at meta-data model and clear decomposition which being consistent accross all formulas in SaltStack- Formulas project. Collateral Goodies Adhering to the standards allows further services to be declared and configured in dynamic way, consuming metadata of surrounding services. This include following domains: 3 SaltStack-Formulas Documentation, Release master • Dynamic monitoring: Event collecting, telemetry with dashboards, alarms with notifications • Dynamic backup: Data backuping and restoring • Dynamic security: Firewall rules, router configurations • Dynamic documentation, topology visualizations • Dynamic audit profiles and beacons All these can be generated out of your existing infrastructure without need for any further parametrisation. • Documentation Home • Project Introduction • Installation and Operations Manual • Development Documentation Home SaltStack-Formulas Project Introduction Project History • Beginnings • tcp cloud Era • openstack-salt Project • saltstack-formulas Project Beginnings The initial formula structure was created in 2013. The formulas were not even called formulas back then, but states. It was time of great confusion and the quality of newly created salt-formulas was low. tcp cloud Era The majority of formulas were rewritten to current standard structure and were used in production for cloud deploy- ments. All the formulas were open-sourced and support metadata were instroduces in 2015. openstack-salt Project OpenStack-Salt project was OpenStack Big Tent initiative project in 2015/16 and provided resources for installing and operating OpenStack deployments. It used subset of the formulas and project was abandoned when tcp cloud was bought by Mirantis. 4 Chapter 2. Contents SaltStack-Formulas Documentation, Release master saltstack-formulas Project The scope of current project is much wider than management of OpenStack installations and provides generic formula ecosystem capable of managing multiple heterogenous infrastructures. • Documentation Home • Project Introduction • Installation and Operations Manual • Development Documentation Home SaltStack-Formulas Project Introduction Introduction to SaltStack • Pillar Metadata SaltStack-Formulas uses Salt configuration platform to install and manage infrastructures. Salt is an automation plat- form that greatly simplifies system and application deployment. Salt uses service formulas to define resources written in the YAML language that orchestrate the individual parts of system into the working entity. Pillar Metadata Pillar is an interface for Salt designed to offer global values that are distributed to all minions. The ext_pillar option allows for any number of external pillar interfaces to be called to populate the pillar data. Pillars are tree-like structures of data defined on the Salt Master and passed through to the minions. They allow confidential, targeted data to be securely sent only to the relevant minion. Pillar is therefore one of the most important systems when using Salt. • Documentation Home • Project Introduction • Installation and Operations Manual • Development Documentation • Documentation Home • Project Introduction • Installation and Operations Manual • Development Documentation 2.1. Project Introduction 5 SaltStack-Formulas Documentation, Release master 2.1.2 Quick Start Chapter 2. Quick start Home SaltStack-Formulas Project Introduction Deployment Preparation Guidelines • Salt Master Formulas • Salt Master Metadata Let’s consider simple deployment of single configuration node with one application and one database node. • Config node [salt master] • Application node [python app] • Database node [postgres db] To start the simple deployment you need first setup the Salt master. Installation of salt minions on controlled nodes is then very simple. Salt Master Formulas States are delivered by formulas and are stored in /srv/salt/env/<env>/ directory. Environment can be either production [prd] or development [dev]. This directory is correlates with salt_files root for given environment. You can serve multiple environments from single salt master at once, but this setup is not recommentded. Usually production environment formulas are delivered by packages and development environment formulas are de- livered by git sourced formulas. /srv/salt/env/<env>/ |-- service1/ | |-- itit.sls | |-- role1/ | | |-- service.sls | | `-- resource.sls | `-- role2.sls `-- service2/ |-- itit.sls `-- role.sls For example basic linux, python-app and openssh services for development environment in a little shortened version. /srv/salt/env/dev/ |-- linux/ | |-- itit.sls | |-- system/ | | |-- repo.sls | | `-- user.sls | `-- network/ | |-- interface.sls | `-- host.sls (continues on next page) 6 Chapter 2. Contents SaltStack-Formulas Documentation, Release master (continued from previous page) |-- python-app/ | |-- itit.sls | |-- server.sls `-- openssh/ |-- itit.sls |-- server.sls `-- client.sls More about structure and layout of the formulas can be found in Development documentation. Salt Master Metadata Metadata then define what state formulas in given specific context are projected to managed nodes. Following trees shows simple metadata structure for simple python application deployment. Important parameters are cluster_name labeling individual deployments and cluster.domain giving the deployment nodes domain part of the FQDN. /srv/salt/reclass/ |-- classes/ | |-- cluster/ | | `-- deployment/ | | |-- infra/ | | | `-- config.yml | | |-- python_app/ | | | |-- database.yml | | | `-- web.yml | | `-- init.yml | |-- system/ | | |-- python_app/ | | | `-- server/ | | | |-- [dev|prd].yml | | | `-- [single|cluster].yml | | |-- postgresql/ | | | `-- server/ | | | |-- cluster.yml | | | `-- single.yml | | |-- linux/ | | | `-- system/ | | | `-- init.yml | | `-- deployment2.yml | `-- service/ | |-- linux/ [formula metadata] | |-- python-app/ [formula metadata] | `-- openssh/ [formula metadata] `-- nodes/ `-- cfg.cluster.domain.yml You start with defining single node cfg.cluster.domain in nodes directory and that is core node pointing to your clus- ter.deploy.infra.config class. Content of the nodes/cfg.cluster.domain.yml file: classes: - cluster.deploy.infra.config parameters: (continues on next page) 2.1. Project Introduction 7 SaltStack-Formulas Documentation, Release master (continued from previous page) _param: reclass_data_revision: master linux: system: name: cfg01 domain: cluster.domain Contains pointer to class cluster.deploy.infra.config and some basic parameters. Content of the classes/cluster/deploy/infra/config.yml file: classes: - system.openssh.client - system.salt.master.git - system.salt.master.formula.git - system.reclass.storage.salt - cluster.cluster_name parameters: _param: salt_master_base_environment: dev reclass_data_repository: [email protected]:reclass-models/salt-model.git salt_master_environment_repository:"https://github.com/salt-formulas" reclass_data_revision: master reclass_config_master: ${_param:infra_config_deploy_address} single_address: ${_param:infra_config_address} reclass: storage: node: python_app01: name: app01 domain: ${_param:cluster_domain} classes: - cluster.${_param:cluster_name}.python_app.application params: salt_master_host: ${_param:reclass_config_master} single_address: ${_param:python_application_node01_single_address} database_address: ${_param:python_database_node01_single_address} python_dbs01: name: dbs01 domain: ${_param:cluster_domain} classes: - cluster.${_param:cluster_name}.python_app.database params: salt_master_host: ${_param:reclass_config_master} single_address: ${_param:python_database_node01_single_address} More about structure and layout of the metadata can be found in Metadata chapter. • Documentation Home • Project Introduction • Installation and Operations Manual • Development Documentation Home SaltStack-Formulas Project Introduction 8 Chapter 2. Contents SaltStack-Formulas Documentation, Release master Bootstrap Salt-Formulas infrastructure • TL;DR • Quick bootstrap – Bootstrap salt-master – Bootstrap salt-minion • Advanced usage – Additional