Hubblestack Documentation Release 2016.7.1
Total Page:16
File Type:pdf, Size:1020Kb
HubbleStack Documentation Release 2016.7.1 Christer Edwards Nov 10, 2016 Contents 1 Components 3 i ii HubbleStack Documentation, Release 2016.7.1 Welcome to the HubbleStack documentation! Hubble is a modular, open-source security compliance framework built on top of SaltStack. The project provides on- demand profile-based auditing, real-time security event notifications, automated remediation, alerting and reporting. Hubble can “dock” with any existing SaltStack installation, and requires very little work to get started. This document describes installation, configuration and general use. Contents 1 HubbleStack Documentation, Release 2016.7.1 2 Contents CHAPTER 1 Components Hubble is made up of four different components, each playing a role in the overall auditing of your systems. These components are described here: • Nova - Nova is Hubble’s profile-based auditing engine. • Pulsar - Pulsar is Hubble’s real-time event system. • Nebula- Nebula is Hubble’s security snapshot utility. • Quasar - Quasar is Hubble’s flexible reporting suite. Each of these components are modular, flexible, and easy to drop into place for any size infrastructure. While each of these components can be used standalone it is often required to combine each components with it’s corresponding Quasar module. Quasar modules are what connects Nova, Nebula and Pulsar to external endpoints such as Splunk, Slack, etc. New to HubbleStack? Explore some of these topics: 1.1 Nova Nova is the best place to get started with Hubble. Using pre-built security and compliance “profiles”, Nova will give you a complete picture of your security stance. Check out the installation docs: • Package Installation (stable) • Manual Installation (develop) Have a look at the Nova module list, and learn how audit modules work. • Nova Modules ... or read through some of the pre-built profiles: • Nova Profiles Tip: Once you have Nova installed, check out Quasar next. 3 HubbleStack Documentation, Release 2016.7.1 1.2 Nebula See also: Nebula has a hard dependency on osquery. See install requirements here https://osquery.io/downloads/ Nebula allows you to take snapshots of your systems by scheduling specific queries. These queries capture information such as: • running processes • established outbound connections • listening processes • suid binaries • crontab • installed packages • ...anything else you’d like to query Check out the installation docs: • Package Installation (stable) • Manual Installation (develop) Have a look at the Nebula modules: • Nebula Modules. Tip: Once you have Nebula installed, checkout Quasar next. 1.3 Pulsar See also: Pulsar has a dependency on the Python pyinotify library. See: Pulsar Required Packages Pulsar watches for filesystem events as they happen and notify you in real-time regarding any changes. • Package Installation (stable) • Manual Installation (develop) You can also take a look at the Pulsar module list: • Pulsar Modules Tip: Next step? Check out the Quasar modules to collect Pulsar event data. 4 Chapter 1. Components HubbleStack Documentation, Release 2016.7.1 1.4 Quasar Quasar modules are integral in collecting and tracking your security data. In general you’ll want to combine each HubbleStack component (Nova, Pulsar, Nebula) with it’s corresponding Quasar module. • Package Installation (stable) • Manual Installation (develop) You can also take a look at the Pulsar module list: • Quasar Modules 1.4.1 Components Nova Nova Introduction Nova is designed to audit the compliance and security level of a system. It is composed of multiple modules, which ingest YAML configuration profiles to run a single or series of audits against a system. Two different installation methods are outlined below. The first method is more stable (and therefore recommended). This method uses Salt’s package manager to track versioned, packaged updates to Hubble’s components. The second method installs directly from git. It should be considered bleeding edge and possibly unstable. Installation Each of the four HubbleStack components have been packaged for use with Salt’s Package Manager (SPM). Note that all SPM installation commands should be done on the Salt Master. Required Configuration Salt’s Package Manager (SPM) installs files into /srv/spm/{salt,pillar}. Ensure that this path is defined in your Salt Master’s file_roots: file_roots: - /srv/salt - /srv/spm/salt Note: This should be the default value. To verify run: salt-call config.get file_roots Tip: Remember to restart the Salt Master after making this change to the configuration. 1.4. Quasar 5 HubbleStack Documentation, Release 2016.7.1 Installation (Packages) Installation is as easy as downloading and installing packages. (Note: in future releases you’ll be able to subscribe directly to our HubbleStack SPM repo for updates and bugfixes!) Nova packages have been divided into modules and profiles. This way we can iterate policy changes separate from the code. Nova Modules wget https://spm.hubblestack.io/nova/hubblestack_nova-2016.10.2-1.spm spm local install hubblestack_nova-2016.10.2-1.spm Nova Profiles wget https://spm.hubblestack.io/nova/hubblestack_nova_profiles-20161101-1.spm spm local install hubblestack_nova_profiles-20161101-1.spm You should now be able to sync the new modules to your minion(s) using the sync_modules Salt utility: salt \* saltutil.sync_modules Once these modules are synced you are ready to run a HubbleStack Nova audit. Skip to Usage. Installation (Manual) Place _modules/hubble.py into your salt/_modules/ directory, and sync it to the minions. git clone https://github.com/hubblestack/nova.git hubblestack-nova.git cd hubblestack-nova.git mkdir -p /srv/salt/_modules/ cp _modules/hubble.py /srv/salt/_modules/ cp -a hubblestack_nova_profiles /srv/salt/ cp -a hubblestack_nova /srv/salt/ salt \* saltutil.sync_modules salt \* hubble.sync Installation (GitFS) This installation method subscribes directly to our GitHub repository, pinning to a tag or branch. This method requires no package installation or manual checkouts. Requirements: GitFS support on your Salt Master. /etc/salt/master.d/hubblestack-nova.conf gitfs_remotes: - https://github.com/hubblestack/nova: - base: v2016.10.2 Tip: Remember to restart the Salt Master after applying this change. 6 Chapter 1. Components HubbleStack Documentation, Release 2016.7.1 Skip to Usage. Usage There are four primary functions in the hubble.py module: 1. hubble.sync will sync the hubblestack_nova_profiles/ and hubblestack_nova/ directories to the minion(s). 2. hubble.load will load the synced audit modules and their yaml configuration files. 3. hubble.audit will audit the minion(s) using the YAML profile(s) you provide as comma-separated argu- ments 4. hubble.top will audit the minion(s) using the top.nova configuration. hubble.audit takes two optional arguments. The first is a comma-separated list of paths. These paths can be files or directories within the hubblestack_nova_profiles directory. The second argument allows for toggling Nova configuration, such as verbosity, level of detail, etc. If hubble.audit is run without targeting any audit configs or directories, it will instead run hubble.top with no arguments. hubble.audit will return a list of audits which were successful, and a list of audits which failed. Here are some example calls: # Run the cve scanner and the CIS profile: salt \* hubble.audit cve.scan-v2,cis.centos-7-level-1-scored-v1 # Run hubble.top with the default topfile (top.nova) salt \* hubble.top # Run all yaml configs and tags under salt://hubblestack_nova_profiles/foo/ # and salt://hubblestack_nova_profiles/bar, but only run audits with tags # starting with "CIS" salt \* hubble.audit foo,bar tags='CIS*' Nova Topfiles Nova topfiles look very similar to saltstack topfiles, except the top-level key is always nova, as nova doesn’t have environments. nova: '*': - cve.scan-v2 - network.ssh - network.smtp 'web*': - cis.centos-7-level-1-scored-v1 - cis.centos-7-level-2-scored-v1 'G@os_family:debian': - network.ssh - cis.debian-7-level-1-scored:'CIS *' Additionally, all nova topfile matches are compound matches, so you never need to define a match type like you do in saltstack topfiles. 1.4. Quasar 7 HubbleStack Documentation, Release 2016.7.1 Each list item is a string representing the dot-separated location of a yaml file which will be run with hubble.audit. You can also specify a tag glob to use as a filter for just that yaml file, using a colon after the yaml file (turning it into a dictionary). See the last two lines in the yaml above for examples. Examples: salt' *' hubble.top salt' *' hubble.top foo/bar/top.nova salt' *' hubble.top foo/bar.nova verbose=True Compensating Control Configuration In some cases, your organization may want to skip certain audit checks for certain hosts. This is supported via compensating control configuration. You can skip a check globally by adding a control: <reason> key to the check itself. This key should be added at the same level as description and trigger pieces of a check. In this case, the check will never run, and will output under the Controlled results key. Nova also supports separate control profiles, for more fine-grained control using topfiles. You can use a separate YAML top-level key called control. Generally, you’ll put this top-level key inside of a separate YAML file and only include it in the top-data for the hosts for which it is relevant. For these separate control configs, the audits will always run, whether they are controlled or not. However, controlled audits which fail will be converted from Failure to Controlled in a post-processing operation. The control config syntax is as follows: control: - CIS-2.1.4: This is the reason we control the check - some_other_tag: reason: This is the reason we control the check - a_third_tag_with_no_reason Note that providing a reason for the control is optional. Any of the three formats shown in the yaml list above will work. Once you have your compensating control config, just target the yaml to the hosts you want to control using your topfile.