Linters Hooks Pipelines Automation to Save Your Bacon Elliot Jordan End-User Platform Security Nordstrom “I’m not really a software developer. I just think I’m a software developer because I develop software.” — Arjen van Bochoven ‣ Package sources ‣ Scripts and extension plist, yaml, json, shell, python attributes ‣ AutoPkg recipes/ shell, python overrides ‣ MDM profiles plist, shell, python plist ‣ Munki repos ‣ Documentation plist, python, shell text, markdown, reStructuredText Mac Software "Dev Ops" Admin Developer Reducing errors Streamlining development Automating tedious tasks Ground Rules Protected "master" branch Peer review Remote Git hosting Production code in Git Code standards Linters Linters Linters Linters Linters Linters Linters Atom + Shellcheck Linters Atom + Shellcheck Terminal $ brew install shellcheck ==> Downloading https://homebrew.bintray.com/bottles/ shellcheck-0.6.0_1.mojave.bottle.tar.gz ==> Pouring shellcheck-0.6.0_1.mojave.bottle.tar.gz ! /usr/local/Cellar/shellcheck/0.6.0_1: 8 files, 7.2MB $ which shellcheck /usr/local/bin/shellcheck ⌘C $ Linters Atom + Shellcheck Linters Atom + Shellcheck Linters Atom + Shellcheck Linters Atom + Shellcheck Linters Atom + Shellcheck Linters Atom + Shellcheck Linters Atom + Shellcheck ⌘V Linters Atom + Shellcheck Linters Atom + Shellcheck Linters Atom + Shellcheck Linters Atom + Shellcheck Click to learn more! Linters Atom + Shellcheck Linters Atom + Shellcheck Typo caught Linters Atom + Shellcheck Suggestions for improving resiliency Linters Atom + Shellcheck Deprecated syntax callouts Linters Atom + Shellcheck Generally accepted practices Linters Atom + Shellcheck Useless cat! Photo: Byron Chin Linters Atom + Shellcheck Linters Atom + Shellcheck Linters Shellcheck + zsh ‣ Shellcheck doesn't currently support zsh However: ‣ That might change someday ‣ Bash scripts should work fine in Catalina Linters Atom + Pylint Linters Atom + Pylint Linters Atom + Pylint Linters Atom + Pylint Linters Atom + Pylint Linters Atom + Pylint Linters Atom + Pylint Linters Atom + Pylint Linters Atom + Pylint Linters Atom + Pylint Linters Fine-Tuning Linters E0611 Linters Fine-Tuning Linters E0611 Linters Fine-Tuning Linters # pylint: disable=E0611 Linters Fine-Tuning Linters # pylint: disable=no-name-in-module Terminal $ pylint example2.py No config file found, using default configuration ************* Module example2 C: 1, 0: Missing module docstring (missing-docstring) E: 3, 0: No name 'CFPreferencesCopyAppValue' in module 'CoreFoundation' (no-name-in-module) C: 6, 0: Constant name "munki_repo" doesn't conform to UPPER_CASE naming style (invalid-name) ---------------------------------------------------------------------- Your code has been rated at -13.33/10 (previous run: -13.33/10, +0.00) $ Linters Fine-Tuning Linters # pylint: disable=no-name-in-module # pylint: enable=no-name-in-module Linters Fine-Tuning Linters # pylint: disable=E0611 Linters Fine-Tuning Linters Linters Fine-Tuning Linters # shellcheck disable=SC2115 Linters Fine-Tuning Linters # shellcheck disable=SC2115 Linters Linter Limitations ‣ Installed per-app and per-Mac ‣ Not easily distributed across a team ‣ Suggestions are optional Linters Autoformatters ‣ Python black, yapf, autopep8, isort ‣ Go gofmt ‣ Ruby rubocop Linters Python Black Linters Python Black Linters Python Black Linters Python Black Linters Fine-Tuning Autoformatters # fmt: off # fmt: on Linters When to Avoid Autoformatters ‣ Submitting a very small change to a repo ‣ Contributing to a new repo for the first time ‣ If the maintainers strongly prefer their own style Linters Autoformatter Limitations ‣ You may not agree with the style choices (but you get used to it) ‣ Can be quite jarring to convert a project to use autoformatters for the first time (large diffs) ‣ Not easily distributable across team without help from other frameworks Linters Hooks Hooks git commit -a -m "My great commit" fix the issue and try again pre-commit hook(s) exit exit nonzero zero commit fails commit succeeds Linters Hooks Hooks git commit -a -m "My great commit" fix the issue and try again pre-commit hook(s) exit exit nonzero zero commit fails commit succeeds Linters Hooks Hooks pre-commit hook(s) Linters Hooks Installing Pre-Commit Terminal $ brew install pre-commit Updating Homebrew... ==> Downloading https://homebrew.bintray.com/bottles/pre-commit-1.17.0.moja ve.bottle.tar.gz ==> Pouring pre-commit-1.17.0.mojave.bottle.tar.gz ! /usr/local/Cellar/pre-commit/1.17.0: 703 files, 8.9MB $ which pre-commit /usr/local/bin/pre-commit $ pre-commit --version pre-commit 1.17.0 $ Linters Hooks Installing Pre-Commit Terminal $ cd ~/path/to/git_repo $ touch .pre-commit-config.yaml $ open -a Atom .pre-commit-config.yaml $ Linters Hooks Installing Pre-Commit Terminal $ cd ~/path/to/git_repo .pre-commit-config.yaml $ touch .pre-commit-config.yaml repos:$ open -a Atom .pre-commit-config.yaml - repo:$ https://github.com/pre-commit/pre-commit-hooks rev: v2.2.3 hooks: - id: no-commit-to-branch Linters Hooks Installing Pre-Commit Terminal $ cd ~/path/to/git_repo $ touch .pre-commit-config.yaml $ open -a Atom .pre-commit-config.yaml $ pre-commit install pre-commit installed at .git/hooks/pre-commit $ Linters Hooks Installing Pre-Commit Terminal $ git branch * master $ git commit -am "Add pre-commit config" Don't commit to branch....................................................Failed $ git checkout -b pre-commit A .pre-install-config.yaml Switched to a new branch 'pre-commit' $ git commit -am "Add pre-commit config" Don't commit to branch....................................................Passed [pre-commit 48f2745] Add pre-commit config 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 .pre-install-config.yaml $ Linters Hooks Configuring Pre-Commit .pre-commit-config.yaml repos: - repo: https://github.com/pre-commit/pre-commit-hooks rev: v2.2.3 hooks: - id: no-commit-to-branch Linters Hooks Configuring Pre-Commit .pre-commit-config.yaml repos: - repo: https://github.com/pre-commit/pre-commit-hooks rev: v2.2.3 hooks: - id: no-commit-to-branch - id: check-added-large-files args: [--maxkb=100] - id: check-merge-conflict - repo: https://github.com/python/black rev: 19.3b0 hooks: - id: black Linters Hooks Configuring Pre-Commit .pre-commit-config.yaml repos: - repo: https://github.com/pre-commit/pre-commit-hooks rev: v2.2.3 hooks: - id: no-commit-to-branch P - id: check-added-large-files Text Editor Pre-Commit args: [--maxkb=100] - id: check-merge-conflict - repo: https://github.com/python/black rev: 19.3b0 hooks: - id: black Linters Hooks Configuring Pre-Commit .pre-commit-config.yaml repos: - repo: https://github.com/pre-commit/ - repo: https://github.com/ambv/black pre-commit-hooks rev: 19.3b0 rev: v2.2.1 hooks: hooks: - id: black - id: check-added-large-files args: [--maxkb=200] - repo: https://github.com/asottile/ - id: check-byte-order-marker blacken-docs - id: check-case-conflict rev: v0.5.0 - id: check-docstring-first hooks: - id: check-merge-conflict - id: blacken-docs - id: check-symlinks additional_dependencies: - id: check-yaml [black==19.3b0] - id: mixed-line-ending - id: no-commit-to-branch - id: trailing-whitespace args: [--markdown-linebreak-ext=md] Linters Hooks Configuring Pre-Commit .pre-commit-config.yaml repos: - repo: https://github.com/pre-commit/ - repo: https://github.com/ambv/black pre-commit-hooks rev: 19.3b0 rev: v2.2.1 hooks: hooks: - id: black - id: check-added-large-files args: [--maxkb=200] - repo: https://github.com/asottile/ - id: check-byte-order-marker blacken-docs - id: check-case-conflict rev: v0.5.0 - id: check-docstring-first hooks: - id: check-merge-conflict - id: blacken-docs - id: check-symlinks additional_dependencies: - id: check-yaml [black==19.3b0] - id: mixed-line-ending - id: no-commit-to-branch - id: trailing-whitespace args: [--markdown-linebreak-ext=md] Linters Hooks Updating Pre-Commit Hooks Terminal $ pre-commit autoupdate Updating https://github.com/pre-commit/pre-commit-hooks...updating v2.2.0 -> v2.2.3. Updating https://github.com/python/black...[INFO] Initializing environment for https://github.com/python/black. updating stable -> 19.3b0. Updating https://github.com/asottile/blacken-docs...[INFO] Initializing environment for https://github.com/asottile/blacken-docs. updating v1.0.0 -> v1.1.0. $ Linters Hooks Configuring Pre-Commit .pre-commit-config.yaml repos: - repo: https://github.com/pre-commit/ - repo: https://github.com/ambv/black pre-commit-hooks rev: 19.3b0 rev: v2.2.1 hooks: hooks: - id: black - id: check-added-large-files args: [--maxkb=200] - repo: https://github.com/asottile/ - id: check-byte-order-marker blacken-docs - id: check-case-conflict rev: v0.5.0 - id: check-docstring-first hooks: - id: check-merge-conflict - id: blacken-docs - id: check-symlinks additional_dependencies: - id: check-yaml [black==19.3b0] - id: mixed-line-ending - id: no-commit-to-branch - id: trailing-whitespace args: [--markdown-linebreak-ext=md] Linters Hooks Pre-Commit Hooks for Mac Admins https://github.com/homebysix/pre-commit-macadmin Linters Hooks Pre-Commit Hooks for Mac Admins ~/Developer/pkg-sources/.pre-commit-config.yaml repos: - repo: https://github.com/homebysix/pre-commit-macadmin rev: v1.3.0 hooks: - id: check-munkipkg-buildinfo - id: check-outset-scripts - id: check-plists Linters Hooks Pre-Commit Hooks for Mac Admins ~/Developer/pkg-sources/.pre-commit-config.yaml repos: - repo: https://github.com/homebysix/pre-commit-macadmin rev: v1.3.0 hooks: - id: check-munkipkg-buildinfo - id: check-outset-scripts - id: check-plists