Best Practices for Privileged Access Management on Elastic Infrastructure
Total Page:16
File Type:pdf, Size:1020Kb
Best Practices for Privileged Access Management on Elastic Infrastructure BY GRAVITATIONAL Abstract In this guide we go through the basics of SSH access management and discuss the best practices for managing access to modern, elastic infrastructure environments. Table of Contents Introduction 3 Access Management Components Background 3 SSH 3 SSH Public Key Authentication 3 OpenSSH Certificates 6 OpenSSH Certificate authentication 6 LDAP 8 Identity Provider 8 PAM Production Patterns and Anti-Patterns 9 Anti Pattern: Host based authentication 9 Anti Pattern: Trust on first use 10 Anti Pattern: Using password based authentication 11 Anti-Pattern: Shared private key 12 Anti Pattern: Using LDAP with password authentication 13 Anti-Pattern: unknown source of user identity 13 Production Pattern: Using LDAP to setup trusted user and CA keys 13 Production pattern: Identity providers 14 Production pattern: Second Factor Auth with Identity providers 14 Anti Pattern: Local or poorly managed second factor 15 Production Pattern: Short lived user OpenSSH certificates 15 Production Pattern: Host OpenSSH Certificates 16 Anti-pattern: Forever certificates 18 Production Pattern: Certificate authority rotation 18 Production pattern: Centralized discovery 19 Production pattern: Role based access control 19 Production pattern: Centralized audit logging 20 Production pattern: Anomaly Detection and Alerting 21 Production pattern: Rate limiting and user locking 21 Anti-pattern: Intercepting interactive commands 22 Production pattern: Session capture 22 Anti-Pattern: Non-segmented, publicly accessible network 22 Production Pattern: Bastion servers 23 Anti-pattern: Source IP based access 23 Anti Pattern: Weak SSH and TLS setups 24 © Gravitational, Inc. All Rights Reserved [email protected] | 855-867-2538 1 Production pattern: Idle timeouts 24 Production pattern: outbound reverse tunnels 24 Production pattern: Inter-org trust 25 Conclusion 27 © Gravitational, Inc. All Rights Reserved [email protected] | 855-867-2538 2 Introduction This guide addresses how to manage access to modern server fleets. Today, organizations are dealing with elastic infrastructure that includes thousands of servers with VMs that are launched and deleted every hour. In addition, the people that need to access the infrastructure may come and go in the organization and their roles may change while they are at the organization. This makes it difficult to implement a scalable system of Privileged Access Management (“PAM”) to the IT infrastructure. This guide does not attempt to be a complete overview of the PAM landscape and omits many topics such as Kerberos, SSSD and GSS-API. Instead, it focuses on patterns and anti-patterns that have we have seen implemented by system administrators building access management on top of OpenSSH systems, while trying to adopt to the new regulatory and scalability requirements. We adopted many of the SSH infrastructure patterns mentioned here while building Teleport, a modern SSH server which “manages privileged access to elastic infrastructure, without getting in the way.” Background of Access Management Components Before we dive into the details of the PAM and SSH patterns and anti-patterns, let's cover key components that are used as building blocks of privileged access management and referenced throughout this guide. SSH SSH stands for Secure Shell - it's a cryptographic network protocol widely adopted in the internet to access servers using remote logins. Access is done via SSH client that dials in to SSH server. SSH Public Key Authentication SSH keys are based on Public Key Cryptography. The public key is distributed openly, and anyone holding the public key can encrypt data. However, only the owner of private key can decrypt it. With this method, a user owns and never shares the SSH private key and SSH servers are set up to trust the user identity of the users who can prove that the trusted private keys. © Gravitational, Inc. All Rights Reserved [email protected] | 855-867-2538 3 In addition to encryption, private keys can be used to sign any message that can be verified by using only the public key. Public key signatures are used in SSH public key authentication. You’ll notice that it's not necessary to send the plaintext that was signed alongside the signature assuming both parties already know the plaintext that was pre-negotiated. Here are more detailed steps of public key authentication: © Gravitational, Inc. All Rights Reserved [email protected] | 855-867-2538 4 * Client sends SSH login username, public key corresponding to the public key and public key algorithm name. This is done to check if authentication method is supported by the server. * Client uses the private key to sign a message composed of a unique session id of the SSH connection and a public key and sends it as an authentication request. * Server looks up the public key in the `authorized_keys` file and verifies the signature based on the session id and public key of the client and, if the signature matches, accepts the authentication request. You can read more here. OpenSSH Certificates The OpenSSH certificate is special extension built by the OpenSSH team. OpenSSH certificate are built using public keys. The OpenSSH certificate authority is a special trusted party that holds its own public and private key pair. OpenSSH CA keys are not used to authenticate, but to sign SSH certificates. An OpenSSH certificate consists of a collection of fields signed by the certificate authority: * Nonce is a unique random string that is used to prevent signature collision attacks. * Public Key is public part of user or host key-pair that identifies a user or a host. * Type identifies user or host certificate. * Key ID identifies the user or a host in log messages. * Valid principals is a list of usernames or hostnames. © Gravitational, Inc. All Rights Reserved [email protected] | 855-867-2538 5 * Valid after and Valid before are timestamps that define starting time and expiration of the certificate. * Critical Options is a set of options requested by client that should be supported and turned on by the server. * Extensions is a set of optional SSH extensions that could be ignored by the server. * Signature key is a public key of OpenSSH certificate authority used to sign the certificate with its private key. OpenSSH Certificate authentication OpenSSH certificate authentication extends public-key based authentication and uses the same protocol messages: © Gravitational, Inc. All Rights Reserved [email protected] | 855-867-2538 6 * Client sends SSH login username, ssh certificate and public key algorithm name. This is done to check if authentication method is supported by the server. * Alice uses her private key to sign a message composed of a unique session id of the SSH connection and a certificate and sends it as an authentication request. * Server first checks that certificate is signed by the trusted CA by checking if public key of the certificate is in authorized_keys file. * Server then verifies the signature of the certificate sent by Alice to make sure the certificate was not tampered. * Server verifies that certificate is not expired and is not too early to use by checking expiration and starting dates against local server time. * If all checks above passed, server then can assume that this Alice's certificate and public keys are trusted and verifies the signature sent by Alice based on the session id and public key of Alice and, if the signature matches, accepts the authentication request. LDAP LDAP is a directory access protocol widely used in the enterprise server deployments. LDAP servers provide information about user and group membership and sometimes are used to store users passwords and SSH public keys. Identity Provider Identity provider (IdP) is a system that manages centralized identity for all principals (users or computers). Modern identity providers are used to set up a SSO (single sign on) service for many computer systems ranging from office software to SSH computer systems. © Gravitational, Inc. All Rights Reserved [email protected] | 855-867-2538 7 Identity providers are widely used in the enterprise and getting more traction in smaller teams because they allow organizations to provide internal authentication service and different applications can offload the process of authentication to the centralized service. Examples of providers are Okta, Auth0, ADFS and there are many others. There are special protocols that allow applications to integrate between identity providers. In our experience, the most widely used one is SAML 2.0. However, OIDC is getting more traction, as it provides easier authentication flows designed for mobile applications. PAM Production Patterns and Anti-Patterns With those components as a background, we’ll now go through some best practices for PAM. “Patterns” are methods we have seen that are useful in production and sometimes necessary to pass compliance audits. “Anti-patterns” are generally things you want to avoid in production as they can lead to problems at scale. While some of the methods listed here as anti-patterns could be a good solution for small scale server deployments, they will quickly fall apart with larger clusters and organizations. Sometimes an anti-pattern should work well in theory, however the practical implementation is usually problematic. © Gravitational, Inc. All Rights Reserved [email protected] | 855-867-2538 8 Anti Pattern: Host based authentication To be honest, we have never encountered any cases of host based authentication, however here is a warning just in case - avoid using it in any environment. With this authentication method, the client sends signature and public key signed by the host, not the user. The achilles heel of this approach is the behavior of a compromised host: * A compromised host can impersonate other hosts in the system, so servers have to add extra checks over insecure network that source address and hostname matches, which in many cases, is impossible. * Users on compromised hosts can impersonate almost any other user allowed to login from this host.