Amazon Managed Service for Prometheus User Guide Amazon Managed Service for Prometheus User Guide Amazon Managed Service for Prometheus: User Guide Copyright © Amazon Web Services, Inc. and/or its affiliates. All rights reserved. Amazon's trademarks and trade dress may not be used in connection with any product or service that is not Amazon's, in any manner that is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored by Amazon. Amazon Managed Service for Prometheus User Guide Table of Contents What is Amazon Managed Service for Prometheus? ................................................................................ 1 Preview release limitations .......................................................................................................... 1 Setting up ......................................................................................................................................... 3 Get an AWS account and your root user credentials ........................................................................ 3 Creating an IAM user .................................................................................................................. 3 Signing in as an IAM user ........................................................................................................... 4 Creating IAM user access keys ...................................................................................................... 5 Getting started .................................................................................................................................. 6 Create a workspace .................................................................................................................... 6 Ingest Prometheus metrics to the workspace ................................................................................. 7 Secure the ingestion of your metrics .................................................................................... 7 Ingestion methods and how to set them up .......................................................................... 8 Send high-availability data from your Kubernetes cluster (Deduplication) ................................. 21 Understanding your Prometheus server output .................................................................... 21 Query your Prometheus metrics ................................................................................................. 22 Securing your metric queries ............................................................................................. 22 Set up Grafana open source or Grafana Enterprise for use with AMP ....................................... 23 Query using Grafana running in an Amazon EKS cluster ........................................................ 25 Query using Prometheus-compatible APIs ........................................................................... 28 Managing workspaces ....................................................................................................................... 29 Create a workspace .................................................................................................................. 29 Edit a workspace ...................................................................................................................... 30 Find your workspace ARN .......................................................................................................... 30 Delete a workspace .................................................................................................................. 31 Security ........................................................................................................................................... 32 Data protection ........................................................................................................................ 33 Data collected by AMP ...................................................................................................... 33 Identity and Access Management ............................................................................................... 34 Audience ......................................................................................................................... 34 Authenticating with identities ............................................................................................ 34 Managing access using policies .......................................................................................... 36 How Amazon Managed Service for Prometheus works with IAM ............................................. 38 Identity-based policy examples .......................................................................................... 42 Troubleshooting ............................................................................................................... 44 IAM permissions and policies ..................................................................................................... 46 AMP permissions .............................................................................................................. 46 Built-in AWS-managed policies ........................................................................................... 46 Sample IAM policies ......................................................................................................... 48 Compliance Validation .............................................................................................................. 48 Resilience ................................................................................................................................ 49 Infrastructure Security .............................................................................................................. 49 Set up IAM roles for service accounts ......................................................................................... 49 Set up service roles for the ingestion of metrics from Amazon EKS clusters .............................. 50 Set up IAM roles for service accounts for the querying of metrics ........................................... 52 Interface VPC endpoints ............................................................................................................ 54 Creating a VPC endpoint for AMP ...................................................................................... 54 CloudTrail logs ......................................................................................................................... 55 AMP information in CloudTrail ........................................................................................... 55 Understanding AMP log file entries .................................................................................... 56 Troubleshooting ............................................................................................................................... 58 429 errors ............................................................................................................................... 58 I see duplicate samples ............................................................................................................. 58 I see an error message related to a limit ..................................................................................... 59 Tagging ........................................................................................................................................... 60 iii Amazon Managed Service for Prometheus User Guide Add a tag to an AMP workspace ................................................................................................ 61 Add a tag to an AMP workspace (console) ........................................................................... 61 Add a tag to an AMP workspace (AWS CLI) .......................................................................... 61 View tags for an AMP workspace ............................................................................................... 62 View tags for an AMP workspace (console) .......................................................................... 62 View tags for an AMP workspace (AWS CLI) ......................................................................... 62 Edit tags for an AMP workspace ................................................................................................. 62 Edit a tag for an AMP workspace (console) .......................................................................... 63 Edit tags for an AMP workspace (AWS CLI) .......................................................................... 63 Remove a tag from an AMP workspace ....................................................................................... 63 Remove a tag from an AMP workspace (console) .................................................................. 63 Remove a tag from an AMP workspace (AWS CLI) ................................................................. 64 Service quotas ................................................................................................................................. 65 Additional quotas on ingested data ............................................................................................ 67 API reference ..................................................................................................................................