Analytics Lens AWS Well-Architected Framework Analytics Lens AWS Well-Architected Framework Analytics Lens: AWS Well-Architected Framework Copyright © Amazon Web Services, Inc. and/or its affiliates. All rights reserved. Amazon's trademarks and trade dress may not be used in connection with any product or service that is not Amazon's, in any manner that is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored by Amazon. Analytics Lens AWS Well-Architected Framework Table of Contents Abstract ............................................................................................................................................ 1 Abstract .................................................................................................................................... 1 Introduction ...................................................................................................................................... 2 Definitions ................................................................................................................................. 2 Data Ingestion Layer ........................................................................................................... 2 Data Access and Security Layer ............................................................................................ 3 Catalog and Search Layer .................................................................................................... 3 Central Storage Layer ......................................................................................................... 4 Processing and Analytics Layer ............................................................................................ 5 User Access and Interface Layer ........................................................................................... 5 General Design Principles .................................................................................................................... 6 Scenarios ........................................................................................................................................... 7 Data Lake .................................................................................................................................. 7 Characteristics ................................................................................................................... 7 Reference Architecture ........................................................................................................ 8 Configuration Notes .......................................................................................................... 10 Batch Data Processing ............................................................................................................... 10 Characteristics .................................................................................................................. 11 Reference Architecture ...................................................................................................... 12 Configuration Notes .......................................................................................................... 13 Streaming Ingest and Stream Processing ..................................................................................... 14 Characteristics .................................................................................................................. 14 Reference Architecture ...................................................................................................... 15 Configuration Notes .......................................................................................................... 16 Lambda Architecture ................................................................................................................. 17 Characteristics .................................................................................................................. 17 Reference Architecture ...................................................................................................... 18 Configuration notes: ......................................................................................................... 19 Data Science ............................................................................................................................ 19 Characteristics .................................................................................................................. 20 Reference Architecture ...................................................................................................... 21 Configuration notes .......................................................................................................... 21 Multi-tenant Analytics ............................................................................................................... 22 Characteristics .................................................................................................................. 22 Reference Architecture ...................................................................................................... 23 Configuration Notes: ......................................................................................................... 24 The Pillars of the Well-Architected Framework ..................................................................................... 25 Operational Excellence Pillar ...................................................................................................... 25 Design Principles .............................................................................................................. 25 Definition ........................................................................................................................ 25 Best Practices .................................................................................................................. 26 Resources ........................................................................................................................ 29 Security Pillar .......................................................................................................................... 29 Definition ........................................................................................................................ 29 Design Principles .............................................................................................................. 30 Best Practices .................................................................................................................. 30 Resources ........................................................................................................................ 36 Reliability Pillar ........................................................................................................................ 37 Definition ........................................................................................................................ 37 Design Principles .............................................................................................................. 37 Best Practices .................................................................................................................. 38 Resources ........................................................................................................................ 39 Performance Efficiency Pillar ..................................................................................................... 40 Definition ........................................................................................................................ 40 iii Analytics Lens AWS Well-Architected Framework Design Principles .............................................................................................................. 40 Best Practices .................................................................................................................. 41 Resources ........................................................................................................................ 43 Cost Optimization Pillar ............................................................................................................ 44 Definition ........................................................................................................................ 44 Design Principles .............................................................................................................. 44 Best Practices .................................................................................................................. 45 Resources ........................................................................................................................ 51 Conclusion ....................................................................................................................................... 52 Contributors .................................................................................................................................... 53 Further Reading ............................................................................................................................... 54 Document Revisions .........................................................................................................................