Power BI Architecture End-to-End Melissa Coates Coates Data Strategies March 10, 2021 Download slides: CoatesDS.com/Presentations Melissa Coates Data architect | Technical trainer | Consultant Specialist in Power BI governance & administration Microsoft Data Platform MVP Owner of Coates Data Strategies @SQLChick | @CoatesDS Power BI Deployment & Governance Workshop Power BI Data Culture & Architecture Security & Adoption & Governance Center of Excellence Decisions Data Protection Administration Power BI Architecture End-to-End Agenda Overview, Options Other Key Authoring for Components in Tools & Content Delivery the Architecture Data Storage Modes Power BI Service Data Gateway Power BI Report Server Power BI Premium Embedding Dataflows Other Other Where to Download Materials Slides: CoatesDS.com/Presentations Diagram: CoatesDS.com/Diagrams Power BI Overview, Authoring Tools & Data Storage Modes What is Power BI? A suite of tools to support both self-service BI and enterprise BI: • Data acquisition • Calculations • Data preparation & cleansing • Reports & visuals • Data modeling • Content delivery & viewing Power BI is a broad collection of tools, services, and features Data Architecture Components Ingest Prepare Model Serve Access to Data cleansing, Data warehouse Reports original preparation & and/or & source data transformations semantic model dashboards Power BI can be used for some or all of these components Authoring Tools Power BI Authoring Source Query Data Reports & Data Editor Calculations Visuals Relationships Skills: Data Data Data preparation modeling visualization Languages: Power Query Data Analysis Formula eXpressions Language (“M”) (“DAX”) Native Tools for Creating Power BI Content Power BI Desktop (.pbix) + Power BI Desktop for Report Server (.pbix) Power BI Paginated Report Builder (.rdl) Excel (.xlsx) Power BI Service Azure Synapse Studio Visual Studio Code Visual Studio SQL Server Data Tools Tools for Creating Content Native tools intended for self-service authors: Power BI Paginated Power BI Excel Desktop Report Builder Service Native tools intended for more advanced authors: Azure Synapse Visual Studio Visual Studio Studio Code SQL Server Data Tools External tools for advanced dataset authoring & deployment: DAX Tabular ALM + many more Studio Editor Toolkit community tools Data Storage Modes Dataset Storage Modes Data Data stored in remains Power BI in source Data is imported to Power BI’s in- Import memory columnar data storage Queries are passed at execution DirectQuery time to a relational data source Forms of data Queries are passed at execution Live virtualization: time to an Analysis Services model Connection querying or a Power BI dataset the data where it lives Composite A combination of Import, Live Model Connection and/or DirectQuery Terminology Clarifications Live Connection -vs- DirectQuery Conceptually similar in that they both query data which resides elsewhere. Examples: • Live Connection from a report to a Power BI dataset or Azure AS • DirectQuery from a report to an Azure SQL Database Two Types of DirectQuery (1) DirectQuery to (usually) a relational database Standard DirectQuery mode (2) DirectQuery for Power BI datasets & Azure One type of composite model Analysis Services models Data Storage Modes Local model Remote model Data Import Mode Live Connection Mode stored in A PBIX file which contains A PBIX file with reports that Power BI both the schema & the obtain data by querying a data Power BI shared dataset Data DirectQuery Mode Live Connection Mode remains A PBIX file which contains a A PBIX file with reports that in source model that is schema only; obtain data by querying an data is obtained by querying Analysis Services Tabular the data source when report Model is executed Using a Shared Dataset w/ Live Connection Sales Analysis Shared dataset: Dataset Reports make a live connection to the remote model to access data MoM YTD Sales Daily Sales Sales by Sales Revenue Targets Product Reports: Trends Extending & Customizing a Shared Dataset DirectQuery for Power BI Datasets and Azure Analysis Services Organizational Shared Sales Analysis Dataset dataset: Live Connection is converted to a DirectQuery connection Composite Organizational data: model: Departmental DirectQuery to remote model Sales Analysis Dataset Departmental data: Local model Customize & extend organizational model with additional data sources & calculations Options for Content Delivery with Power BI Power BI Service Publishing to the Power BI Service Power BI Service Power BI publish Desktop .pbix file My Workspace view publish Workspace App Excel .xlsx file content A A Paginated publish Workspace App .rdl file Report Builder B B Collaborate on content creation & validation Storing Content in the Power BI Service My Workspace Personal Content & Data Content & Data Workspace Organizational Dashboards, Content & Data reports, datasets, dataflows Viewing Content in the Power BI Service My Workspace Personal BI Content Sharing Distribution & Data Workspace of individual Team collaboration & pieces of Dashboards, small team distribution content reports, datasets, dataflows App Broad content distribution Objects in the Power BI Service Dashboard 1 Dashboard 2 Report 1 Report 2 Report 3 Report 4 Dataset 1 Dataset 2 Workspace Dataflow A Licenses for Using Power BI Service User Licenses: User Free Personal analytics (my workspace) identity Team collaboration and content from Azure Pro distribution (workspaces, apps, sharing) Active Directory PPU Add-on to Pro for most Premium features Core-Based Capacity Licenses (optional): Premium Broad content distribution scenarios + additional features Top Reasons for Using the Power BI Service Collaboration, Sharing & Content Distribution Delivering reports using Multiple methods & security Power BI Desktop on a file options to collaborate with both system is difficult to do well internal and external users based & is not user-friendly on Azure Active Directory Top Reasons for Using the Power BI Service Collaboration, Sharing & Content Distribution Friendler experience Integration with Microsoft Teams: for report consumers full capabilities of Power BI Service with apps + easier to so users can be productive where discover content they already are working Top Reasons for Using the Power BI Service Collaboration, Sharing & Content Distribution Row-level security Use of the Power BI Report & dashboard works mobile app & the comments Windows application Top Reasons for Using the Power BI Service Automation Subscriptions Scheduled Integration with data refresh Power Automate & Power Apps Top Reasons for Using the Power BI Service Centralization & Reusability Centralized dataset Certified & promoted Centralized data reusability across (endorsements) for cleansing with Power numerous reports datasets & dataflows BI dataflows with shared datasets & live connections Top Reasons for Using the Power BI Service Governance, Compliance & Administration Tenant settings to manage Administrator visibility into features & user experience usage & activity logging + administrator access to all for adoption, workspaces to support user administration & support population Top Reasons for Using the Power BI Service Governance, Compliance & Administration Integration with Integration with Microsoft Microsoft 365 information Cloud App Security to protection & sensitivity labels implement data loss prevention policies Power BI Report Server Power BI Report Server (PBIRS) Power BI Power BI collaborate Report Server on content Desktop for publish creation & Report Server .pbix file validation publish Excel .xlsx file view Web Server content Paginated publish Report Server Report Builder .rdl file Report Server TempDB Top Reasons for Using PBIRS Need an alternative to Existing significant Desire a simple folder- the Power BI Service investment in based reporting portal; for delivery of reports Reporting Services a slower release cadence & a subset of feature support is ok Embedding Solutions Embedding Content in a Custom App Power BI Service collaborate on content Power BI publish Workspace creation & .pbix file validation Desktop Report JavaScript + REST APIs Custom App view iFrame content Report No-Code Embedding Power BI Service collaborate on content Power BI publish Workspace creation & .pbix file validation Desktop Report Generate Secure Embed Code SharePoint view iFrame content Report Options for Embedding Content Internal Embedding External Embedding Embed for Embed for Embed for productivity Public (no-code) your org customers (user owns data) (app owns data) URL Embedding Embedding Embedding REST APIs: REST APIs: Publish to web: for internal websites in in Custom portals ISV apps and public portals, open and portals Teams SharePoint and custom apps services data initiatives Secured by organizational Azure AD Secured by No user application authentication authentication Image adapted from: Planning a Power BI Deployment Whitepaper Using Other Tools Using the XMLA Endpoint XMLA = XML for Analysis An industry standard protocol supported by BI vendors Interacts with dataset via the Tabular Object Model (TOM) Power BI Power BI Service Desktop XMLA endpoint Premium or PPU for data Workspace Tabular management & Editor deployment Dataset ALM XMLA endpoint Toolkit for querying Other Key Components in a Power BI Architecture On-Premises Data Gateway Three Types of Gateways Virtual Network Data Gateway Power BI Service Personal Mode Data Gateway Standard Mode Data Gateway Cluster Three Types of Gateways VNet