Bringing AI to BI: Enabling Visual
Analytics of Unstructured Data in a
Modern Business Intelligence Platform
Darren Edge Abstract
The Business Intelligence (BI) paradigm is challenged Microsoft Research Cambridge, UK by emerging use cases such as news and social media [email protected] analytics in which the source data are unstructured, the analysis metrics are unspecified, and the appropriate
Jonathan Larson visual representations are unsupported by mainstream tools. This case study documents the work undertaken Microsoft Research Silverdale, WA, USA in Microsoft Research to enable these use cases in the [email protected] Microsoft Power BI product. Our approach comprises: (a) back-end pipelines that use AI to infer navigable Christopher White data structures from streams of unstructured text, Microsoft Research media and metadata; and (b) front-end representations
Redmond, WA, USA Figure 1: Overview page from of these structures grounded in the Visual Analytics “Advanced Search with Bing [email protected] literature. Through our creation of multiple end-to-end News” solution template for data applications, we learned that representing the Power BI. Shows interactive Permission to make digital or hard copies of all or part of this work for varying quality of inferred data structures was crucial personal or classroom use is granted without fee provided that copies are summaries of Bing News results for making the use and limitations of AI transparent to for search terms of interest. not made or distributed for profit or commercial advantage and that Back-end pipeline uses AI copies bear this notice and the full citation on the first page. Copyrights users. We conclude with reflections on BI in the age of services to structure articles for components of this work owned by others than the author(s) must be AI, big data, and democratized access to data analytics. honored. Abstracting with credit is permitted. To copy otherwise, or based on shared key phrases, republish, to post on servers or to redistribute to lists, requires prior named entities, topics, and specific permission and/or a fee. Request permissions from Author Keywords sentiment levels. Front-end [email protected] . Business Intelligence; Visual Analytics; Data; AI; HCI “dashboards” combine visual representations for exploring CHI'18 Extended Abstracts, April 21–26, 2018, Montreal, QC, Canada inferred structures. One of four © 2018 Copyright is held by the owner/author(s). Publication rights ACM Classification Keywords applications for unstructured data licensed to ACM. H.5.2. Information interfaces and presentation (e.g., ACM 978-1-4503-5621-3/18/04…$15.00 analysis in this case study. HCI): User Interfaces https://doi.org/10.1145/3170427.3174367
Text and metadata Introduction aggregations (e.g., sum, average) for targeted subsets Text is a primary source of This paper documents work undertaken in Microsoft of rows and columns. In 2000, Polaris [15] extended unstructured data, such as Research to extend the Microsoft Power BI product with pivot tables to enable graphical summaries of large from the following sources: support for analysis of unstructured data such as social multidimensional databases, in what laid the media, news, and cyber intelligence. The starting point foundations of the Tableau software product. Microsoft 1. Social media messages for this project was the observation that unstructured later extended Excel with similar capabilities, leading to 2. Message board posts data streams are of growing importance to general the release of Microsoft Power BI as an independent 3. Email message bodies business audiences, yet modern BI platforms require product in 2015. Use of BI platforms is now 4. Online news articles structured data tables prepared and visualized using mainstream in the business world, and adoption is 5. Enterprise documents specialized data science skills. We identified two related growing in the public spheres of science, engineering, Text is often accompanied by opportunities that could help bridge this gap: inferring education, and government. additional metadata that can information structures from unstructured data using “AI provide an initial means of services” that commoditize the results of machine Challenges of unstructured text and metadata grouping associated texts. learning, and supporting the visualization of text and BI platforms present data as “dashboards” of multiple Typical metadata include: metadata by creating representations grounded in the linked visualizations that both summarize and enable Visual Analytics literature. From Gartner’s industry interactive filtering of a common dataset. However, 1. Title and length advisory perspective [3], the resulting work has had much of the data relevant to modern organizations is 2. Authors and recipients significant impact on Microsoft’s 2017 position as a not in the form of structured numerical tables – it is in 3. Keywords and hashtags market leader in BI and Analytics, notably in terms of the form of unstructured text and metadata, spread 4. Timestamps and geotags “completeness of vision”. In this case study, we report across documents, social media, and the web (sidebar). 5. Views, shares, “likes”, etc. on both the artifacts produced through our research While the scale of such data makes it a candidate for Metadata can be intrinsic or (Figures 1–4) and the lessons learned from their dashboard analytics, as of early 2016, no major BI extrinsic to the text, and deployment, release, and user adoption and feedback. platform supported such unstructured data use cases. either given or derived (e.g., using AI services): Background Opportunities to extend Power BI for new use cases Evolution of Business Intelligence Microsoft Power BI offered two extensibility frameworks Metadata intrinsic extrinsic Business Intelligence platforms evolved from the need that could be adopted to extend the functionality of the given title url to make sense of largely numeric business data in the platform: “visuals” that can be used alongside native structured tables of spreadsheets and databases. visual representations such as bar, line, and pie charts, derived sentiment impact Historically, it has been unwieldly to work with such and “solution templates” that automate data access, Textual metadata can also be tables because of their large size and high processing, and representation in turnkey data derived from unstructured dimensionality. However, the invention of the pivot applications running in the Microsoft Azure cloud. Azure images, e.g., using OCR, table (pioneered in Lotus Improv in 1991 and also offers AI as a service through Microsoft Cognitive object recognition, and scene popularized by Microsoft Excel since 1994) gave users Services and Azure Machine Learning, providing key classification and description. the ability to explore tabular summaries of such tables capabilities for the structuring of unstructured data. by interactively pivoting between different numeric
Phase 1: Representations of text & metadata leads indirectly to potential insights, i.e., by guiding In the world of numeric data, aggregation functions like users to filter text collections down to meaningful sum, average, and count scale to data of arbitrary size. subsets that are of manageable size for per-text Similarly, visual representations of such aggregate review. These observations provided grounding values (e.g., bar, line, and pie charts) have the same principles for design: visual representations should visual complexity whatever the aggregate values. The collectively provide complementary views of both consequence is that all data subsets are self-similar summaries and content, and individually embody visual from a comprehension perspective, and that the notations that are agnostic of both the data domain and purpose of interactive “drill down” is to specify data the size of the data subset to be rendered. subsets whose aggregations provide direct answers to the user’s analytic questions, such as “How many We drew inspiration from the inherent scalability of
Figure 2: Power BI visuals
AI services Our Power BI visuals are available as open-source However, each step also poses an obstacle to users 1 There is a trend across large software on Github and as free-to-use downloads who are not both domain experts and data specialists 2 software companies to within Power BI or via the Office Store . The (e.g., data scientists, architects, or engineers): “metadata” visuals of Attribute Slicer, Time Brush, commoditize the results of 1. domain data of interest often require access through Network Navigator, and Table Sorter were released in machine learning as “AI database scripting or programmatic APIs; May 2016 [6], followed by the “document” visuals of services” accessible via APIs, 2. extracting meaningful structure from text requires Cluster Map, Facet Key, and Strippet Browser in July such as Amazon AWS AI programmatic analysis (e.g., using AI service APIs); 2016 [7]. Installing users are typically BI specialists Services and IBM Watson 3. data processing must anticipate the required visuals who compose visuals and datasets into reports that are Cognitive Capabilities. Our so the appropriate columns are available for binding; then shared within an organization for interactive data back-end data pipelines use 4. visual composition must anticipate the right domain exploration by non-specialists. Since each visual is both Microsoft Cognitive questions and the best interfaces for answering them. Services and Azure Machine typically incorporated into multiple reports, with each Learning modules, including: report accessed by multiple users across multiple The second phase of our work aimed to democratize sessions, it is crucial for visuals to be fast, reliable, and access to data analytics – enabling a large base of Sentiment Scoring usable by a general audience. It is also important for users at low cost and without specialized training. We Scores text on a continuous visuals to be useful across domains: emails to our sought not just to streamline the above process for scale from most positive support alias reveal a core user base in the functional existing users of Power BI, but to reach new audiences sentiment to most negative. business areas of sales, operations, and IT, but also through the turnkey generation of “data applications” Key Phrase Extraction use in a wide range of specialized domains including bound to specific data sources and search queries. Extracts key words that logistics, insurance, defense, security, energy, summarize a text and make infrastructure, aid, and healthcare. Phase 2: Data applications powered by AI The modular and composable nature of visuals and AI connections between texts. Releasing our representations of text and metadata as services (sidebar) allowed rapid construction of end-to- visuals enables visual analytics of unstructured data in Named Entity Recognition end data applications in partnership with customers and Power BI, provided users can: NER extracts mentions of business groups across Microsoft, supporting their need entities (e.g., people, places, 1. access the data of interest for analysis; to make sense of unstructured data in diverse areas organizations) within a text. 2. process data into the tables required by the visuals; including news, social media, and cyber intelligence. In line with our goal of democratizing data analytics, we Topic Modelling 3. bind the appropriate table columns to visual fields; have released several data applications as “solution Infers a topic model from 4. compose visuals into appropriate dashboard template” products for Power BI. We now present three multiple texts that assigns a combinations and filtering relationships for the of these products, plus an internal data application that dominant topic to each text. analytic questions. supports the work of the Microsoft Digital Crimes Unit. Optical Character Recognition Adoption of our visuals indicates users have OCR extracts text from successfully completed all four steps independently. images containing text areas. 1 Github visuals: https://github.com/Microsoft 2 Office Store visuals: https://appsource.microsoft.com/en- us/marketplace/apps?product=power-bi-visuals
Visual use in Twitter Campaign/Brand Management for Twitter solution template Our Twitter solution template was released in August Strippet Browser . Browse text, 2016 as a way for social media brand and campaign metadata of filtered tweets. managers to monitor relevant activity on Twitter [8]. Attribute Slicer . View and This template allows anyone with a Twitter API key and filter by author, hashtag, etc. Microsoft Azure subscription to create a live report on Time Brush . View tweet tweet activity around user handles, hashtags, and volume and filter by time. search terms of interest. Sentiment scoring provides Network Navigator . Explore additional structure for exploring tweets by their author-hashtag relationships. positive, negative, or neutral sentiment, and tracking Table Sorter . Explore tweets the overall tone of social conversations. ranked by sentiment, impact. Advanced Search with Bing News Visual use in Bing News Our Bing News solution template was released in March solution template 2017 as a way for news analysts to track breaking Bing Strippet Browser . Browse text, News stories matching search terms of interest [9]. It metadata of filtered articles. uses AI services for sentiment scoring, key phrase Attribute Slicer . View and extraction, topical clustering, and named entity filter by key phrase, domain. recognition. These complementary structures provide Time Brush . View publication users with multiple ways to both summarize the volume and filter by time. collection of news results and drill down to individual Cluster Map . View and filter articles of interest, which can be opened in a web articles by topical cluster. browser for further reading. Figure 1 shows a typical Facet Key . View and filter filtering interaction sequence. articles by mentioned entity.
Visual use in Facebook Campaign/Brand Management for Facebook solution template Our Facebook solution template was released in June Strippet Browser . Browse text, 2017 as a way for social media brand and campaign metadata of filtered posts. managers to monitor relevant activity on Facebook Table Sorter . Explore posts Pages [10]. The template allows anyone managing a ranked by sentiment, impact. Facebook page to analyze posts and comments by Network Navigator . Explore likes, authors, and hashtags, as well as AI-inferred Figure 3. Top : Page from Twitter solution template showing co-posting relationships as a sentiment levels and key phrases. Network analytics also reveal patterns of coordinated posting across Table Sorter ranking tweets based on the combination of sign of organic, coordinated, retweets, user followers and friends. The top tweet is selected. users, indicative of organic shared interests, or automated user interaction. Bottom : Page from Facebook solution template showing coordinated brigading, or even automated bot activity. Network Navigator of users co-posting in the last 7 days.
Extending custom visuals Tech Support Fraud Investigation Tool Phase 3: Representations of AI-structured data to represent data quality A 2016 global survey by Microsoft revealed that 2 out A recurring problem we faced in phases 1 and 2 of 3 people had experienced a tech support scam in the resulted from the varying quality of AI-inferred data News analytics example: previous 12 months [14]. 1 in 5 users continued with a structures. In some cases, AI services augment their revealing uncertainty in fraudulent interaction leading to the download of outputs with confidence or uncertainty scores, such as topical clustering and entity malicious software, granting of remote device access, the confidence that a machine translation is accurate. recognition over news articles or sharing of credit card or banking details for In other cases, such scores are mapped to specific unnecessary repairs or maintenance services. 1 in 10 semantics like reputation and trust. In yet other cases, users ultimately lost money. Such scams are typically AI outputs are themselves aggregated to communicate initiated by browser pop-ups that urge the user to call a derived metrics like weight, strength, distance, and toll-free number for live support, often masquerading similarity. Such data qualities arising from the as a familiar technology company. The Microsoft Digital inferential nature of AI span all types of uncertainty in
Crimes Unit receives over 10,000 complaints about information visualization [12]: measurement precision Cluster Map . Arcs segmented such scams each month, and tracking down the (e.g., of sentiment scores), completeness (e.g., of and coloured by quality level, scammers is further complicated by the ever-shifting IP entity recognition), inference (e.g., of topical models), e.g., view articles by topic fit. addresses which serve the pop-ups and the concealing and credibility and disagreement (e.g., of an ensemble of scam details in images rather than plain text [5]. text classifier spanning multiple input models).
We partnered with the Digital Crimes Unit to build a For back-end processing, the problem lies in deciding data application that enables interactive investigation of how to use data quality values as thresholds for dataset tech support fraud. This application mines scam pop-up inclusion: set the threshold too low, and the results can Facet Key. Bars segmented images, extracts embedded phone numbers using OCR, be unmanageably noisy and large; set the threshold too and coloured by quality level, connects related scams through image analysis, and high, and the results can omit crucial data points that e.g., view entity mentions by represents the resulting data structures using our happen to have low quality values. With visualization, entity recognition rank. visuals in Power BI: Network Navigator for viewing the the problem is one of transparency: for data that have resulting scam networks, Attribute Slicer for searching been pre-filtered by a data quality threshold, it is and filtering by phone number and network size, and unclear (a) what data were filtered out, and (b) what Strippet Browser for examining pop-up images and quality variations exist in the data that remain. their extracted details. Use of this tool by DCU analysts was central to Microsoft’s participation in Operation Our solution to these problems has been to modify our Strippet Browser. Entity Tech Trap, announced by the US Federal Trade data pipelines and visual representations such that: mention icons showing entity Commission in May 2017 [2]. Just one of the deceptive 1. elements of data structures arising from AI ambiguity, e.g., view entity tech support organizations targeted by the resulting processing are assigned quantized quality levels; mentions in news article text actions, Client Care Experts, was responsible for 2. visual representations show the distribution of data by entity recognition rank. defrauding 40,000 people out of more than $25 million qualities across levels and support filtering by level. (USD) over the period November 2013-2016.
Extending custom visuals Such “quality aware” interfaces avoid premature Discussion to represent data quality commitment to a threshold, whose appropriate value Across the phases of this case study, we sought to (continued) cannot be determined in advance of its creation and is make the structure of text and meta-data navigable , dependent on the user’s analysis task. Instead, they the operations of data acquisition, processing, and Twitter analytics example: make data quality a first-class interface element, analysis accessible , and the role of AI in inferring using language detection and allowing users to interactively explore the tradeoff navigable structures transparent . On a theoretical level, machine translation to analyze between data coverage (showing all data) and visual our work has been influenced by prior review of the tweets in a common language clarity (showing data subsets of given quality levels). Visual Analytics (VA) literature through the lens of Data quality keys. Attribute The resulting information seeking strategy can be Activity Theory and HCI [1] – seeking to understand Slicers for interpreting and captured in a refinement of Shneiderman’s mantra the broader systems of activity to be supported by VA filtering data by quality level, [11]: high quality overview first, zoom and filter, then tools. This review identifies interaction qualities to aim e.g., tweet counts by details-on-demand for lower quality levels in areas of for when designing such tools, each addressing a core translation quality (Fig. 4ab) . interest . By juxtaposing AI-inferred structures against trade-off in the activity design space (sidebar, page 8). the unstructured data they describe, users can calibrate We now present three lessons in a similar form – as Attribute Slicer. Frequency system-assigned quality levels against their own quality tensions we encountered in the design space, bars segmented and colored judgements or the requirements of the use case. reflections on our practice, and implications for design. by quality level, e.g., tweet counts by language and translation quality (Fig. 4ab). Time Brush . Time bars segmented and colored by quality level, e.g., tweet counts by time and translation quality (Fig. 4c). Network Navigator. Links weighted and colored by quality level, e.g., hashtag- language links by co- occurrence level (Fig. 4d) . Table Sorter . Rows showing how percentile rank varies based on the incremental Figure 4. (a-b) Attribute Slicer. Top : distribution of non-English tweets across machine translation confidence levels. Bottom: inclusion of quality levels, distribution of non-English tweets across languages, segmented and colored by translation quality. (a b) Filtering tweets to level 1 only e.g., tweets by retweets and – those with the highest translation quality. (c) Time Brush showing volume of non-English tweets over time, segmented and colored by translation quality (Fig. 4e). translation quality. (d) Network Navigator showing hashtag-language connections weighted and colored by tweet co-occurrence level. (e) Table Sorter ranking by retweet count. Columns to right show how rank percentiles change as lower-quality translations are added.
Target qualities of Visual Meaningful summaries vs manageable subsets number of visual representations per page and the Analytics tools [1] We approached the design of our visual representations number of pages required to cover all use cases. with a focus on creating navigable summaries of text Presentable analysis: and metadata. Through our repeated use of these Automatic insights vs interactive oversight ability to curate presentable representations for building data applications, we Using AI services and data visualization to marshal summaries of the analytic observed such summaries carry limited meaning in unstructured data into meaningful representations discovery process . Supported isolation – accurate comprehension relies on context automates the initial stages of insight discovery at the by visuals that resolve the from the documents being summarized. For example, cost of generating inferred structures of varying quality. tension of acting to make interpreting a key phrase distribution requires viewing Building interfaces around data qualities provides a new sense vs artifacts : familiar common phrases in juxtaposition with document text. kind of “interactive oversight” for human consumers of metaphors aid sense-making Until the underlying documents have been filtered to a AI services that enables quality-aware filtering of data and can be presented directly manageable subset for review, the main value of visual to meaningful and manageable subsets. The downside to general audiences. representations is their ability to guide such filtering is that each visual used as a key for data quality levels Portable analysis: ability to towards document subsets of interest. Future work occupies space that could have been used for an transfer analytic work across includes using this insight to create sample-driven additional and complementary view of data structure. people, places, time, and summaries of big data that only reach full fidelity once Future work includes investigating the interactive devices. Supported by the data have been filtered to a manageable volume. assignment of “human verified” quality levels shared solution templates that among the users of long-lasting, widely-used reports. resolve the tension of acting Analyzing datasets vs monitoring datastreams as data collector vs analyst : Builders of Power BI reports typically have specialist Conclusion automation of data collection data preparation skills, as well as specific datasets to This case study described our transformation of Power frees time for analysis and analyze. In contrast, users of BI reports instantiated BI for visual analytics of unstructured data. The impact enables contributions from from our solution templates only need to specify of the work includes fundamental visual representations “citizen data analysts”. standing search queries for persistent interests, and to with wide adoption, AI-powered “solution templates” Provisional analysis : ability monitor the resulting datastreams through pre-built that shape the view of Microsoft as a market leader [3], to view and proactively reports. While solution templates have the potential to an AI-powered data application used for the successful reduce the uncertainty of democratize access to data analytics through ease of identification and prosecution of major cybercriminal analytic work at any time. use, it remains a challenge to create dashboard operations, and a design philosophy around “data Supported by representations interfaces that are sufficiently capable for domain qualities” that anticipates the growing role of AI in of data quality that resolve experts whilst also being approachable and learnable by democratizing access to data analytics. the tension of competing novice users of BI tools. We have adopted a range of interpretations vs demands : assistive strategies, including labelling representations Acknowledgements filtering by data quality level by functional role rather than column bindings, We would like to thank our collaborators in Microsoft enables systematic review of arranging and numbering representations by page Research, Uncharted Software, Microsoft Power BI, and uncertain data structures. workflows, and ordering pages by activity workflows. the Power BI Solution Templates team for their Future work includes tackling the tradeoff between the substantial contributions to the work of this case study.
References 9. Microsoft Power BI Blog (7 March 2017). Retrieved 1. Darren Edge, Nathalie Henry Riche, Jonathan 1 Oct. 2017 from Larson, and Christopher White. 2018. Beyond https://powerbi.microsoft.com/en- Tasks: An Activity Typology for Visual Analytics. us/blog/announcing-the-advanced-search-solution- IEEE Transactions on Visualization and Computer template-for-bing-news/ Graphics (TVCG) 24(1), 267-277. 10. Microsoft Power BI Blog (26 June 2017). Retrieved http://ieeexplore.ieee.org/document/8019880/ 1 Oct. 2017 from 2. Federal Trade Commission Press Release (12 May https://powerbi.microsoft.com/en- 2017). Retrieved 1 Oct. 2017 from us/blog/announcing-the-campaign-brand- https://www.ftc.gov/news-events/press- management-for-facebook-pages/ releases/2017/05/ftc-federal-state-international- 11. Ben Shneiderman. 1996. The eyes have it: A task partners-announce-major-crackdown by data type taxonomy for information 3. Gartner Report (16 February 2017). Magic visualizations. IEEE Symposium on Visual Quadrant for Business Intelligence and Analytics Languages, 336-343. Platforms. Retrieved 1 Oct. 2017 from http://ieeexplore.ieee.org/document/545307/ https://www.gartner.com/doc/reprints?id=1- 12. Meredith Skeels, Bongshin Lee, Greg Smith, and 3TYE0CD&ct=170221&st=sb George G. Robertson. 2010. Revealing uncertainty 4. Samuel Gratzl, Alexander Lex, Nils Gehlenborg, for information visualization. Information Hanspeter Pfister, and Marc Streit. 2013. Lineup: Visualization 9(1), 70-81. Visual analysis of multi-attribute rankings. IEEE http://journals.sagepub.com/doi/abs/10.1057/ivs.2 Transactions on Visualization and Computer 009.1 Graphics (TVCG) 19(12), 2277-2286. 13. John Stasko, Carsten Görg, and Zhicheng Liu. http://ieeexplore.ieee.org/document/6634146/ 2008. Jigsaw: supporting investigative analysis 5. Microsoft AI Blog (15 June 2017). Retrieved 1 Oct. through interactive visualization. Information 2017 from visualization 7(2), 118-132. https://blogs.microsoft.com/ai/2017/06/15/micros http://journals.sagepub.com/doi/10.1057/palgrave oft-used-ai-help-crack-tech-support-scams- .ivs.9500180 worldwide/ 14. Stay Safe Online Blog (16 October 2016). Retrieved 6. Microsoft Power BI Blog (9 May 2016). Retrieved 1 1 Oct. 2017 from Oct. 2017 from https://powerbi.microsoft.com/en- https://staysafeonline.org/blog/new-data-reveals- us/blog/custom-visualizations-visual- two-thirds-of-global-consumers-have-experienced- awesomeness-your-way/ tech-support-scams 7. Microsoft Power BI Blog (11 July 2016). Retrieved 1 15. Chris Stolte, Diane Tang and Pat Hanrahan. 2002. Oct. 2017 from https://powerbi.microsoft.com/en- Polaris: A System for Query, Analysis and us/blog/new-power-bi-custom-visuals-for- Visualization of Multi-dimensional Relational browsing-and-analyzing-collections-of-text/ Databases. IEEE Transactions on Visualization and Computer Graphics (TVCG) 8(1), 52-65. 8. Microsoft Power BI Blog (15 August 2016). http://ieeexplore.ieee.org/document/981851/ Retrieved 1 Oct. 2017 from https://powerbi.microsoft.com/en-us/blog/twitter- solution-template/