Transforming Sharepoint Into a Unified Information Access Platform How the BA Insight Software Portfolio Extends Sharepoint 2013

Transforming SharePoint into a Unified Information Access Platform How the BA Insight Software Portfolio Extends SharePoint 2013

BA Insight 2015 How BA Insight Turns SharePoint 2013 Into a Unified Information Access Platform

Table of Contents

The Need for Unified Information Access ...... 2

Search-Driven Applications and UIA ...... 2 Approaches to UIA Platforms ...... 3 Leveraging SharePoint ...... 4 The BA Insight Software Portfolio ...... 5

Going Beyond SharePoint 2013 OOB ...... 6 Connectivity Beyond OOB SharePoint ...... 6 MultiSearch: Beyond OOB SharePoint Federation ...... 7 Content Enrichment Beyond OOB SharePoint...... 8 User Experience Beyond OOB SharePoint ...... 9 Incorporating Other SharePoint 2013 Capabilities ...... 11 Summary ...... 12

References ...... 12

About BA Insight ...... 13

The Need for Unified Information Access

The modern enterprise is struggling to make sense of the structured and unstructured information that is being generated and leveraged in exponentially growing volumes across an increasing variety of application silos. The fragmentation, diversity, and overall inaccessibility of this data can lead to a number of suboptimal outcomes, such as the lack of responsiveness to customer inquiries, widespread duplication of effort, general inability to locate and nurture expertise, and decreased employee productivity.

As awareness grows of the challenges posed by the volume, variety, and velocity of content, today’s IT leaders are increasingly evaluating technologies that facilitate access to and consolidation of structured and unstructured content to gain timely insights into various business critical scenarios. One such technology is Unified Information Access (UIA), an advanced implementation of Enterprise Search technology that transcends simple keyword searching to allow for secure, ad hoc interrogation of multiple content sources in a highly targeted and contextually specific manner.

This white paper discusses UIA platforms and applications, and in particular describes how the BA Insight Software Portfolio transforms SharePoint into a powerful UIA platform, providing capabilities beyond SharePoint 2013 out-of-the-box.

Search-Driven Applications and UIA

A search-driven application is a software application that uses a search engine as the primary information access backbone, the main purpose of which is to surface critical information at the right time within the right context. There are many examples, including:

Customer Service – empowers customers to engage and resolve issues online, enables self-service, and reduces repetitive questions and answers.

Professional Services – enables identification of experts who have addressed similar needs, delivers immediate and relevant knowledge to drive projects forward, and supports subscriptions to particular topics for automatic notification of updates.

Helpdesk – provides actionable resolutions through instant access to critical information through one portal, and delivers a complete 360 degree view of customer information across multiple sources such as CRM and ERP systems.

Research and Development – minimizes duplication of effort, increases the speed of innovation, finds product information quickly, and creates tailored R&D environments. e-Commerce – provides a responsive, contextual, and remarkable user experience to allow customers to interact with a brand, find relevant products, and follow up on orders.

While Internet Search has evolved rapidly over the past two decades, Enterprise Search has, for the most part, remained the same. Internet Search has gone from being a largely horizontal keyword driven catchall for web content (Lycos, Alta Vista, Google) to being the driver of a number of targeted

© Copyright 2015 BA Insight. All Rights Reserved. Page | 2 How BA Insight Turns SharePoint 2013 Into a Unified Information Access Platform applications that have fundamentally transformed the way information is accessed. Search now propels applications that help us to book travel, navigate from point A to point B, research general interest topics and purchase vital goods and services (Travelocity, MapQuest, and Wikipedia). These ‘Search-Driven Applications’ (SDAs) have a number of notable characteristics:

1. They are contextually specific: far from being one search box that yields “ten blue links”, SDAs are Hotel Room Finders, Flight Finders, and Product Finders, along with research apps for travel and general interest.

2. They leverage search to access and surface content from a number of sources of both structured and unstructured data in a unified view.

3. The user experience involves exploration and navigation rather than being simply query-driven, and it features the use of graphical refinement options like pick-lists, map parts, and sliders that obviate the need for more complex query formulation.

SDAs are an intuitive, dynamic, and targeted means of rapidly accessing a diverse and fragmented data set. This approach “offers businesses a rapid, low risk way to eliminate some of the peskiest and most common information systems (IS) problems: siloed data, poor application usability, shifting user requirements, systemic rigidity, and limited scalability” (Grefenstette, 2011). The pattern of multiple tailored and targeted applications is familiar to us in the consumer world and, as our experience suggests, it can be a successful pattern in the enterprise as well.

Unified Information Access platforms support search-driven applications particularly well. Typically, there is not one but many different sources of information needed in a search-driven application, and there may be structured as well as unstructured information. Deploying multiple applications on a UIA platform is far more cost-effective than fielding each application on a standalone basis.

Approaches to UIA Platforms

UIA platforms include a wide range of technology such as connectivity, text analytics, and data visualization. There are different approaches you can take to cover the range of capabilities involved in UIA (IDC, 2014):

. Assemble the capabilities as ‘bits and pieces’, with components from different sources and vendors.

. Procure a standalone platform, with all of the capabilities included, and integrate it into your environment.

. Leverage the existing infrastructure (such as search core, workflow engine, and BI stack) and add a pre-integrated software or components to cover the range of UIA capabilities.

The BA Insight Software Portfolio takes the third approach and leverages existing infrastructure, in particular SharePoint.

Leveraging SharePoint

SharePoint is the most common content and collaboration platform found in enterprises today. It focuses on providing pre-integrated capabilities in a wide range of areas and is an extensible infrastructure for business applications. SharePoint 2013 in particular has strong capabilities in a number of areas relevant to UIA:

 Presentation Framework: SharePoint provides a Familiar UI that is integrated, extensive, and easy to brand and extend using standard tools.

 FAST Search Core: SharePoint includes a Powerful, Scalable, Extensible search core based on FAST technology.

 Integrated BI: Dashboards, Analytics, and Excel Services are all built into SharePoint.

 Integrated SharePoint Workloads: WCM, ECM, Workflow, and e-Discovery capabilities are all included with SharePoint and can be leveraged and extended.

SharePoint has an extensive supporting ecosystem which includes numerous ISVs and System Integrators, as well as trained administrators within many organizations. SharePoint 2013 is also built with search-driven applications in mind, and is set up to support multiple search-driven applications running on the same infrastructure. (Tordgeman, 2013)

SharePoint 2013 provides a great infrastructure to leverage for UIA, as the underlying search core has many strong capabilities. (BA Insight, 2013) Managed Metadata is pre-integrated via the SharePoint term store; and workflow, BI, and social features are easily accessed and leveraged.

However, SharePoint 2013 by itself is not a UIA platform. It does not include many of the connectivity, text analytics, and user interface capabilities needed for Unified Information Access. Nor does SharePoint claim or aspire to be a UIA platform – its focus is on providing a general purpose, mass market infrastructure with pre-integrated workloads.

The rest of this paper illustrates how to transform SharePoint 2013 into a Unified Information Access Platform using BA Insight’s Software Portfolio (BA Insight, 2014). After introducing the platform, we examine each part of it with specific emphasis on how it extends SharePoint 2013 and the capabilities that go beyond out-of-the-box (OOB) SharePoint 2013.

The BA Insight Software Portfolio

BA Insight’s Software Portfolio extends the core capabilities of Microsoft SharePoint and FAST to deliver Unified Information Access and, in doing so, facilitates the delivery of a truly Internet-based search experience. It enables organizations of all sizes to leverage their existing investment to rapidly deliver role-specific SDAs without the risk and cost of custom development.

The platform is shown in Figure 1 below.

Figure 1: BA Insight's Software Portfolio

BA Insight’s software portfolio provides a comprehensive set of capabilities:

User Experience Applications - Smart Previews, User-Generated InfoSites, and Matter Comparison to help users find the right information faster for improved productivity.

Content Classification - Auto-tagging, metadata generation, and text analytics to increase findability.

Content Connectivity - Secure connectors and federation to a wide variety of content systems, enabling unified views of all knowledge assets.

This flexible portfolio can be implemented as a comprehensive solution suitable for organizations at the beginning of a unified information access project, or components of the portfolio can be leveraged to augment existing technology.

The Software Portfolio is based in SharePoint. It fits in with the SharePoint paradigms and toolsets so that IT Pros and developers who are familiar with SharePoint can leverage their existing skills to implement and work with BA Insight products. Most importantly, it fully leverages the capabilities of SharePoint. The

Software Portfolio can be used with OOB SharePoint, or it can be combined with SharePoint customizations and add-ons in a solution. SharePoint capabilities such as workflow and BI are available as part of applications created with the Software Portfolio.

Going Beyond SharePoint 2013 OOB

Each component within the Software Portfolio provides capabilities that go well beyond SharePoint 2013 OOB. BA Insight does not duplicate capabilities available with SharePoint OOB, but instead adds capabilities that complement the OOB features.

All of these capabilities support the creation of unified, role-based environments in which users are able to find relevant information. These solutions need to be search-driven, highly scalable, and connected to the rest of the enterprise systems in order to make information access dynamic, ad hoc, and relevant to users.

Connectivity Beyond OOB SharePoint

Any Unified Information Access platform requires what IDC calls “smart connectors” (IDC, 2013). BA Insight’s Content Connectivity software provides secure connectivity to a wide variety of content systems to provide a unified view of all knowledge assets.

Connectors: The most obvious connectivity capabilities are indexing connectors. BA Insight provides connectors to 50+ enterprise sources – all with high performance, flexibility, and security. These work alongside the OOB connectors (for SharePoint, SharePoint profiles, file shares, and websites).

Connector Development: New connectors, including custom connectors, are built using either the Universal SQL toolkit or a Web Services API, both of which complement and extend the BCS framework. These both build upon a Connector Framework, which provides more sophisticated control and faster development compared to working directly with BCS. The Connector Framework includes pre-built administrative controls, which new connectors gain automatically, resulting in significant time savings when creating new connectors as well as better and easier administration.

Targets: These provide the ability to replicate and aggregate information outside of the SharePoint Search Index, making a variety of interesting scenarios possible. Targets include a search index other than SharePoint, a SharePoint List, the User Profile Store, or custom targets. As an example, enhanced People Search uses a User Profile Store target to aggregate a consolidated profile based on information from many different systems.

Security: BA Insight extends the early binding OOB security (claims and ADFS) to handle the most difficult search security scenarios. Advanced security capabilities such as role-based and attribute-based security are supported, and security mapping can be extended into custom fields. These security capabilities have proven to be essential for Unified Information Access applications, which require strong and flexible security across many different sources.

Smart Mapping: The manual property mapping available OOB is enhanced to reduce administrative effort and make sophisticated scenarios possible. This includes Dataset Connectors, Scripting, and Automated Mapping. Access to all Crawled Properties and Managed Properties is maintained so that administrators have full visibility and control.

Test Bench: BA Insight provides an integrated Test Bench so that connections and sources can be tested and debugged in isolation from the rest of the search system. Testing content sources and seeing the metadata and performance without putting content into the search index has proven to be a huge timesaver for administrators and developers alike. Scheduled tasks are also provided so that repetitive or off-hours work can be managed easily.

Associated Crawls and Dataset Connectors: BA Insight includes capabilities for sophisticated content crawling and association. These extend the OOB crawling mechanism to reduce data quality issues and make sophisticated scenarios possible. Data Set connectors provide a lookup into external systems during indexing and allow metadata to be normalized. Associated Crawls allow additional content to be retrieved and included as part of an indexed item. For example, metadata from a database can be indexed along with associated content in a file system.

MultiSearch: Beyond OOB SharePoint Federation

Query Federation is an important tool in the Unified Information Access platform toolkit, and it comes in several forms (Schubmehl, 2013). Requirements for this come up in several ways:

. Data residency, information export, and contract restrictions may make centralized indexing impossible. . Organizational control, varying requirements, or logistical needs may make separate search implementations necessary or desirable. . External content such as subscription content, news sources, web search engines, or large-scale research repositories may not allow crawling, or it may be prohibitively expensive to crawl and index. These scenarios require a unified view but do not allow for all content to be in a single index. To address them, a capability to query against multiple indexes and collect the results is necessary.

In SharePoint 2013 there is an OOB query federation capability called Remote Result Sources. BA Insight provides a MultiSearch capability which builds on the OOB mechanism to provide a seamless unified view across multiple remote sources. It uses the OOB federated security model and provides several important capabilities beyond OOB query federation.

Interleaved Results: MultiSearch extends the OOB result source mechanism to provide a true unified view experience which extends beyond result blocks and local refiners to a fully interleaved result set. Refiner values are merged, and metadata can be augmented – including the creation of a source refiner which allows users to see which results come from which source, as well as the ability to drill down to a specific source or set of sources. A single result set has proven to be much more natural and effective

© Copyright 2015 BA Insight. All Rights Reserved. Page | 7 How BA Insight Turns SharePoint 2013 Into a Unified Information Access Platform than different tabs or lists in many scenarios. The more sources, the more important it is to have an interleaved result set.

Federated Back-ends: These complement the OOB result sources, which are sealed and limited to remote SharePoint, Exchange, and OpenSearch. BA Insight provides additional sources (called back- ends) which include full security and refiners and work cross-platform. These include FS4SP, FAST ESP, Google Drive, SharePoint 2010, and SharePoint 2013.

Custom Back-ends: A back-end toolkit makes it possible to create custom federated sources with all the same attributes as the native back-ends. This includes relevance scores, refiners, and full security, which goes far beyond the OpenSearch mechanism available OOB.

Hybrid Search: BA Insight extends the OOB hybrid search model to support a wider variety of scenarios for unified search results between Office 365 and on-prem SharePoint. These include cross-version federation, content enrichment, and federation across multiple farms and tenancies. MultiSearch supports either SP2013 or SP2010 on-prem, working with SharePoint 2013 in Office 365. There is also an O365 connector which supports crawling O365 from on-prem SharePoint search.

Content Enrichment Beyond OOB SharePoint

Another essential element of a Unified Information Access platform is text analytics applied to pre- processing of content. BA Insight’s Content Classification software increases the findability of content through the creation and normalization of metadata. It generates and extends managed metadata and provides a collection of powerful text analytic capabilities.

SharePoint includes a Managed Metadata Service (MMS) which holds taxonomies and ontologies and allows for manual tagging of content to drive findability, workflows, compliance, etc. It also includes a number of other powerful elements used in information architecture, such as content types. There are also OOB verbatim extractors (string matchers) to populate managed properties in search. However, SharePoint does not include text analytics or auto-classification. This is where the Content Classification software comes in.

Automatic Metadata Creation: Complements OOB metadata, leveraging the SharePoint MMS (aka term store) to assign metadata to content by machine rather than by hand. Event triggering allows composition of metadata generation, forms, and workflow. Metadata generation can be done in-place (in SharePoint lists and libraries), or during indexing (against any content source with a connector). Automatic metadata creation enables a wide variety of metadata-driven scenarios around UIA, search, ECM, workflow, and compliance.

Entity Extraction: is one of the forms of automatic metadata creation. BA Insight complements and extends the OOB verbatim extractors (string matchers) to support sophisticated scenarios. This includes support for Regular Expressions, Partial Matchers, and Custom Extractors.

AutoClassification: adds the ability to create metadata beyond string-matching. It supports taxonomy and ontology-based scenarios such as a hierarchy of product names. The Content Classification software provides Rules-Based classification, Hierarchical (taxonomy-based) classification, and Extended Tags that support content linking and ontologies.

Content Types: are provided in SharePoint and were enhanced with SharePoint 2013. BA Insight adds the capability to automatically assign content types. This can be used with the SharePoint Content Organizer and other OOB facilities to support a variety of different scenarios and provide better compliance and content quality.

Taxonomy Management: a Taxonomy Manager in the Content Classification software complements the OOB taxonomy management UI and can be used alongside or instead of it. The Taxonomy Manager includes a drag & drop UI and a Test Bench which reduces administrative effort and speeds development and troubleshooting of classification rules.

Import/Export: supports interoperability across industry tools by import and export to the SharePoint term store and includes important elements beyond the OOB CSV import/export terms. Support includes CSV, SKOS, RDF, and ARTX formats for import/export; the files include terms, nodes, and relationships plus Synonyms, Rules, and Extended tags.

The Content Classification software brings machine-made metadata into the SharePoint infrastructure. This is an essential element of Unified Information Access and provides a very powerful basis for effective search-driven applications. Since the metadata is integrated into SharePoint, it is immediately accessible for workflow, analytics, forms, and content management. For example, when a document is uploaded to a drop-off library, a document can be auto-classified, with a form presented to the user to verify the metadata. The Content Organizer (part of SharePoint) can then move it to the appropriate library – ensuring consistent metadata including content types as well as consistent organization of document locations.

User Experience Beyond OOB SharePoint

The user interface makes a remarkable difference in the effectiveness of Unified Information Access, and tailored search-driven applications need tailored user interfaces. Yet the effort and expertise required to do this well can be so daunting that many organizations don’t undertake it, or do so and fail. That’s why BA Insight has developed software that reduces the risk and cost of crafting these applications. User Experience Applications enable the creation of compelling and effective user interfaces for search-driven applications.

SharePoint 2013 provides powerful mechanisms for tailoring the UI, including display templates, query rules, and web parts. User Experience Applications use and extend these mechanisms. IT Pros configure the user interface using familiar tools and paradigms, and developers can further customize the result using standard techniques – while leveraging a set of prebuilt, proven components. User Experience Applications provide many capabilities that go beyond OOB SharePoint:

Workspace: adds a per-user workspace capability allowing users to tag, remember, and share content from a search session. Users can Select, Arrange, and Export from the workspace; the workspace is persistent across devices and sessions so you can continue to work seamlessly from the office PC to your iPad at home.

Content Assembly: enables the creation of composite documents once they are tagged into the Workspace. For example, users can create Word documents, PowerPoint presentations, PDFs, Excel manifests, and ZIP fields using search derived content while preserving images and native formatting. This works on selected content in any supported format, stored in any repository with a connector. Content Assembly has proven to be a huge timesaver, as users select and re-use content into a new report automatically, eliminating cut-and-paste work.

Smart Previews: are visual images for quick recognition and exploration of relevant information, including features for exploring information including search inside, hit mapping, hit highlighting, and contextual actions. These complement the OWA previews by providing previews of content stored outside of SharePoint 2013 and in file formats beyond Office 2013. The BA Insight HTML Viewer shows Smart Previews on documents stored anywhere in a wide variety of file formats including Office, PDF, Zip, email, image – and takes the user to the most relevant page.

This capability is based on Smart Preview Generation, which understands the format and structure of many file types and creates data structures for navigating inside documents and connecting across documents. Smart Preview Generation allows large complex documents to be processed at index time, providing a dramatically quicker response time and lower bandwidth usage. Smaller documents can be processed on demand at query time.

Navigation Controls extend the OOB Refiner Panel and Web Part paradigm to provide an extensive set of pre-built controls for intuitive navigation. These include visual controls for drilldown, refinement, and content exploration, including many different presentation elements such as multi-select tree refiners, graphs, and dashboard layouts. Collapsing refiners preserve precious screen real estate while surfacing the many different facets of content. Searching inside refiners allows users to navigate across large amounts of data quickly and with great precision. Query Controls are configurable elements such as breadcrumb, site tree, and date picker that make it easy for the user to form effective queries.

Saved Searches: Adds the ability to save, review, and run queries, including complex queries with multiple refinements applied. A web part provides immediate access to them, along with the display of query history. Saved Results adds the ability to export a result set into excel for analysis, sharing, and support of sophisticated scenarios. These have proven to be highly valuable both in time savings and in providing analytics on content to drive insight and decisions.

Dynamic Query Transformation: brings the concept of Query Rules to the next level by allowing full control of query processing via plug-in processing stages. It can be used alongside or instead of query rules, enables a wide range of search scenarios and provides full control of behavior like linguistic query processing, query steering, and application-driven results. These transformations change and modify search queries to model the intent of the user, expand queries with external information, and affect

© Copyright 2015 BA Insight. All Rights Reserved. Page | 10 How BA Insight Turns SharePoint 2013 Into a Unified Information Access Platform relevance on-the-fly. All of this is done transparently to the UI, which simplifies development and administration.

Dynamic Result Transformation: brings the concept of Result Sources to the next level by allowing full control of result processing via plug-in processing stages. It can be used alongside or instead of result sources. These transformations update the result set, mixing in dynamic data (such as adding real-time inventory levels, stock market values, or weather information with a result), and reorder or translate the values to deliver the best result for each user. This enables a wide range of search scenarios and content mashups.

All together, the User Experience Applications provide a very powerful suite of pre-built components proven to be essential for success. Rather than rebuilding the wheel, enterprises can assemble remarkable applications using these components, and style the UI using standard CSS and JavaScript.

Incorporating Other SharePoint 2013 Capabilities

You have seen how the Software Portfolio turns SharePoint 2013 into a Unified Information Access platform, and read about capabilities that go beyond OOB SharePoint 2013 in every part of the platform. There are also many capabilities of SharePoint 2013 OOB that contribute to UIA. In particular, SharePoint has a complete BI stack including Excel Services, PowerView, PerformancePoint, and PowerBI, so that insights from traditional BI on structured content can be part of search-driven applications. Workflows can be configured to work with metadata generation and also with searches, directly as part of these applications. The social, composite, and e-Discovery workloads within SharePoint can also be tapped.

The combination of SharePoint and the Software Portfolio is a more capable UIA platform than any standalone UIA product, bringing a wider range of pre-integrated capabilities into the mix. The development and administrative tooling available is robust, and there are a large number of professionals familiar with it. The Software Portfolio transforms SharePoint into a UIA platform that is unparalleled in the depth and breadth of its capabilities, its ease of administration and extension, and its cost-effectiveness.

Summary

The BA Insight Software Portfolio is not another search engine or standalone UIA platform. Instead, it leverages the FAST index in SharePoint as well as the metadata management, user profile, and BI capabilities - and extends these with a comprehensive set of connectivity, text analytics, and user interaction capabilities. It enhances the systems you already have in place by connecting to them, enriching the content, and unifying the presentation of this content with information from other systems.

The result enables rapid assembly of powerful search-driven applications that give users the ability to view, manage, and access content in one place - regardless of its structure or where it resides in your organization.

References

BA Insight. (2013). The Essential Guide to Enterprise Search in SharePoint 2013. BA Insight.

BA Insight. (2014). Software Portfolio Data Sheet. available from www.bainsight.com.

Grefenstette, G. a. (2011). Search-Based Applications. Morgan & Claypool.

IDC. (2013). Unified Information Access 2013: Unifying Content SIlos to Uncover Hidden Value.

IDC. (2014). Unified Information Access: Turning Enterprise Search Into a Killer App. available at www.bainsight.com: IDC Analyst Connection.

Schubmehl, D. (2013). Agile Information Integration Using Federated Search. IDC.

Tordgeman, J. (2013). Learning Search-driven Application Development with SharePoint 2013. Packt Publishing.

About BA Insight

BA Insight’s software portfolio accelerates and future-proofs the creation of cloud-based and on-premise SharePoint portals for enterprises, transforming how users find information. Our software dramatically reduces the time, cost, effort, and risk of implementing SharePoint portals while providing a greatly improved user experience. Customers use our applications to improve productivity by finding the right information faster using Smart Previews, User-Generated InfoSites, and Matter Comparison; our classification software to increase findability using auto-tagging, metadata generation, and text analytics; and our pre-built connectors and federation to provide secure connectivity to a wide variety of content systems.

We serve organizations of all sizes around the globe including the Australian Government Department of Defence, CA Technologies, Chevron, Deloitte, Ford Motor Company, Keurig Green Mountain, Pfizer, and Travers Smith. BA Insight is a Microsoft Gold Certified Partner and member of the Microsoft Business- Critical SharePoint Program.

Visit www.BAinsight.com for more information and follow us at @BAinsight.