The Digital Enterprise Demands a Modern Integration Approach
Nada daVeiga, Sr. Dir. of Technical Sales Tony LaVasseur, Territory Leader Yesterday’s approach to data and application integration is a barrier to becoming an agile
connected enterprise… CSV The Digital Divide
Forces Decreasing IT Responsiveness • “Technical Debt” • Growing IT Skills Gap Business • Outmoded Operating Model
Forces Increasing Business IT Needs & Expectations • Rate of business change • Growing Consumerism • Easy access to SaaS
Source: PWC Common Barriers to Big Data Adoption
Common barriers to big data adoption: . Skills shortages . Integrating multiple data sources . Connecting big data technology with existing infrastructure
“Survey Analysis: Big Data Investment Grows But Deployments Remain Scarce in 2014,” Nick Heudecker & Lisa Kart, Gartner, 2014. Hybrid cloud and on-premises Explosion of Big Data brings: application environments: • New opportunities for insights • Create data silos • New data integration and • Increase integration complexity management challenges
Source: Forrester Research, Inc. All rights reserved. Hybrid Enterprise Demands New Approach
Why Not Traditional Or Open Source ETL/ESB Tools? • Costly hardware/software required • Inability to handle unstructured data • Manual processes require expert coders or consultants • Not designed for cloud applications • Long lead times to handle new/changed sources The dilemma for enterprise IT organizations is that their legacy integration technologies were built before the era of big data, social, mobile and cloud computing and simply can’t keep up.
- Gaurav Dhillon, founder and CEO, SnapLogic
• Investors: Andreessen Horowitz & Ignition Partners, Microsoft
• Customers: Adobe, Acxiom, ADP, AstraZeneca, Blackberry, BMS, Box, Cisco, CapitalOne, Cognizant, Danone, Fox Sports, GameStop, GE, HP, Humana, IDG, iRobot, RocketFuel, SANS, Target, Yelp, Uber, Verizon, Xactly, Yahoo!
• Partners: Cloudera, Hortonworks, MapR, Workday, Amazon, Google, Microsoft, PwC, Accenture, Cognizant, HCL, TCS SnapLogic: Connect Faster
SnapLogic helps enterprises connect data and applications faster
Anything Data | Apps | APIs | Things
Anytime Batch | Streaming | Real-time
Anywhere On premises | In the cloud SnapLogic: Connect Faster
SnapLogic helps enterprises connect data and applications faster Anything Reduce total cost of ownership Data | Apps | APIs | Things Do more, faster Anytime Future-proof IT environment Batch | Streaming | Real-time Support business agility Anywhere On premises | In the cloud
SnapLogic in the Modern Data Fabric Consume
DELIVER DELIVER
HANA
Data Warehouses & Data Data Integration and
Marts Transformation Big Data and Data Lakes Store & Process Store &
INGEST INGEST
z z z z
On Prem Relational Cloud Web NoSQL Internet of Source Applications Databases Applications Logs Databases Things
SnapLogic Pipeline Big Data Integration a Differentiator
INGEST PREPARE DELIVER
•
Hadoop + Spark SnapReduce: “Hadoop for Humans”
Eclipse SnapReduce (Programmatic) (Declarative)
This
That
With Cascading, SnapLogic automatically generates complex MapReduce code reducing development costs & accelerating time to value Snaplogic as a native YARN application
DATA DATA ACCESS SECURITY OPERATIONS INTEGRATION
Data Workflow, Batch Script SQL NoSQL Stream Search Authentication Provision, Lifecycle Cloud/On-prem Authorization Manage & Map Reduce Pig Hive/Tez, HBase Storm, Solr Data Access, Data Prep, API Accounting Monitor Sqoop HCatalog Accumulo Spark Delivery Data Protection Flume Ambari NFS Zookeeper YARN : Data Operating System Storage: HDFS WebHDFS Resources: YARN Access: Hive, … 1 ° ° ° ° ° ° ° ° ° ° Pipeline: Falcon Scheduling 300+ Pre-built Cluster: Knox ° ° ° ° ° HDFS° ° ° ° ° ° Oozie (Hadoop Distributed File System) ° ° ° ° ° ° ° ° ° ° N Data Flow Scheduling DATA MANAGEMENT
Linux Windows On-Premise Cloud Modern: Elastic Integration Platform
Control Plane Data Plane Snaps Designer, Manager, Dashboards Snaplex (Buy or Build) (Multi-tenant cloud service)
“Cloudplex”
“Hadooplex” “Sparkplex”
“Groundplex” Modern Architecture: Hybrid and Elastic
Streams: No data is Cloudplex stored/cached Cloud Apps Secure: 100% Cloud-Based Designer, and Data standards-based Manager, Dashboard Elastic: Scales out & handles data and app Firewall Groundplex integration use cases
On Prem Metadata Databases Apps Data Hadooplex Sparkplex Big Data Challenges Faced: • Cloudification of core business processes • Reduce technical debt from old tech (incl. Tibco, Informatica) • Shift toward self-service to support decentralized org • Hadoop adoption required hand coding MapReduce
Why SnapLogic? • Ease of use – self-service model enables 300+ users • Replacing manual coding of MapReduce with SnapReduce enables re-allocation of resources • Estimate ROI of $3.5M per year based on time savings and reduction of technical debt My experience with SnapLogic has been very good. The tool seems very stable, and Challenges Faced: extremely capable… more powerful than • Significant cloudification project; existing tools – Informatica PowerCenter. including Informatica and MQ Series – not built for the cloud • Sought to reduce reliance on third-parties for integration Why SnapLogic? • Enterprise standard for Cloud Integration, Batch Ingestion & Extraction to HDFS, Stream Processing to HDFS, Data Transformation, & Batch ETL • SnapLogic is supporting projects related to: • Cloud Integration – cloud to cloud and cloud to on premise patterns with Workday, Salesforce, SAP, and others • Marketing CRM/Digital Marketing - integrations between Adobe Marketing Cloud, MDM, and other systems • Big Data Lake – batch and streaming integration patterns for Hadoop-based Big Data Lake. Challenges Faced: • Replacing legacy data warehouse infrastructure • Wanted to improve data quality and eliminate FTP and VPN channels • Growing cloud app adoption • Hadoop adoption – large Cloudera customer Why SnapLogic? • SnapReduce generates Map Reduce without coding ideal for small IT team • Broad selection of pre-built SaaS connectors • Cloudera certification SnapLogic Differentiation
Unified platform for Easy to use, intuitive Hybrid, elastic Extensive library of app & data integration UX architecture connectors
• Batch, streaming, real- • Intuitive for “citizen • Scale-out elastic • Applications, analytics, time integrators” architecture big data, social media • High volume, variety, • Powerful enough for • In the cloud, on • REST, SOAP, JSON, and velocity big data experts without premises or on transform Snaps, and scripting Hadoop cluster more We can do more in two hours with SnapLogic than we could in two days with traditional integration solutions. SnapLogic: Connect Faster
Thank You