DOCSLIB.ORG

Google Data Analytics Solutions Overview

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

Happy Aloha Friday! Workshop #1 Data Analytics & Visualization Daniel Liu, Google hacc.hawaii.gov April 24, 2020 Happy Aloha Friday! Welcome from ETS Marc Masuno Cyber Security Manager hacc.hawaii.gov April 24, 2020 Logistic 01 Welcome from ETS 02 1:00 PM to 2:30 PM 03 About Google Meet 04 Introduce to the Google Team 05 Introduce to the HACC Committee 06 Workshop 06 Q & A Confidential & Proprietary Google Meet Option 1: Join Hangouts Meet Meeting ID meet.google.com/odb-krud-dvu Option 2: Phone Numbers ( US ) 475-329-7374 PIN: 438 611 547# Confidential & Proprietary Google Meet Confidential & Proprietary The Google Team Daniel Liu Amanda Stange Rob Grace Cloud Customer Engineer Account Executive Cloud customer Engineer [email protected] [email protected] [email protected] The HACC Committee Google Data Analytics and Visualization Solutions Overview April 24, 2020 01:00 PM ~ 02:30 PM https://hacc.hawaii.gov/ Daniel Liu, [email protected] Customer Engineer Agenda 01 Data Challenges 02 Our Approach to Data Analytics 03 Modernize Your Data Warehouse 04 Big Data & Hadoop 05 Analyze Streaming Data in Real Time 06 Data Visualization Tools 07 Predictive Analytics & Machine Learning 08 How to Get Start with GCP Confidential & Proprietary Google Mission Statement Organize the world’s information and make it universally accessible and useful Google Mission Statement Organize the world’s information and make it universally accessible and useful Data Volume Growth Digital Information Measurement Unit Data Volume Growth Survey in 2009 ● 2K - A typewritten page ● 5M –The complete works of Shakespeare ● 10M – One minute of high fidelity sound ● 2T – Information generated on YouTube in one day ● 10T – 530,000,000 miles of bookshelves at Library of congress ● 20P – All hard disk drives in 1995 ● 700P – Data of 700,000 companies with Revenues less than $200M ● 1E – Combined Fortune 1000 company database (1P each) ● 1E – Next 9000 world company databases (average 100T each) ● 1Z – 1000E (Zettabyte–Grains of sand on beaches) ● 100Y –Yottabytes – Addressable memory 128 -bit Global Datasphere Survey by IDC ● IDC defines the "global datasphere" as "the quantification of the amount of data created, captured, and replicated across the world." ● Google Mission Statement Organize the world’s information and make it universally accessible and useful Core tenets 1 2 3 4 5 6 If users can’t If they don’t If they don’t If they can’t If there is not If the web is too spell, it’s our know how to know what speak the enough content slow, it’s our problem. form the query, words to use, it’s language, it’s on the web, it’s problem. it’s our problem. our problem. our problem. our problem. Machine Learning is the new ground for gaining competitive edge & creating business value Competitive advantage ranked as top goal of machine-learning projects for 46% of IT leaders & 50% of adopters can quantify ROI 2X more 5X faster 3X faster data-driven decisions execution decisions than others *Source: MIT Survey 2017; n=375 Bain Consulting Study Confidential + Proprietary First Step in This Journey Begins with Data “Every Company will be a Data Company” *Source: Wired, Bloomberg, Fortune, McKinsey Proprietary + Confidential Confidential + Proprietary Data Challenges 01 20 Data is Everything Companies win or lose based on how do they use it Governments make the right and wrong decisions based on the data they processed You make your personal decision based on the data you collected Confidential & Proprietary Data analytics is still too hard <1% <50% Unstructured Structured Data Data * Harvard Business Review magazine; May-June 2017 22 Data complexities Unstructured data accounts for 90% of enterprise data 1011101 0100101 11010101 0111100 10001101 Legacy Data silos Changing view Regulatory Limited skills, applications everywhere on value of data environment hard to recruit *Source: IDC, Wired 23 Challenges with Big Data Projects Complexity of building and Finding value in existing Collaboration within or 1 maintaining a Big Data system 4 7 data very easily across organizations with consistent ease of use 2 Capture and store all data for 5 Reducing the time from 8 Keep your data secure all business functions data collection to action Continuously accommodating Hurdles to innovate and Keep system greater data volumes and new 9 3 6 iterate with Big Data reliable/running data sources Confidential & Proprietary Challenges with Big Data Projects Complexity of building and Finding value in existing Collaboration within or 1 maintaining a Big Data system 4 7 data very easily across organizations with consistent ease of use 2 Capture and store all data for 5 Reducing the time from 8 Keep your data secure all business functions data collection to action Continuously accommodating Hurdles to innovate and Keep system greater data volumes and new 9 3 6 iterate with Big Data reliable/running data sources Confidential & Proprietary If you want to unlock the power of your data, you need a customer data platform, not just new tools. Confidential & Proprietary “ If Your Organization Isn’t Good at Analytics, It’s Not Ready for AI” *Source: Harvard Business Review Proprietary + Confidential Our Approach to Data Analytics0 2 28 15+ Years of Tackling Big Data Problems Open Source Map Google GFS BigTable Dremel Flume Java Spanner Millwheel Dataflow Tensorflow Papers Reduce Google Cloud Products 2002 2004 2005 2006 2008 2010 2012 2014 2015 2016 29 15 Years of Tackling Big Data Problems Open Source Map Google GFS BigTable Dremel Flume Java Spanner Millwheel Dataflow Tensorflow Papers Reduce Google Cloud Products 2002 2004 2005 2006 2008 2010 2012 2014 2015 2016 30 15 Years of Tackling Big Data Problems Open Source Map Google GFS BigTable Dremel Flume Java Spanner Millwheel Dataflow Tensorflow Papers Reduce Google Cloud Products BigQuery Pub/Sub Dataflow Bigtable ML 2002 2004 2005 2006 2008 2010 2012 2014 2015 2016 31 Serverless data analytics From infrastructure to platform for insights Monitoring Analysis and insights Performance Resource tuning provisioning Analysis and Utilization Handling insights improvements growing scale Deployment & configuration Reliability 32 Enterprise Challenges in Data to ML Journey Data Silos Missing Out Lacks How-To and Legacy on Real-Time Predict Business Systems Insights Outcomes Limits decision-making Rear-view approach Depends on guts for and is time consuming causes business anxiety predicting the unknown Proprietary + Confidential Key Solutions Powered by CloudData Silos Data MissingStreaming out PredictivePredicting Warehouseand Legacy Dataon real-time Analytics Analyticsunknown / ML system insights because business limitationsModern Data Processof rear-view Streaming Anticipateoutcomes customer Warehousing which Dataapproach along with batch needs and automate builds foundation for AI data to generate delivery with Machine real-time insights Intelligence Proprietary + Confidential Complete foundation for data lifecycle Data ingestion Reliable streaming Data warehousing Advanced analytics at any scale data pipeline and data lake Cloud Dataproc Cloud Pub/Sub Data Transfer Service Cloud Dataflow BigQuery Cloud ML Engine Google Data Studio (Hadoop, Spark) Cloud Storage Cloud IoT Core Apache Beam Cloud Dataprep Tensorflow Sheets (Trifacta) Cloud Composer 35 (Apache Airflow) Modernize Your Data Warehouse Get all your business data in one place for faster and comprehensive analysis 0336 Data warehousing for AI-driven business 90’s 00’s Now Next Data warehouses BI foundations Cloud data AI foundations warehousing From 1st-gen EDWs, Data warehousing formed BigQuery represents We’re working to make increased data collection the foundation of reporting a fundamentally different BigQuery the foundation and analysis has helped and business intelligence. approach to cloud data for organizations that will build more data-driven warehousing. leverage machine businesses. intelligence in their businesses. 37 Google Cloud Data Warehouse: Four Typical Flows ETL Analyze Cloud Dataflow Data BigQuery Storage Cloud Storage Relational Data Proprietary + Confidential What is BigQuery? Google Cloud Platform’s enterprise Petabyte-scale storage and queries data warehouse for analytics Encrypted, durable and Convenience of standard SQL highly available Fully managed and serverless Real-time analytics on streaming data 39 BigQuery: architecture Serverless. Decoupled storage and compute for maximum flexibility. SQL:2011 Replicated, BigQuery High-available Compliant distributed storage cluster compute Streaming (99.9999999999% durability) (Dremel) REST API ingest Distributed Web UI, CLI memory shuffle tier Client libraries Free bulk In 7 loading Petabit network languages 40 Introducing BigQuery ML Making machine learning accessible 41 BigQuery ML Execute ML initiatives without empowers data moving data from BigQuery analysts and data scientists Iterate on models in SQL in BigQuery to increase development speed Automate model selection, and hypertuning 42 43 Analyze GIS data in BigQuery with familiar SQL Accurate spatial analyses with Geography data type over GeoJSON and WKT formats Support for coreGIS functions – measurements, transforms, constructors, etc... – using familiar SQL 44 Unlock big data for all users with BigQuery & Sheets gsuite.google.com/bq-sheets “For analysts spread across the globe, this is a blessing. They can now collaborate easily with a streamlined flow for sharing their insights.” -- Nikhil Mishra @ Yahoo 45 See your BigQuery data in one click with Data Studio Explorer Tight
Recommended publications
  • Starburst Enterprise on Google Cloud

    Starburst Enterprise on Google Cloud

    SOLUTION BRIEF Starburst Enterprise on Google Cloud The Starburst Enterprise Difference As organizations scale up, Starburst Enterprise on Google Cloud drives Available on the Google Cloud Marketplace, the better business outcomes, consistency, and reliability, delighting your data Starburst Enterprise platform is a fully supported, engineers and scientists. Teams look to Starburst Enterprise on Google Cloud production-tested, enterprise-grade distribution for expertise & constant fine-tuning that results in overall lower costs & faster of the open source Trino MPP SQL query engine. time-to-insights: Starburst integrates Google’s scalable cloud storage and computing services with a more Performance: stable, secure, efficient, and cost-effective way Includes the latest optimizations; Starburst Cached Views available for to query all your enterprise data, wherever it frequently accessed data; stable code that minimizes failed queries. resides. Leading organizations across multiple industries Connectivity rely on Starburst Enterprise and Google. 40+ supported enterprise connectors; high performance connectors for Oracle, Teradata, Snowflake, IBM DB2, Delta Lake, and many more. Analytics Anywhere Designed for the separation of storage and Security compute, Trino is ideal for querying data residing in multiple systems, from cloud data lakes to Role-based access control (via Apache Ranger); Kerberos, OKTA, LDAP legacy data warehouses. Deployed via Google integration; data encryption & masking; query auditing to see who is doing what. Kubernetes Engine (GKE), Starburst Enterprise on Google Cloud enables the user to run analytic Management queries across Google Cloud data sources and on-prem systems such as Teradata, Oracle, Enhanced tools for configuration, auto scaling, and Starburst Insights and others via Trino clusters. Within a single monitoring dashboards; easy deployment on Google platforms.
  • Comparative Analysis of NLP Models for Google Meet Transcript Summarization

    Comparative Analysis of NLP Models for Google Meet Transcript Summarization

    EasyChair Preprint № 5404 Comparative Analysis of NLP models for Google Meet Transcript Summarization Yash Agrawal, Atul Thakre, Tejas Tapas, Ayush Kedia, Yash Telkhade and Vasundhara Rathod EasyChair preprints are intended for rapid dissemination of research results and are integrated with the rest of EasyChair. April 28, 2021 Comparative Analysis of NLP models for Google Meet Transcript Summarization Yash Agrawal1,a) Atul Thakre1,b) Tejas Tapas1,c) Ayush Kedia1,d) Yash Telkhade1,e) Vasundhara Rathod1,f) 1) Computer Science & Engineering, Shri Ramdeobaba College of Engineering and Management, Nagpur, India a) [email protected] , +91 7083645470 b) [email protected] , +91 8956758226 c) [email protected] , +918380073925 d) [email protected] , +91 8459811323 e) [email protected] , +91 9021067230 f) [email protected], +918055225407 Abstract. Manual transcription and summarization is a cumbersome process necessitating the development of an efficient automatic text summarization technique. In this study, a Chrome extension is used for making the process of transcription hassle- free. It uses the text summarization technique to generate concise and succinct matter. Also, the tool is accessorized using Google Translation, to convert the processed text into users' desired language. This paper illustrates, how captions can be traced from the online meetings, corresponding to which, meeting transcript is sent to the backend where it is summarized using an NLP model. It also walks through three different NLP models and presents a comparative study among them. The NLTK model utilizes the sentence ranking technique for extractive summarization. Word Embedding model uses pre-trained Glove Embeddings for extractive summarization. The T5 model performs abstractive summarization using transformer architecture.
  • Google Certified Professional - Cloud Architect.Exam.57Q

    Google Certified Professional - Cloud Architect.Exam.57Q

    Google Certified Professional - Cloud Architect.exam.57q Number : GoogleCloudArchitect Passing Score : 800 Time Limit : 120 min https://www.gratisexam.com/ Google Certified Professional – Cloud Architect (English) https://www.gratisexam.com/ Testlet 1 Company Overview Mountkirk Games makes online, session-based, multiplayer games for the most popular mobile platforms. Company Background Mountkirk Games builds all of their games with some server-side integration, and has historically used cloud providers to lease physical servers. A few of their games were more popular than expected, and they had problems scaling their application servers, MySQL databases, and analytics tools. Mountkirk’s current model is to write game statistics to files and send them through an ETL tool that loads them into a centralized MySQL database for reporting. Solution Concept Mountkirk Gamesis building a new game, which they expect to be very popular. They plan to deploy the game’s backend on Google Compute Engine so they can capture streaming metrics, run intensive analytics, and take advantage of its autoscaling server environment and integrate with a managed NoSQL database. Technical Requirements Requirements for Game Backend Platform 1. Dynamically scale up or down based on game activity 2. Connect to a managed NoSQL database service 3. Run customize Linux distro Requirements for Game Analytics Platform 1. Dynamically scale up or down based on game activity 2. Process incoming data on the fly directly from the game servers 3. Process data that arrives late because of slow mobile networks 4. Allow SQL queries to access at least 10 TB of historical data 5. Process files that are regularly uploaded by users’ mobile devices 6.
  • Quick Deployment Guide for Enabling Remote Working with Hangouts Meet

    Quick Deployment Guide for Enabling Remote Working with Hangouts Meet

    Enabling Remote Working with Hangouts Meet and Hangouts Chat: A quick deployment guide Work Transformation: Productivity & Collaboration Contents About this guide 2 1. Requirements 4 1.1 General requirements 4 1.2 Network requirements 4 1.3 Optimize Meet traffic for remote workers 5 2. Set up G Suite 7 2.1 Enroll in G Suite 7 2.2 Verify your domain 12 2.3 Provision your users 14 Step 1: Open the user management interface 14 Step 2: Download the CSV template file 15 Step 3: Add your users to the CSV template 16 Step 4: Upload the CSV file and provision your users 17 Troubleshooting upload errors 17 2.4 Distribute user credentials 18 2.5 Disable out-of-scope G Suite applications 19 2.6 Configure Meet 20 2.7 Configure Chat 22 2.8 Securing your setup 22 3. Appendix: User guide 22 3.1 Documentation hub 22 3.2 Meet/Calendar integrations 23 Schedule your meetings with Google Calendar 23 Deploy the Microsoft Outlook Meet plug-in 24 1 About this guide Highlights To provide companies with a deployment plan and guide to quickly Purpose enable remote working using Google Meet and Google Chat. Intended IT administrators audience Key That the audience has the required access and rights documented in the assumptions general requirements​. This document provides guidance for quickly bootstrapping your company with the adoption of Hangouts Meet (for video conferencing) and Hangouts Chat (for instant messaging). Since Hangouts Meet and Hangouts Chat are part of the G Suite offering, this guide will walk you through the steps required to create a G Suite account, configure the billing, create the users, secure your setup, and teach your users how to use the communication suite.
  • Create Live Dashboards with Google Data Studio

    Create Live Dashboards with Google Data Studio

    Make Your Library's Data Accessible and Usable Create Live Dashboards with Google Data Studio Kineret Ben-Knaan & Andrew Darby | University of Miami Libraries INTRODUCTION OBJECTIVES AND GOALS STRATEGY AND METHODS PROJECT STATUS This poster presents the implementation of a The aim of the project is to implement a shared and Customized dashboards have been built and shared with collaborative approach and solution for data gathering, straightforward data collection platform, which will harvest Our solution involves the following: several library departments. So far, we have connected data consolidation and most importantly, data multiple, diverse data sources and present statistics in a or imported manually the following isolated data sources ● Move to shared Google Sheets and Forms for any accessibility through a live connection to free Google clear and visually engaging manner. into Google Data Studio: Data Studio dashboards. local departmental data collection, then connect these Our key goals are: sheets to Google Data Studio, a free tool that allows Metric/Data Source by type of Google Data Studio Connector Units at the University of Miami Libraries (UML) have ● To facilitate the use of data from isolated data sources, users to create customized and shareable data long been engaged in data collection activities. Data from Metrics Data Sources Real-time connection Imported manually into visualization dashboards. and Systems & automatically Google Data Studio (on instructional sessions, consultations and all types of user not only to encourage evidence-based decision updated a weekly bases) ● Import and connect other library data sources to interactions have routinely been gathered. Other making, but also to better communicate how UM Instructional & Workshops Google Sheets Yes assessment measures, including surveys and user Libraries’ activities support student learning and faculty Google Data Studio.
  • W Shekatkar Committee Report W Atmanirbhar Bharat Abhiyan W

    W Shekatkar Committee Report W Atmanirbhar Bharat Abhiyan W

    MONTHLY MAGAZINE FOR TNPSC EXAMS MAY–2020 w Atmanirbhar Bharat Abhiyan w Cleanest City List w Shekatkar Committee Report w Konark Sun Temple w Char Dham Project w Samagra Shiksha Abhiyan VETRII IAS STUDY CIRCLE TNPSC CURRENT AFFAIRS MAY - 2020 An ISO 9001 : 2015 Institution | Providing Excellence Since 2011 Head Office Old No.52, New No.1, 9th Street, F Block, 1st Avenue Main Road, (Near Istha siddhi Vinayakar Temple), Anna Nagar East – 600102. Phone: 044-2626 5326 | 98844 72636 | 98844 21666 | 98844 32666 Branches SALEM KOVAI No.189/1, Meyanoor Road, Near ARRS Multiplex, (Near Salem New No.347, D.S.Complex (3rd floor), Nehru Street,Near Gandhipuram bus Stand), Opp. Venkateshwara Complex, Salem - 636004. Central Bus Stand, Ramnagar, Kovai - 9 0427-2330307 | 95001 22022 75021 65390 Educarreerr Location Vivekanandha Educational Institutions for Women, Elayampalayam, Tiruchengode - TK Namakkal District - 637 205. 04288 - 234670 | 91 94437 34670 Patrician College of Arts and Science, 3, Canal Bank Rd, Gandhi Nagar, Opposite to Kotturpuram Railway Station, Adyar, Chennai - 600020. 044 - 24401362 | 044 - 24426913 Sree Saraswathi Thyagaraja College Palani Road, Thippampatti, Pollachi - 642 107 73737 66550 | 94432 66008 | 90951 66009 www.vetriias.com My Dear Aspirants, Greetings to all of you! “What we think we become” Gautama Buddha. We all have dreams. To make dreams come into reality it takes a lot of determination, dedication, self discipline and continuous effort. We at VETRII IAS Study Circle are committed to provide the right guidance, quality coaching and help every aspirants to achieve his or her life’s cherished goal of becoming a civil servant.
  • Regeldokument

    Regeldokument

    Master’s degree project Source code quality in connection to self-admitted technical debt Author: Alina Hrynko Supervisor: Morgan Ericsson Semester: VT20 Subject: Computer Science Abstract The importance of software code quality is increasing rapidly. With more code being written every day, its maintenance and support are becoming harder and more expensive. New automatic code review tools are developed to reach quality goals. One of these tools is SonarQube. However, people keep their leading role in the development process. Sometimes they sacrifice quality in order to speed up the development. This is called Technical Debt. In some particular cases, this process can be admitted by the developer. This is called Self-Admitted Technical Debt (SATD). Code quality can also be measured by such static code analysis tools as SonarQube. On this occasion, different issues can be detected. The purpose of this study is to find a connection between code quality issues, found by SonarQube and those marked as SATD. The research questions include: 1) Is there a connection between the size of the project and the SATD percentage? 2) Which types of issues are the most widespread in the code, marked by SATD? 3) Did the introduction of SATD influence the bug fixing time? As a result of research, a certain percentage of SATD was found. It is between 0%–20.83%. No connection between the size of the project and the percentage of SATD was found. There are certain issues that seem to relate to the SATD, such as “Duplicated code”, “Unused method parameters should be removed”, “Cognitive Complexity of methods should not be too high”, etc.
  • Trifacta Data Preparation for Amazon Redshift and S3 Must Be Deployed Into an Existing Virtual Private Cloud (VPC)

    Trifacta Data Preparation for Amazon Redshift and S3 Must Be Deployed Into an Existing Virtual Private Cloud (VPC)

    Install Guide for Data Preparation for Amazon Redshift and S3 Version: 7.1 Doc Build Date: 05/26/2020 Copyright © Trifacta Inc. 2020 - All Rights Reserved. CONFIDENTIAL These materials (the “Documentation”) are the confidential and proprietary information of Trifacta Inc. and may not be reproduced, modified, or distributed without the prior written permission of Trifacta Inc. EXCEPT AS OTHERWISE PROVIDED IN AN EXPRESS WRITTEN AGREEMENT, TRIFACTA INC. PROVIDES THIS DOCUMENTATION AS-IS AND WITHOUT WARRANTY AND TRIFACTA INC. DISCLAIMS ALL EXPRESS AND IMPLIED WARRANTIES TO THE EXTENT PERMITTED, INCLUDING WITHOUT LIMITATION THE IMPLIED WARRANTIES OF MERCHANTABILITY, NON-INFRINGEMENT AND FITNESS FOR A PARTICULAR PURPOSE AND UNDER NO CIRCUMSTANCES WILL TRIFACTA INC. BE LIABLE FOR ANY AMOUNT GREATER THAN ONE HUNDRED DOLLARS ($100) BASED ON ANY USE OF THE DOCUMENTATION. For third-party license information, please select About Trifacta from the Help menu. 1. Quick Start . 4 1.1 Install from AWS Marketplace . 4 1.2 Upgrade for AWS Marketplace . 7 2. Configure . 8 2.1 Configure for AWS . 8 2.1.1 Configure for EC2 Role-Based Authentication . 14 2.1.2 Enable S3 Access . 16 2.1.2.1 Create Redshift Connections 28 3. Contact Support . 30 4. Legal 31 4.1 Third-Party License Information . 31 Page #3 Quick Start Install from AWS Marketplace Contents: Product Limitations Internet access Install Desktop Requirements Pre-requisites Install Steps - CloudFormation template SSH Access Troubleshooting SELinux Upgrade Documentation Related Topics This guide steps through the requirements and process for installing Trifacta® Data Preparation for Amazon Redshift and S3 through the AWS Marketplace.
  • Economic and Social Impacts of Google Cloud September 2018 Economic and Social Impacts of Google Cloud |

    Economic and Social Impacts of Google Cloud September 2018 Economic and Social Impacts of Google Cloud |

    Economic and social impacts of Google Cloud September 2018 Economic and social impacts of Google Cloud | Contents Executive Summary 03 Introduction 10 Productivity impacts 15 Social and other impacts 29 Barriers to Cloud adoption and use 38 Policy actions to support Cloud adoption 42 Appendix 1. Country Sections 48 Appendix 2. Methodology 105 This final report (the “Final Report”) has been prepared by Deloitte Financial Advisory, S.L.U. (“Deloitte”) for Google in accordance with the contract with them dated 23rd February 2018 (“the Contract”) and on the basis of the scope and limitations set out below. The Final Report has been prepared solely for the purposes of assessment of the economic and social impacts of Google Cloud as set out in the Contract. It should not be used for any other purposes or in any other context, and Deloitte accepts no responsibility for its use in either regard. The Final Report is provided exclusively for Google’s use under the terms of the Contract. No party other than Google is entitled to rely on the Final Report for any purpose whatsoever and Deloitte accepts no responsibility or liability or duty of care to any party other than Google in respect of the Final Report and any of its contents. As set out in the Contract, the scope of our work has been limited by the time, information and explanations made available to us. The information contained in the Final Report has been obtained from Google and third party sources that are clearly referenced in the appropriate sections of the Final Report.
  • Real-Time High-Resolution Background Matting

    Real-Time High-Resolution Background Matting

    Real-Time High-Resolution Background Matting Shanchuan Lin* Andrey Ryabtsev* Soumyadip Sengupta Brian Curless Steve Seitz Ira Kemelmacher-Shlizerman University of Washington flinsh,ryabtsev,soumya91,curless,seitz,[email protected] Zoom input and background shot Zoom with new background Our Zoom plugin with new background Figure 1: Current video conferencing tools like Zoom can take an input feed (left) and replace the background, often introducing artifacts, as shown in the center result with close-ups of hair and glasses that still have the residual of the original background. Leveraging a frame of video without the subject (far left inset), our method produces real-time, high-resolution background matting without those common artifacts. The image on the right is our result with the corresponding close-ups, screenshot from our Zoom plugin implementation. Abstract ment can enhance privacy, particularly in situations where a user may not want to share details of their location and We introduce a real-time, high-resolution background re- environment to others on the call. A key challenge of this placement technique which operates at 30fps in 4K resolu- video conferencing application is that users do not typically tion, and 60fps for HD on a modern GPU. Our technique is have access to a green screen or other physical props used to based on background matting, where an additional frame of facilitate background replacement in movie special effects. the background is captured and used in recovering the al- pha matte and the foreground layer. The main challenge is While many tools now provide background replacement to compute a high-quality alpha matte, preserving strand- functionality, they yield artifacts at boundaries, particu- level hair details, while processing high-resolution images larly in areas where there is fine detail like hair or glasses in real-time.
  • Portable Stateful Big Data Processing in Apache Beam

    Portable Stateful Big Data Processing in Apache Beam

    Portable stateful big data processing in Apache Beam Kenneth Knowles Apache Beam PMC Software Engineer @ Google https://s.apache.org/ffsf-2017-beam-state [email protected] / @KennKnowles Flink Forward San Francisco 2017 Agenda 1. What is Apache Beam? 2. State 3. Timers 4. Example & Little Demo What is Apache Beam? TL;DR (Flink draws it more like this) 4 DAGs, DAGs, DAGs Apache Beam Apache Flink Apache Cloud Hadoop Apache Apache Dataflow Spark Samza MapReduce Apache Apache Apache (paper) Storm Gearpump Apex (incubating) FlumeJava (paper) Heron MillWheel (paper) Dataflow Model (paper) 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 Apache Flink local, on-prem, The Beam Vision cloud Cloud Dataflow: Java fully managed input.apply( Apache Spark Sum.integersPerKey()) local, on-prem, cloud Sum Per Key Apache Apex Python local, on-prem, cloud input | Sum.PerKey() Apache Gearpump (incubating) ⋮ ⋮ 6 Apache Flink local, on-prem, The Beam Vision cloud Cloud Dataflow: Python fully managed input | KakaIO.read() Apache Spark local, on-prem, cloud KafkaIO Apache Apex ⋮ local, on-prem, cloud Apache Java Gearpump (incubating) class KafkaIO extends UnboundedSource { … } ⋮ 7 The Beam Model PTransform Pipeline PCollection (bounded or unbounded) 8 The Beam Model What are you computing? (read, map, reduce) Where in event time? (event time windowing) When in processing time are results produced? (triggers) How do refinements relate? (accumulation mode) 9 What are you computing? Read ParDo Grouping Composite Parallel connectors to Per element Group
  • What's New for Google in 2020?

    What's New for Google in 2020?

    Kevin A. McGrail [email protected] What’s new for Google in 2020? Introduction Kevin A. McGrail Director, Business Growth @ InfraShield.com Google G Suite TC, GDE & Ambassador https://www.linkedin.com/in/kmcgrail About the Speaker Kevin A. McGrail Director, Business Growth @ InfraShield.com Member of the Apache Software Foundation Release Manager for Apache SpamAssassin Google G Suite TC, GDE & Ambassador. https://www.linkedin.com/in/kmcgrail 1Q 2020 STORY TIME: Google Overlords, Pixelbook’s Secret Titan Key, & Googlesplain’ing CES Jan 2020 - No new new hardware was announced at CES! - Google Assistant & AI Hey Google, Read this Page Hey Google, turn on the lights at 6AM Hey Google, Leave a Note... CES Jan 2020 (continued) Google Assistant & AI Speed Dial Interpreter Mode (Transcript Mode) Hey Google, that wasn't for you Live Transcripts Hangouts Meet w/Captions Recorder App w/Transcriptions Live Transcribe Coming Next...: https://mashable.com/article/google-translate-transcription-audio/ EXPERT TIP: What is Clipping? And Whispering! Streaming Games - Google Stadia Android Tablets No more Android Tablets? AI AI AI AI AI Looker acquisition for 2.6B https://www.cloudbakers.com/blog/why-cloudbakers-loves-looker-for-business-intelligence-bi From Thomas Kurian, head of Google Cloud: “focusing on digital transformation solutions for retail, healthcare, financial services, media and entertainment, and industrial and manufacturing verticals. He highlighted Google's strengths in AI for each vertical, such as behavioral analytics for retail,