Cassie Kozyrkov, Chief Decision Scientist, Google

Total Page:16

File Type:pdf, Size:1020Kb

Cassie Kozyrkov, Chief Decision Scientist, Google Cassie Kozyrkov, Chief Decision Scientist, Google @quaesita @quaesita @quaesita @quaesita @quaesita @quaesita What is machine learning? @quaesita @quaesita @quaesita @quaesita @quaesita @quaesita @quaesita @quaesita Why do teams fail at machine learning? @quaesita Machine Learning Data Algorithms Models Predictions Ingredients Appliances Recipes Dishes @quaesita How does it work? @quaesita @quaesita @quaesita @quaesita Support Vector Classifier Decision Tree Neural Network @quaesita Label: Y or N @quaesita Label: Y or N @quaesita Label: Y or N Applied machine learning is a team sport @quaesita Meet the team Decision-maker @quaesita Data Decisions Is it tasty? Y or N ? Inputs Outputs @quaesita Meet the team Decision-maker Software engineer @quaesita Programmer Model Decisions Y or N ? Ada, Countess of Lovelace, 1815-1852 Expert Recipe Outputs @quaesita Meet the team Decision-maker Software engineer Data engineer @quaesita Data Programmer Models Decisions Is it tasty? Y or N ? Ingredients Expert Recipes Outputs @quaesita Meet the team Decision-maker Software engineer Data engineer Descriptive analyst @quaesita Data Analyst Models Decisions Y or N ? Is it tasty? Karl Friedrich Gauss 1777-1855 Y or N ? Florence Nightingale 1820-1910 Ingredients Expert Recipes Outputs @quaesita Meet the team Decision-maker Software engineer Data engineer Descriptive analyst Machine learning engineer @quaesita Data Algorithms Models Decisions Y or N ? Is it tasty? Y or N ? Y or N ? Ingredients Tools Recipes Outputs @quaesita Data Algorithms Models Decisions SV Classifier Decision Tree Neural Network Ingredients Tools Recipes Outputs @quaesita Meet the team Decision-maker Software engineer Data engineer Descriptive analyst Machine learning engineer (Researcher) @quaesita Data Algorithms Models Decisions SV Classifier Is it tasty? Decision Tree Neural Network Ingredients Tools Recipes Outputs @quaesita Data Algorithms Models Decisions SV Classifier Y or N ? Is it tasty? Decision Tree Y or N ? Neural Network Y or N ? Ingredients Tools Recipes Outputs @quaesita Data Models Decisions Is it tasty? Ingredients Recipes Outputs @quaesita Data Models Decisions Is it tasty? Ingredients Recipes Outputs @quaesita Data Models Decisions Is it tasty? Y Y N Y N N Y Y N Y N N Y Y N Y N N Ingredients Recipes Outputs @quaesita Meet the team Decision-maker Software engineer Data engineer Descriptive analyst Machine learning engineer (Researcher) Statistician @quaesita Training Validation Test Build Final Taste Make Recipes Casual Taste Test Add to Menu Test @quaesita Training Validation Test Build 75% 90% 85% Final Taste Make Recipes Casual Taste Test Add to Menu Test @quaesita Training Validation Test Build 75% 90% Pass 85% Final Taste Make Recipes Casual Taste Test Add to Menu Test @quaesita Training Validation Test Build 75% 90% Pass 85% Final Taste Make Recipes Casual Taste Test Add to Menu Test @quaesita Meet the team Decision-maker Software engineer Data engineer Descriptive analyst Machine learning engineer (Researcher) Statistician (Reliability engineer) @quaesita What is data science? @quaesita A map of data science? SQL Python R Descriptive Machine Statistical Analytics Learning Inference @quaesita A map of data science? Histogram Neural network Student’s t-test Descriptive Machine Statistical Analytics Learning Inference @quaesita A map of data science! None Many Few Descriptive Machine Statistical Analytics Learning Inference @quaesita A map of data science! Get inspired Make a recipe Decide wisely Descriptive Machine Statistical Analytics Learning Inference @quaesita Seeing with Google Photos [beach] Is machine learning right for you? @quaesita @quaesita Meet the team Decision-maker Software engineer Data engineer Descriptive analyst Machine learning engineer (Researcher) Statistician (Reliability engineer) (Ethicist) @quaesita Reliable or unreliable? @quaesita Technology is a lever. @quaesita Decisions at scale need skilled leaders. @quaesita Design metrics that can’t be gamed. @quaesita Warning! Decision-makers make or break a data science project. Please train them. @quaesita Set performance criteria up front. @quaesita Would you stay at this hotel? @quaesita Data-inspired or data-driven? @quaesita Frame the decision context 1. What if you get no information? 2. What if you get full information? 3. What if you get partial information? Role: Decision-maker @quaesita Get help! From a teammate. From data. Meet the team Decision-maker Software engineer Data engineer Descriptive analyst Machine learning engineer (Researcher) Statistician (Reliability engineer) (Ethicist) (Qualitative expert) @quaesita Help from the social sciences 1. Psychology a. Judgment & decision-making b. Social psychology c. Neuroeconomics 2. Economics a. Experimental game theory b. Behavioral economics 3. Managerial sciences Role: Qualitative expert @quaesita Get help! From a teammate. From data. Warning! Datasets used for inspiration can’t be used for rigorous decision-making. @quaesita Have your cake and eat it too. Split your data. ALL OF IT Empower everyone to look at data. @quaesita You are already a data analyst. Who, me? Yes, you. @quaesita @quaesita @quaesita Split your data. Inspiration (everyone) ALL OF IT Rigor (experts only) Statistician “Does it actually work?” “Let’s come to conclusions safely.” Empower everyone to look at data. Be careful coming to conclusions. @quaesita Inspiration is cheap, rigor takes expertise. @quaesita Warning! Avoid rigor for rigor’s sake. @quaesita Meet the team Decision-maker Software engineer Data engineer Descriptive analyst Machine learning engineer (Researcher) Statistician (Reliability engineer) (Ethicist) (Qualitative expert) Data science manager @quaesita Rigor should start with the decision-maker. @quaesita Why trust machine learning? @quaesita Why trust your student? @quaesita Catch memorization! Don’t let overfitting take you by surprise; expect performance on examples you’ve studied from to be too good to be true. @quaesita Testing keeps you safe. It’s the key to responsible ML and AI. Make sure your solution actually works on relevant, new data. @quaesita Never forget the basics of learning and teaching! @quaesita Your ingredients matter. @quaesita “The world represented by your training data is the only world you can expect to succeed in.” @quaesita Where do data come from? @quaesita The future of work @quaesita We’re all being promoted. @quaesita The future of work is creativity. @quaesita Thank you @quaesita Summary Decision intelligence engineering is a team sport. The decision-maker’s role is vital in machine learning. Data-driven decisions require up front context framing. Ensure that your metrics can’t be gamed. Splitting your data lets you have your cake and eat it too. Inspiration is cheap, rigor takes expertise. Rigor should come from the decision-maker. Don’t reinvent the wheel with research. Test carefully on relevant, new examples. @quaesita.
Recommended publications
  • Decision Intelligence in Public Health – DIONE Stolk, J. and Nyon, S
    22nd International Congress on Modelling and Simulation, Hobart, Tasmania, Australia, 3 to 8 December 2017 mssanz.org.au/modsim2017 Decision Intelligence in Public Health – DIONE Jacob Stolk1 and Simone Nyon1 1 Dione Complex Systems, Gold Coast, Queensland, Australia Email: [email protected] Abstract: Public health surveillance is the ongoing systematic collection, analysis, interpretation, and dissemination of health data for the planning, implementation, and evaluation of public health action. To achieve effective public health interventions, it is pivotal to analyse and interpret the vast amounts of data collected by surveillance systems to enable good understanding of all factors having an impact on health. For example, we can consider child protection, which is an important public health issue. Often relatively extensive data exist on families in official statistics, research reports, social services reports, school and medical records, etc. However, these data are dispersed and hard if not impossible to relate and compare. This leads to numerous interventions that are conducted without adequate knowledge of target families that are to benefit from these interventions. Inadequate knowledge also leads to lack of intervention where it is needed, such as undetected cases of child abuse. In many cases the basic data that are needed for intervention decisions exist, but are not available to decision makers due to inadequate communication and lack of data integration, analysis and interpretation. Chronic condition management is another area where extensive disparate data exist from statistics and various health services and intervention agencies. In this area too there are numerous organizations offering services and a great need to better coordinate these services to achieve better outcomes for patients and also to reduce soaring costs of the healthcare system.
    [Show full text]
  • Introduction to Decision Intelligence - Towards Data Science
    10/26/2019 Introduction to Decision Intelligence - Towards Data Science Listen to this story --:-- 18:26 Introduction to Decision Intelligence A new discipline for leadership in the AI era Cassie Kozyrkov Follow Aug 3 · 13 min read Curious to know what the psychology of avoiding lions on the savannah has in common with responsible AI leadership and the challenges of designing data warehouses? Welcome to decision intelligence! Source: xijian/Getty Decision intelligence is a new academic discipline concerned with all aspects of selecting between options. It brings together the best of applied data science, social science, and managerial science into a unified field that helps people use data to improve their lives, their businesses, and the world around them. It’s a vital science for the AI era, covering https://towardsdatascience.com/introduction-to-decision-intelligence-5d147ddab767 1/12 10/26/2019 Introduction to Decision Intelligence - Towards Data Science the skills needed to lead AI projects responsibly and design objectives, metrics, and safety-nets for automation at scale. Decision intelligence is the discipline of turning information into better actions at any scale. Let’s take a tour of its basic terminology and concepts. The sections are designed to be friendly to skim-reading (and skip-reading too, that’s where you skip the boring bits… and sometimes skip the act of reading entirely). What’s a decision? Data are beautiful, but it’s decisions that are important. It’s through our decisions — our actions — that we affect the world around us. We define the word “decision” to mean any selection between options by any entity, so the conversation is broader than MBA-style dilemmas (like whether to open a branch of your business in London).
    [Show full text]
  • Demystifying AI in Cybersecurity
    WHITE PAPER | Demystifying AI in Cybersecurity Demystifying AI in Cybersecurity A look at the technology, myth vs. reality, and how it benefits the cybersecurity industry Table of Contents Introduction ...................................................................................................................3 AI: What it is and What it isn’t .......................................................................................3 Machine Learning: the Next Step ...................................................................................3 Deep Learning: Putting a Finer Point on it ......................................................................3 Plenty of Promise, for Better or Worse ..........................................................................3 The Difference AI Makes in Cybersecurity .....................................................................4 Malware Detection ....................................................................................................4 Classification and Scoring .........................................................................................4 Phishing Detection ....................................................................................................4 Key Requirements for AI/ML-based Security ..................................................................4 Data ..........................................................................................................................4 Speed ........................................................................................................................5
    [Show full text]
  • The Dynamics Duo: Microsoft Dynamics GP and AP Automation
    Summer 2015 Self-service Business Intelligence for GP: What does that mean for me? The Dynamics Duo: Microsoft Dynamics GP and AP Automation Managing Long-Term Investment Assets in Dynamics GP GP OPTIMIZER I On the Cover GP Optimizer Magazine 5 Published by Rockton Software Self-service Business PO Box 921 Lafayette, CO 80026 Intelligence for GP: Editor: Mark Rockwell Welcome to the Summer 2015 Edition of the GP Optimizer Magazine. Our original goal [email protected] What does that mean for me? of The GP Optimizer Magazine was to reach 15,000 Microsoft Dynamics GP Users. Last Design: Lori Hartmann, Feline Graphics publication, we exceeded this goal by reaching over 50,000 users. [email protected] Advertising Inquiries: Steven Solomon The GP Optimizer Magazine contains articles written by Microsoft Dynamics GP Add-On [email protected] Partners; these articles are focused on making your investment in Microsoft Dynamics Editorial Inquiries: Steven Solomon GP more worthwhile by solving an issue that you might be experiencing. [email protected] Rockton Software has been in the Microsoft Dynamics GP Channel for over 15 years. You may have seen us at Convergence dressed up as bartenders, pirates, Vikings, or The GP Optimizer Magazine is published by cavemen. Regardless of our crazy costumes, we’ve established ourselves as fervent Rockton Software, with principal offices in Lafayette, CO. If you wish to receive this publication, please go to supporters of the greater GP Channel, and we want you to excel in business by www.rocktonsoftware.com and click on the GP leveraging other tools and knowledge from our friends in the community.
    [Show full text]
  • Lorien Pratt, Ph.D. Curriculum Vita
    Lorien Pratt, Ph.D. Curriculum Vita Sunnyvale, CA USA +1.303.589.7476 [email protected] Blog: www.lorienpratt.com SUMMARY Professional machine learning and decision intelligence consultant, developer, entrepreneur, writer, evangelist. Author of Link: How Decision Intelligence Connects Data, Actions, and Outcomes for a Better World. Co-organizer of the annual Responsible AI/DI Summit. Two TEDx talks. Inventor of machine learning inductive transfer. Co-Founder and Chief Scientist of Quantellia, LLC. Co-inventor and evangelist for Decision Intelligence. Over three decades delivering applied machine learning solutions, including for the Human Genome Project, the US Department of Energy, budget analysis for a $100 million annual spend for the Administrative Office (AO) of the US Courts, award-winning forensic analysis for the Colorado Bureau of Investigation, RBS, DARPA, and many more. EDUCATION Ph.D., Computer Science, Rutgers University, 5/93. Thesis: Transferring previously learned back-propagation neural networks to new learning tasks. M.S., Computer Science, Rutgers University, 5/88. A.B., Computer Science, Dartmouth College, 5/83. CONSULTING A variety of engagements including the following: Regular invited speaker to user groups, executive roundtables, keynotes, webinars, and workshops for organizations and companies like SAP, HP, Unisys, the TM Forum, Comptel, SAS. Machine learning consulting on a wide variety of projects for cyber security, web advertising, financial prediction, medical decision making, cable company capital expenditure, LIDAR image analysis, and more. Five-star rated consultant on the Experfy platform (www.experfy.com), from the Harvard Innovation Laboratory. Strategic consulting in decision intelligence, analytics, and telecom OSS/BSS to companies and organizations like the US Department of Defense, Solvina, ConceptWave, the Colorado Bureau of Investigation (CBI), Sun, IBM, Comptel, SAP, HP, Amdocs, Telcordia (now Ericsson).
    [Show full text]
  • Decision Intelligence: a Two-Part Course
    Decision Intelligence: A two-part course Every decade or so, a new business discipline is born. Did you know that there was a time before project planning? Before Business Intelligence? Before Business Process Management? Today, Decision Intelligence is the next such breakthrough: it is a proven approach that has saved hundreds of millions of dollars for organizations worldwide. It brings the best of Big Data, Predictive Analytics, Deep Learning, and Systems Modeling to every role in the organization, including executive management, program managers, and quantitative modeling experts. Join us for workshop Getting Started with Decision Intelligence plus, optionally, Decision Intelligence Hands-On. You’ll learn: - How to answer the question: “If I make this decision today, what will be the outcome tomorrow?” - How to create a decision collaboration team to design, test, and update your organization’s most important decisions, creating continuous improvement and organizational learning - How to combine intangible factors , like employee engagement and customer experience, with tangible ones, like cost of goods and closed sales - Two secrets that, together, will increase your team’s decision-making intelligence tenfold - How to make decisions that have multiple outcomes - How to avoid the most common trap that causes Big Data projects to fail, and how to radically accelerate the value that you receive from all data, big and small - The three secrets of small data, and how you can use it to supercharge your decision intelligence - How to ensure
    [Show full text]
  • A Global Civic Debate FUTURE SOCIETY on Governing the Rise of Artificial Intelligence
    THE THE A Global Civic Debate FUTURE SOCIETY on Governing the Rise of Artificial Intelligence Report 2018 O1O1●●●●●●●●●O1OOO11O1O●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●. OOO 11OO1O1OO1OOOO● 1●●●1 ●O111O1O1 O1●●●●●●●●●●●●11O1OOO111O1O1O11 1OO●●●11OO1O1OO1OO●●●●●●●●●●●●●● ●●●●●●●●●OOOO1O1OO11O11O1111O1● ●●O11O11O1OO1O11OO1O1O111O1OOO1 111O●●●O1O11OOOO1O●●●●OOOOO1OOO OO1OO1OOOOOO1●●●111O1●●●●1O11OO. O11O1111O11OOO1OO11OO.•●●●•●●● OO 1●●●●1O11OOOO1OOOOO● OOOO11O11O 1OO1O111●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●O11OO11O1OO1O11OOO●●● ●●●●●●●●●●●●11OO1OOOOOO1OOO1OOO. ●●●●●●●●●●●●●●●.. 11OO1O1O11●●●●●● ●●●●●● ● ●●●●●●●●●●●●●●●OOO1OO. 11OOOO1O111O1OO. ●●●●●●O11OO1 ●●● 1OO ●OO11O1111●●●●●●1OOO1OOOOOO 1O●●●11O11O1111O11●●●●●●1O11OO 11OO1O1O111OO1●●●•••●1OO11O1OO1 O11O111OO11OO111OO1●●●OO●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●111O1OOO11O1OOOO11OO1O1OO1 OOOOO●●●1OO1OO11O1OO1O111OO11O1 “The Future Society, a Global Civic Debate” translated by the binary table of the American THE AI INITIATIVE Standard Code for Information Interchange. Contents : Executive Sumary p. 4 Our Promise p. 15 Key insights p. 17 Top recommandations p. 51 Tales from Imaginaries p. 65 Our Methodology p. 76 Debate Health p. 78 Beyond the Global Civic Debate p. 80 Authors : Nicolas Miailhe Arohi Jain Caroline Jeanmaire Yolanda Lannquist Josephine Png The Future Society © 2018 www.thefuturesociety.org The Future Society 2 A Global Civic Debate on Governing the rise of Artificial Intelligence 5 languages Engagement & Participation : English, French, Chinese, 70 390 Page views Japanese,
    [Show full text]
  • Marketing Optimization
    Marketing Optimization An Experian white paper Table of contents Executive summary .....................................................................................................................................2 Background - the marketing challenge ...................................................................................................4 The traditional approach - data driven marketing .................................................................................5 The next step - marketing optimization .................................................................................................10 What is marketing optimization? ............................................................................................................11 Marketing optimization applications .....................................................................................................15 How optimization improves traditional marketing approachs ..........................................................21 Optimization solutions from Experian ...................................................................................................22 About Experian Decision Analytics ........................................................................................................24 An Experian white paper | Page 1 Executive summary For your organization to thrive it is important to make the most of each customer interaction and maximize customer value. In a world where there are thousands or millions of customers, multiple touch
    [Show full text]
  • Sample Chapter [PDF]
    LINK Link is an exercise in abstraction, causality, and modeling. It is about discovering and making visible interdependencies in complex systems. The author distills what she has learned in pithy insights. It takes discipline. You won’t regret reading this book. À Vint Cerf, Internet Pioneer Link is the missing link to our understanding of unintended consequences of many of our decisions and actions. This is a book for the ages, moving both technology (like AI) and human decision making to the next level. Must read. À V R Ferose, SVP and Head of SAP Academy for Engineering There is an explosion today in the impacts, risks, and opportunities of many decisions by private or governmental entities intended to impact future events. Link is about understanding these decisions and causal relationships. Societal transactions are accelerating: many more than ever are intangible, and there are substantial complexities created by newly discovered information as well as the resulting increase of global interdependencies. Surveillance capitalism, especially as enhanced by AI, is also a substantial risk today. Link is part of the solution: a crucial resource to understand causal chains, especially with the goal of avoiding unintended consequences of decisions involving data and technology. À Bill Fenwick, Partner Emeritus, Fenwick & West LLP LINK How Decision Intelligence Connects Data, Actions, and Outcomes for a Better World BY LORIEN PRATT United Kingdom À North America À Japan À India À Malaysia À China Emerald Publishing Limited Howard House, Wagon Lane, Bingley BD16 1WA, UK First edition 2019 Copyright r Lorien Pratt, 2019. Published under exclusive licence.
    [Show full text]
  • Artificial Intelligence in Finance
    Artificial Intelligence in Finance: Forecasting Stock Market Returns Using Artificial Neural Networks Alexandra Zavadskaya Department of Finance Hanken School of Economics Helsinki 2017 HANKEN SCHOOL OF ECONOMICS Department of: Finance Type of work: Thesis Author: Alexandra Zavadskaya Date: 29.09.2017 Title of thesis: Artificial Intelligence in Finance: Forecasting Stock Market Returns Using Artificial Neural Networks Abstract: This study explored various Artificial Intelligence (AI) applications in a finance field. It identified and discussed the main areas for AI in the finance: portfolio management, bankruptcy prediction, credit rating, exchange rate prediction and trading, and provided numerous examples of the companies that have invested in AI and the type of tasks they use it for. This paper focuses on a stock market prediction, and whether Artificial Neural Networks (ANN), being proxies for Artificial Intelligence, could offer an investor more accurate forecasting results. This study used two datasets: monthly returns of S&P500 index returns over the period 1968-2016, and daily S&P 500 returns over the period of 2007-2017. Both datasets were used to test for univariate and multivariate (with 12 explanatory variables) forecasting. This research used recurrent dynamic artificial neural networks and compared their performance with ARIMA and VAR models, using both statistical measures of a forecast accuracy (MSPE and MAPE) and economic (Success Ratio and Direction prediction) measures. The forecasting was performed for both in-sample and out-of-sample. Furthermore, given that ANN may produce different results during each iteration, this study has performed a sensitivity analysis, checking for the robustness of the results given different network configuration, such as training algorithms and number of lags.
    [Show full text]
  • Lorien Pratt, Ph.D. Curriculum Vita Sunnyvale, CA
    Lorien Pratt, Ph.D. Curriculum Vita Sunnyvale, CA . Denver, CO USA +1.303.589.7476 [email protected], @lorienpratt, www.lorienpratt.com SUMMARY Professional machine learning and decision intelligence consultant, developer, entrepreneur, writer, evangelist. Author of Link: How Decision Intelligence Connects Data, Actions, and Outcomes for a Better World. Co- organizer of the annual Responsible AI/DI Summit. Two TEDx talks. Inventor of transfer learning (TL). Co- Founder and Chief Scientist of Quantellia, LLC. Co-inventor and evangelist for Decision Intelligence (DI). Over three decades delivering applied machine learning solutions, including for the Human Genome Project, the US Department of Energy, budget analysis for a $100 million annual spend for the Administrative Office (AO) of the US Courts, award-winning forensic analysis for the Colorado Bureau of Investigation, RBS, DARPA, and many more EDUCATION • Ph.D., Computer Science, Rutgers University, 5/93. Thesis: Transferring previously learned back-propagation neural networks to new learning tasks. • M.S., Computer Science, Rutgers University, 5/88. • A.B., Computer Science, Dartmouth College, 5/83. CONSULTING A variety of engagements including: • Regular invited speaker to user groups, executive roundtables, keynotes, webinars, and workshops for organizations and companies like Global Knowledge, MCubed, MLConf, SAP, HP, Unisys, the TM Forum. • Machine learning consulting on a wide variety of projects for wellness, call centers, cyber security, web advertising, financial prediction, medical decision making, cable company capital expenditure, LIDAR image analysis, and more. • Five-star rated consultant on the Experfy platform (www.experfy.com), from the Harvard Innovation Laboratory. • Strategic consulting in decision intelligence, analytics, and telecom OSS/BSS to companies and organizations like the US Department of Defense, Solvina, ConceptWave, the Colorado Bureau of Investigation (CBI), Sun, IBM, Comptel, SAP, HP, Amdocs, Telcordia (now Ericsson).
    [Show full text]
  • Decision Intelligence for Wet Weather Management Xylem Digital Solutions
    ™ Intelligent Urban Watersheds Decision Intelligence for Wet Weather Management Xylem Digital Solutions Who we are A new Xylem platform of disruptive technologies to help water utilities substantially reduce the capital and operating costs of operating a water network What we do We use data gathering, artificial intelligence and machine learning tools to help utilities monitor, optimize and control the condition and performance of their networks Our impact We create outsize economic benefits (10x), substantially reducing non-revenue water and capital and operating costs through predictive analytics and design optimization Xylem Digital Solutions XDS major site Other major MCS site Nanjing Edmonton Manila Calgary Middletown Toronto South Bend Singapore Yellow Springs San Francisco Columbia Raleigh San Diego Atlanta Dallas Miami Sydney Problem: Increasing intensity and frequency of extreme wet weather Largest ever infrastructure spend for many cities Poorest citizens face largest economic burden Industry Status Quo Industry Status Quo . Intelligent Urban Watersheds Blue Infrastructure: Utilize data analytics to solve water challenges economically and efficiently. What is the Problem? Infrastructure Investment Affordability Status Quo Computing Capacity 1BillionX Era when many CSO interceptors and treatment plants were designed and built Intelligent Urban Watersheds The Internet of Things Big Data Analytics Machine Learning Digital Copy of Urban Watersheds Observation/ Digital Copy Tuning Urban Watershed Implementation Intelligent Urban
    [Show full text]