Analytics: the Agile Way Cover Vote Source: Data Generated Via Google Forms
Total Page:16
File Type:pdf, Size:1020Kb
ANALYTICS THE AGILE WAY BY PHIL SIMON Contents Figure P.1 5 Figure P.2 6 Preface Review and Discussion Questions 7 Table I.1 8 Introduction Review and Discussion Questions 9 Figure 1.1 10 Figure 1.2 11 Chapter 1 Review and Discussion Questions 12 Table 2.1 13 Table 2.2 14 Figure 2.1 15 Figure 2.2 16 Figure 2.3 17 Figure 2.4 18 Figure 2.5 19 Figure 2.6 20 Figure 2.7 21 Figure 2.8 22 Figure 2.9 23 Chapter 2 Review and Discussion Questions 24 Figure 3.1 25 Table 3.1 26 Table 3.2 27 Table 3.3 28 Chapter 3 Review and Discussion Questions 29 Figure 4.1 30 Chapter 4 Review and Discussion Questions 31 Figure 5.1 32 Figure 5.2 33 Figure 5.3 34 Figure 5.4 35 Figure 5.5 36 Figure 5.6 37 Figure 5.7 38 Table 5.1 39 Table 5.2 40 Table 5.3 41 Figure 5.8 42 Figure 5.9 43 Table 5.4 44 Figure 5.10 45 Figure 5.11 46 Chapter 5 Review and Discussion Questions 47 Figure 6.1 48 Table 6.1 49 Table 6.2 50 Figure 6.2 51 Table 6.3 52 Chapter 6 Review and Discussion Questions 53 Table 7.1 54 Table 7.2 55 Table 7.3 56 Figure 7.1 57 Figure 7.2 58 Figure 7.3 59 Figure 7.4 60 Figure 7.5 61 Figure 7.6 62 Figure 7.7 63 Figure 7.8 64 Chapter 7 Review and Discussion Questions 65 Figure 8.1 66 Chapter 8 Review and Discussion Questions 67 Figure 9.1 68 Figure 9.2 69 Chapter 9 Review and Discussion Questions 70 Table 10.1 71 Figure 10.1 72 Figure 10.2 73 Figure 10.3 74 Chapter 10 Review and Discussion Questions 75 Figure 11.1 76 Figure 11.2 77 Figure 11.3 78 Figure 11.4 79 Figure 11.5 80 Chapter 11 Review and Discussion Questions 81 Table 12.1 82 Chapter 12 Review and Discussion Questions 83 Chapter 13 Review and Discussion Questions 84 Figure 14.1 85 Chapter 14 Review and Discussion Questions 86 Chapter 15 Review and Discussion Questions 87 100 75 50 25 March 1, 2009 August 1, 2011 January 1, 2014 June 1, 2016 Figure P.1 Foursquare Interest over Time, March 1, 2009, to March 29, 2017 Source: Google Trends. 5 fpref xx 31 May 2017 9:31 AM fpref xxi 31 May 2017 9:31 AM 10% 7.5% E. coli Frequent Chipotle Goers Outbreak Employees Sick 5% 2.5% Infrequent Chipotle Visitors Jan. 2015 July 2015 Jan. 2016 Figure P.2 Chipotle Share of Restaurant Foot Traffic (Week over Week) Source: Foursquare Medium feed. 6 PREFACE REVIEW AND DISCUSSION QUESTIONS ■ Foot traffic has always mattered. What’s different about it today? ■ Why would hedge funds and even individual investors be interested in data related to “digital” foot traffic? Can you think of any other uses for this data? ■ Why has Foursquare struggled to meet its financial goals? What is it trying to do to finally meet them? Do you think that the company will succeed? 7 fpref xxiv 31 May 2017 9:31 AM fpref xxv 31 May 2017 9:31 AM Table I.1 Project Plan for Launch of Generic BI Application Phase Description Start Date End Date 1 Evaluate proposals from software vendors, check 2/1/02 5/31/02 references, and perform general due diligence. 2 Select winning bid. Negotiate terms and sign contract. 6/1/02 7/31/02 3 Extract data from legacy systems, clean up errors, and 8/1/02 8/31/02 deduplicate records. 4 Implement and customize software, typically with help of 9/1/02 10/31/02 expensive consultants. 5 Train users on new application. 11/1/02 2/28/03 6 Load purified data into BI application and address errors. 3/1/03 3/31/03 7 Launch application and squash bugs. 4/1/03 4/30/03 8 Engage vendor in on-site or remote application support. 5/1/03 6/30/03 Source: Phil Simon. 8 cintro 2 31 May 2017 9:30 AM cintro 3 31 May 2017 9:30 AM INTRODUCTION REVIEW AND DISCUSSION QUESTIONS ■ What are Waterfall or phase-gate projects? ■ Do they lend themselves to successful outcomes? Why or why not? ■ What are a few examples of automated decision making? Is this currently the exception or the rule? Do you expect that to change? 9 cintro 14 31 May 2017 9:30 AM cintro 15 31 May 2017 9:30 AM $1,000,000.00 $100,000.00 $10,000.00 $1,000.00 $100.00 $/GB $10.00 $1.00 $0.10 $0.01 January January 1980 2010 figure 1.1 Data-Storage Costs over Time Source: Data from Matthew Komorowski (see www.mkomo.com/cost-per-gigabyte) Figure from Phil Simon. 10 c01 26 31 May 2017 9:23 AM c01 27 31 May 2017 9:23 AM Apple 567.75 Alphabet 546.49 Microsoft 445.14 Amazon 366.95 Facebook 364.26 0100 200 300 400 500 Value (billions) Figure 1.2 The World’s Most Valuable Companies by Market Cap as of July 29, 2016, at 10:50 a.m. ET Source: Data from Google Finance. Figure from Phil Simon. 11 c01 32 31 May 2017 9:23 AM c01 33 31 May 2017 9:23 AM CHAPTER 1 review and disCussion Questions ■ What do you think of Steve Jobs’s stance regarding the New York Times? ■ What types of things could Apple do with this type of customer information? What types of apps could it recommend? ■ What types of data does Uber capture? How can it analyze that data in ways that traditional taxi and transportation companies cannot? ■ What types of experiments can Uber run? ■ How else would you use this data? ■ How could you change the Uber app to collect even more information? ■ Now imagine that you are Airbnb CEO Brian Chesky. What kinds of questions could you ask and answer of your company’s data? ■ How would you use that information? ■ Could bookstores and traditional publishers have embraced new technologies and data sources more quickly? What specifically could each have done? ■ Do you think that any of these moves ultimately would have made a difference, or are disruption and marginalization inevitable? 12 c01 42 31 May 2017 9:23 AM c01 43 31 May 2017 9:23 AM Table 2.1 Sample of Structured Data from Fictional Employee Table* Employee First Name Last Name Salary 7777 Mark Kelly 100,000 7778 Steve Hogarth 110,000 7779 Pete Trawavas 99,000 7780 Steven Rothery 103,455 7781 Ian Mosley 105,000 Source: Phil Simon. *The astute reader will recognize these names and how underpaid these men are in this example. 13 c02 46 31 May 2017 9:23 AM c02 47 31 May 2017 9:23 AM Table 2.2 Affiliate Payments from GMC Check Date Check Number Check Amount 1/31/13 1234 $28.12 2/28/13 1254 $30.07 3/31/13 1275 $31.04 Source: Phil Simon. 14 c02 50 31 May 2017 9:23 AM c02 51 31 May 2017 9:23 AM Figure 2.1 Tweet about Data Scientists Source: @josh_wills, Twitter, May 3, 2012. 15 c02 52 31 May 2017 9:23 AM c02 53 31 May 2017 9:23 AM Figure 2.2 Write and Rant Facebook Post Source: Facebook post on March 2, 2017. 16 c02 54 31 May 2017 9:23 AM c02 55 31 May 2017 9:23 AM Which cover do you like the most? 1 2 3 4 5 7 01234 Figure 2.3 Initial Results of Cover Poll for Analytics: The Agile Way Cover Vote Source: Data generated via Google Forms. Figure from Phil Simon. 17 Figure 2.4 Data Roundtable Front Page as of December 16, 2016 Source: SAS/Phil Simon. 18 c02 56 31 May 2017 9:23 AM c02 57 31 May 2017 9:23 AM author_name date title page_views 1Jim Harris February 29, 2016 How big of a deal is big data quality? 2463 2 Dylan Jones February 23, 2016 Scaling a vision for data quality 2966 3 Phil Simon February 18, 2016 Who’s in charge of data quality? 2210 Figure 2.5 Results of Web Scraping via import.io Source: import.io/Phil Simon. 19 6,000 4,956 4,500 3,696 3,418 3,311 3,000 3,047 2,438 2,087 1,500 1,756 0 Phil Simon David Loshin Jim Harris Dylan Jones Joyce Norris- Matthew Daniel Carol Montanari Magne Teachey Newcomb Figure 2.6 Page Views by Author on Data Roundtable Source: Data generated from Data Roundtable, scraped via import.io, and analyzed with Google Sheets. Figure from Phil Simon. 20 c02 56 31 May 2017 9:23 AM c02 57 31 May 2017 9:23 AM firstName,lastName Emilio,Koyama Skyler,White Hank,Schrader Figure 2.7 Simple CSV Example Source: Phil Simon. 21 c02 58 31 May 2017 9:23 AM c02 59 31 May 2017 9:23 AM {"employees":[ { "firstName":"Emilio", "lastName":"Koyama" }, { "firstName":"Skyler", "lastName":"White" }, { "firstName":"Hank", "lastName":"Schrader" } ]} Figure 2.8 Simple JSON Example Source: Phil Simon. 22 c02 58 31 May 2017 9:23 AM c02 59 31 May 2017 9:23 AM <employees> <employee> <firstName>Emilio</firstName> <lastName>Koyama</lastName> </employee> <employee> <firstName>Skyler</firstName> <lastName>White</lastName> </employee> <employee> <firstName>Hank</firstName> <lastName>Schrader</lastName> </employee> </employees> Figure 2.9 Simple XML Example Source: Phil Simon.