Internet measurement platform Measuring Broadband America
Measuring Broadband America February 23, 2018
Presented to: MBA Collaborative Washington DC
London Washington DC 135 Park Street, London, SE1 9EA 1775 Tysons Boulevard, Virginia, VA 22102
SamKnows Limited | Confidential Internet measurement platform Measuring Broadband America
2017 Review
SamKnows Limited | Confidential Internet measurement platform Measuring Broadband America
2017 MBA new updates
• 2017 report status
• Service/modem upgrades policy
• Increase number of Whiteboxes, per speed tier to 40
• Extend the reporting period by 14 days to account for hurricane
• New sign-up page URL with country, ISP and speed tier pre-populated
• Volunteer accepts the EULA when signing up to be part of the project
• Started to automate the validation process for some ISPs
• New lightweight test
SamKnows Limited | Confidential 3 Internet measurement platform Measuring Broadband America
2018 Review
SamKnows Limited | Confidential Internet measurement platform Measuring Broadband America
End-to-end measurement
Customers home Local exchange Regional ISP National ISP Transit providers Content providers
1 2 3 4
Home tests On-net tests Of-net tests Application tests Router - Whitebox Whitebox to on-net server Whitebox to of-net server Whitebox to Web application
SamKnows Limited | Confidential 5 Internet measurement platform Measuring Broadband America
New developments
New analytics system
New test suite
New router integrations
New mobile apps with Wi-Fi measurements
Live data
APIs
New mapping tools
SamKnows Limited | Confidential 6 Internet measurement platform Measuring Broadband America
Project re-branding
SamKnows Limited | Confidential 7 Internet measurement platform Measuring Broadband America
Over 100,000
active probes Close sidebar Dashboard Charts Admin My account Data source Visual chart Data tables Unit info
Ofcom Dashboard ✓ AT&T, Brighthouse, CableONE, Download Speed split by ISP Metric Last 7 days, Daily
CenturyLink, Charter, Cincinnati Bell, Download speed 160 24th July Combined Comcast, Cox, EarthLink, EnTouch, 140 24th July 78.378.3 Mbps Add another metric Frontier, Hargray Communications, 120 Chart type 100
Hawaiian Telcom, Hughes, Mediacom, Line chart 80
Always display 0 on the y-axis i Online, Optimum, PenTeleData, RCN, 60 Date range 40
Skyline, T-mobile USA, Tarr, Time Warner Last 7 days 20
Hourly Cable, Verizon, Verizon Business, Verizon 0
Normalise data i 6am 9am 12am 3pm 6pm 9pm 12pm
Wireless, WideOpenWest Virgin Media TalkTalk BT Filter data by
(WOW),Wildblue/ViaSat, Windstream Select a filter Export Data Save image Print Enter presenter mode
© SamKnows Ltd 2017 All Rights Reserved Help Contact Privacy Terms English UK
SamKnows Limited | Confidential 8 Internet measurement platform Measuring Broadband America
Home Wi-Fi and data usage
✓ Measures mobile and Wi-Fi performance from the handset
✓ Key metrics: download speed, upload speed, latency, packet loss, jitter, web browsing performance, and YouTube
✓ Configured to run tests to pre-defined measurement schedule (only available on Android)
✓ Allows the user to configure a data cap for the mobile and Wi-Fi measurements
✓ A real-time continuous measurement
✓ Granular visibility of all historical results
✓ Environmental capture
SamKnows Limited | Confidential 9 Internet measurement platform Measuring Broadband America
Current tests and metrics
Region County/sub-region Environmental Associated SSID Network LAN WiFi Upload test Upload speed Packet loss Packets sent DNS resolution DNS server being used Postcode information (Wi- Link rate Cellular Upload test duration Packets received test Name being queries Geography Fi) Encryption mode Fixed Upload bytes Average packet loss on an Response time Line length Channel Upload speed at diferent hourly basis Success and failure rate Frequency (2.4Ghz vs 5Ghz) intervals during the test HT Mode (optional) RSSI Nearby APs Wi-Fi Standard
Probe Information Unit ID Handset App version Cross trafic Timestamp of cross-trafic Download Download speed Web browsing Website being tested Generic Unit ID that ran the test Whitebox model information Handset manufacturer Activity breach event test Download test duration test Number of elements on the information Timestamp in UTC Whitebox capabilities (max (cellular/WiFi) Handset model Cause of cross-trafic (wired Downloaded bytes web page available with all Test server hostname speeds) Handset OS type or Wi-Fi) Download speed at Page load time tests: Test server IP address Whitebox OS & version Handset OS version Volume of cross-trafic in diferent intervals during TCP connection establishment IMEI bytes per second the test (optional) time IMSI DNS resolution time
Environmental Public IPv4 addressPublic IPv6 Environmental SIM operator name YouTube test Google Global Cache (GGC) Latency test Mean latency, on an hourly Netflix test Netflix Open Connect VoIP test Round-trip latency Information address information SIM operator code content server hostname basis Appliance (OCA) content Upstream packet loss LAN IPv4 address (cellular) Network operator name GGC IP addressVideo ID Maximum latency, on an server hostnameOCA IP Downstream packet loss LAN IPv4 gateway Network operator code being measured hourly basis address Upstream jitter Link speed and duplex Roaming status Codec and container Minimum latency, on an Bitrate being measured Downstream jitter CPE MAC address Active network type Bitrate being measured hourly basis Bitrate that could be reliably MoS CPE brand Network bearer (2G/3G/4G/ Bitrate that could be reliably Standard deviation of streamed Packet Loss Rate CPE model (available in most LTE) streamed hourly latency Stall count (available in non- Packet Discard Rate cases via UPnP) Signal strength (RSSI) Stall count (available in non- Packets sent BRS mode) Burst Density CPE sync speed (available in most LAC BRS mode) Packets received Stall duration (available in non- Gap Density cases via UPnP) Cell tower ID Stall duration (available in BRS mode) Burst Duration IP-geolocated ISP name Other visible cell towers non-BRS mode) TCP connection establishment Gap Duration IP-geolocated country, city & state Physical longitude TCP connection time Round Trip Delay Physical latitude establishment time Video startup delay End System Delay Location accuracy Video startup delay Signal Level Location provider (GPS/ Video resolution Noise Level network/WiFi) R-Factor MOS for listening quality MOS for conversation quality Current and maximum jitter delay
SamKnows Limited | Confidential 10 Internet measurement platform Measuring Broadband America
New tests and metrics (new SK test dev team)
✓ Dropbox
✓ Google Drive
✓ Microsoft OneDrive
✓ Amazon Drive
✓ Twitch streaming
✓ World of Warcraft
✓ Spotify
✓ Box
✓ Facebook CDN (Facebook caches)
✓ Steam
SamKnows Limited | Confidential 11 Internet measurement platform Measuring Broadband America
New ISP dashboard Dashboard Charts Admin My account
Trevor deluxe dashboard Customise dashboard Enter presenter mode ✓ View key metrics in a glance Download speed by region Download speed by product Last 2 days, hourly Last 2 days, hourly ✓ Configure your own dashboard, with reports 70 you have created and saved Washington 60
50 Average speed 32.5 Mbps
✓ Monitor performance across a whole network 40 Unit count 287
30 Sample count 1633 ✓ See in realtime issues that might afect 20 customer experience 01 / 09 10 / 09 20 / 09 30 / 09 BT 17 BT 38 BT 52 BT 76 ✓ See the MBA data being collected in realtime Key performance indicators Last 2 days, hourly
Product Download Upload Latency
BT 17 Mbps 17.2 Mbps 6.7 Mbps 8.2 ms
BT 38 Mbps 35.8 Mbps 10.3 Mbps 6.4 ms
BT 52 Mbps 48.4 Mbps 21.9 Mbps 7.3 ms
> 100Mbps > 60 Mbps > 30Mbps > 10 Mbps BT 76 Mbps 75.6 Mbps 33.8 Mbps 4.8 ms
SamKnows Limited | Confidential 12 Internet measurement platform Measuring Broadband America
New charts
Close sidebar Dashboard Charts Admin My account
✓ Advanced analytics and reporting Data source Visual chart Data tables Unit info
Ofcom Dashboard Download Speed split by ISP ✓ Multiple charts including all those used in the Metric Last 7 days, Daily
MBA program Download speed 160 24th July Combined 140 78.324th Mbps July ✓ View measurement agents, users and metadata 78.3 Mbps Add another metric 120
Chart type ✓ View test schedules 100 Line chart 80
Always display 0 on the y-axis i ✓ Map data in realtime 60 Date range 40
Last 7 days ✓ An advanced permissions 20
Hourly and access control system 0 Normalise data i 6am 9am 12am 3pm 6pm 9pm 12pm Virgin Media TalkTalk BT Filter data by
Select a filter Export Data Save image Print Enter presenter mode
© SamKnows Ltd 2017 All Rights Reserved Help Contact Privacy Terms English UK
SamKnows Limited | Confidential 13 Internet measurement platform Measuring Broadband America
New reports
✓ Sam’s favourites!
✓ View all of your saved reports in one place
✓ Quick and easy to use
✓ Click to load charts, save reports
✓ Import measurements from ISPs
SamKnows Limited | Confidential 14 Internet measurement platform Measuring Broadband America
MBA 2018 Consumer Report
Initial findings Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis parturient montes, DATE nascetur ridiculus mus. Donec quam felis, ultricies nec, pellentesque eu, pretium quis, sem. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Donec quam felis, ultricies nec, pellentesque eu, pretium quis, Test name sem. 50
Test name 37.5
50 25
37.5 Spee
25 12.5
Spee 12.5 0 Upload Download 0 Upload Download
Measuring Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis parturient montes, broadband dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Donec quam felis, ultricies nec, pellentesque eu, pretium quis, nascetur ridiculus mus. Donec quam felis, ultricies nec, pellentesque eu, pretium quis, sem. America sem. MBA, 2018 Report Test name Lorem ipsum dolor sit amet, consectetuer adipiscing elit. 50 Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur 37.5 ridiculus mus. Donec quam felis, ultricies nec, pellentesque eu, 25
pretium quis, sem. Spee 12.5 What’s next 0 Upload Download This is the first of many reports which will be released quarterly by the MBNZ project which will track improvements in internet performance over time. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis parturient montes, Please participate by volunteering to receive a Whitebox. Sign up at nascetur ridiculus mus. Donec quam felis, ultricies nec, pellentesque eu, pretium quis, https://www.measuringbroadbandamerica.com sem.
MBA, Consumer Report 2018 The information in this document is correct as of Feb 2018 MBA, Consumer Report 2018 The information in this document is correct as of Feb 2018 MBA, Consumer Report 2018 The information in this document is correct as of Feb 2018
SamKnows Limited | Confidential 15 Internet measurement platform Measuring Broadband America
Charlotte Wilson Government team: North America [email protected]
SamKnows Limited | Confidential Internet measurement platform Measuring Broadband America
END
SamKnows Limited | Confidential