Data Anonymizing API – Phase II TM Forum Digital Transformation World 2018 Nice, France 14-16 May © 2018 TM Forum | 1 Catalyst Champion & Participants Orange, Emilie Sirvent-Hien, Anonymization project manager and Sophie Nachman, Standards Manager To get standardized anonymization API allows sharing of data internally within Orange and externally with partners in order to unleash services innovation , together with guarantying privacy of customers, in compliance with GDPR, General Data Protection Regulation Vodafone Atul Ruparelia, Data Architect and Imo Ekong, Big Data Communication Specialist (Analytics CoE) Contribute towards standard open API for anonymization/pseudonymization (using rich TMF assets) allowing data sharing with internal/external partners to drive service innovation but also protecting PII data in compliance to GDPR. Cardinality Steve Bowker, CEO & Co-Founder, and Dejan Vujic, Head of Data Science Cardinality have implemented one of the largest Hadoop based analytics solutions in a telco in Europe which leverages a containerised microservices based architecture making extensive use of APIs, including different data anonymization, pseudonymization and encryption within their solution. Brytlyt Richard Heyns, CEO & Founder Brytlyt Brytlyt leverage advanced processing on GPUs in natively parallelizable algorithms which form the foundation of their high performance data analytics and machine learning Liverpool John Moores University, Professor Paul Morrissey, Amongst other things Paul is the Global Ambassador for the TMForum with responsibility for Big Data Analytics and Customer Experience Management. Provided input on business drivers, CurateFX, Osterwalder Business Canvass. Data Science and Software Programmers 5G Innovation Centre (5GIC) hosted by University of Surrey 5GIC members are collaborating closely to drive forward advanced wireless research, reduce the risks of implementing 5G (through their 5G testbed) and contribute to global 5G standardisation. The 5GIC has to date attracted an additional £68m from industry and regional partners. © 2018 TM Forum | 2 Catalyst Background Kicking off where we left before – Data Anonymization Phase I Catalyst Mobile network operators are increasing using third parties to help deliver their overall solution offerings. As a service provider, they can’t share data or allow access to their systems without data protection (for example data anonymization) as they have to protect the privacy of their subscribers and to adhere to the regulatory environment which is demanding with new mandatory regulations requiring data privatization. © 2018 TM Forum | 3 Data Anonymization - Marketing Campaign Example Use Case Transaction Web Transaction Data Sources DataMart Data Analytics Data Data Anonymization API Marketing Database/Solution Provider Data Repository Platform Customer data may contain Personal Identifiable information(PII) such as name address and date of birth. A marketing campaign may decide to leverage on specific data to recommend a new product or services to customers. Data needs to be retrieved and transferred from the source to a data repository externally or 3rd party for Marketing company to be analysed for insights to offer some products to customers (Cross sell, up-sell) and drive some marketing with current or new customers for revenue growth, Vodafone would use 3rd parties for creation of marketing offers For legal privacy and security purposes personal identifiable information may need to be anonymised before data is transferred to a © 2018 TM Forum | 4 third party or external database. What is Data Anonymization? A process by which personally identifiable information (PII) is irreversibly altered in such a way that a PII principal can no longer be identified directly or indirectly, either by the PII controller alone or in collaboration with any other party (ISO 29100:2011) In accordance with Article G29 Opinion*, there are Three Main Principles: v Singling out: Is it possible to isolate someone in particular? v Linkability: Is it possible to linK, at least, two records concerning the same data subject ? v Inference: Is it possible to deduce information about one person? Once a dataset is truly anonymized, GDPR no longer applies Anonymization technologies : 3 Technical Families v Generalization (K-anonymization) v Randomization (noise addition, Differential Privacy) v Obfuscation *http://ec.europa.eu/justice/data-protection/article-29/documentation/opinion-recommendation/files/2014/wp216_en.pdf © 2018 TM Forum | 5 What is Pseudo-Anonymization ? Data Masking ? Pseudonymization is a data management and de-identification procedure by which personally identifiable information fields within a data record are replaced by one or more artificial identifiers, or pseudonyms. A single pseudonym for each replaced field or collection of replaced fields makes the data record less identifiable while remaining suitable for data analysis and data processing. Pseudonymized data can be restored to its original state with the addition of information which then allows individuals to be re-identified, while anonymized data can never be restored to its original state. We can enlarge pseudonymization with data masking technics • Substitution • Shuffling • Number and date variance • Encryption • Nulling and deletion • Scrambling When is it used to share data? § inside a company as soon as possible § with trusted partner (e.g. Vodafone – BT Openreach) § Pseudonymised data are still personal data but less sensible © 2018 TM Forum | 6 Privacy by design methodology Qualify the use case • What type of data? Collected on which basis? Who has access to it? • What to you want to do with protected data? • Do you need a reversible step? • What are the reidentification risks? Find the Trade-off between utility and privacy • Choose the right technology • Validate with a privacy risk assessment © 2018 TM Forum | 7 Catalyst Objective OBJECTIVE: Build a prototype and develop a new TMForum OPEN API that will allow serviCe Providers to protect sensitive data so it Can be shared with third parties without Compromising privaCy and meeting new regulatory requirements suCh as GDPR. Technical Objective: v Building upon industry agreed standards enabling a sCalable platform, improving interoperability, effiCienCy and transparenCy. v Investigating various methods of anonymizing and protecting data for use in the API v Prototyping initial API for providing Anonymized data to third parties without Compromising the privaCy of the data Content v Assessing the effeCtiveness of anonymization of various approaChes that Could be used, and the impaCt on the "value" of the data shared if Data is protected 100% v Exploring ways to support a 2-way OPEN API for Anonymization (maybe pseudo anonymization / data proteCtion ) Business Objective: v Exploring New Business Models for TelCo Data-as-a-ServiCe v Exploring potential uses outside TelCo domain e.g. for Banking Services, Media Content providers etC © 2018 TM Forum | 8 Use Cases Data Protection for different needs § Operational usage: When treating personal data for non-production usage samples needed for test, training, sharing with internal or external supplier, employees outside Europe § Marketing usage: Customer knowledge improvement or new services development § Business usage: Data monetization with external customer § GDPR right to be forgotten § Artificial Intelligence Platform Sharing or AI Algorithm Model Training : example 5G Location and Content Prediction Algorithms § Data publishing for example scientific studies and open data © 2018 TM Forum | 9 Use Case Example 1 : Offer content As A Third Party Marketing Company (Exec) I Need To Offer Content (film/tv and VoD) to a specific group of subscribers in a specific Geographic Market based on various criteria So That I Can Increase Content Usage and Increase Revenue for our customers To Do This, I Need Access Data from the Telco's customer based (subscribers) to To understand which subscribers (based on various criteria) are appropriate to target. In need this capability to be conducted under the auspice of National Data Governance rules and regulations to guarantee the Data anonymisation of the Telco subscribers I Know I Am There is an increase in sales of the content to a point where the service Successful When is profitable and is accepted by the Telco Governance and Security policies © 2018 TM Forum | 10 Business Canvas - To drive revenue growth and to unleash offers/products (eTOM terminology) © 2018 TM Forum | 11 Data Flow Data Originated Data Data Deanonymized Pseudonymized Data Analyzed via Machine Learning © 2018 TM Forum | 12 Anonymization API (see Demo at Catalyst stand level 2) © 2018 TM Forum | 13 CATALYST Data for Demo Demo needed to be based on publicly available data, so we selected the ADULT data set from the University of California, Irvine – Machine Learning Repository © 2018 TM Forum | 14 Enriched UCI “Adult” dataset used (extra synthetic Telco Fields) • Based on UCI “Adult” dataset, enhanced with Telco Fields e.g. IMEI, IMSI etc…. INPUT FILE to API OUTPUT FILE after processing © 2018 TM Forum | 15 Example Anonymization using “K-Anonymization” • AGE data is binned into ranges of 10 years INPUT FILE to API OUTPUT FILE after processing © 2018 TM Forum | 16 Example of processing of data using “Discretization” • Human Readable Information Mapped to discrete values on sensitive fields INPUT FILE to API OUTPUT FILE after processing © 2018 TM Forum | 17 Example Pseudonimization using one-way