Issue Editor
Total Page:16
File Type:pdf, Size:1020Kb
Bulletin of the Technical Committee on Data Engineering June 2018 Vol. 41 No. 2 IEEE Computer Society Letters Letter from the Editor-in-Chief . David Lomet 1 Letter from the Special Issue Editor . Guoliang Li 2 Special Issue on Large-scale Data Integration Data Integration: The Current Status and the Way Forward . Michael Stonebraker, Ihab F. Ilyas 3 BigGorilla: An Open-Source Ecosystem for Data Preparation and Integration. ........................... Chen Chen, Behzad Golshan, Alon Halevy, Wang-Chiew Tan, AnHai Doan 10 Self-Service Data Preparation: Research to Practice . Joseph M. Hellerstein, Jeffrey Heer, Sean Kandel 23 Toward a System Building Agenda for Data Integration (and Data Science) . ..........................AnHai Doan, Pradap Konda, Paul Suganthan G.C., Adel Ardalan, Jeffrey R. Ballard, Sanjib Das,Yash Govind, Han Li, Philip Martinkus, Sidharth Mudgal, Erik Paulson, Haojun Zhang 35 Explaining Data Integration . Xiaolan Wang, Laura Haas, Alexandra Meliou 47 Making Open Data Transparent: Data Discovery on Open Data . Renee´ J. Miller, Fatemeh Nargesian, Erkang Zhu, Christina Christodoulakis, Ken Q. Pu, Periklis Andritsos 59 Big Data Integration for Product Specifications . Luciano Barbosa, Valter Crescenzi, Xin Luna Dong, Paolo Merialdo, Federico Piai, Disheng Qiu, Yanyan Shen, Divesh Srivastava 71 Integrated Querying of SQL database data and S3 data in Amazon Redshift . Mengchu Cai, Martin Grund, Anurag Gupta, Fabian Nagel, Ippokratis Pandis, Yannis Papakonstantinou, Michalis Petropoulos 82 Robust Entity Resolution Using a CrowdOracle . .............................. Donatella Firmani, Sainyam Galhotra, Barna Saha, Divesh Srivastava 91 Human-in-the-loop Rule Learning for Data Integration . Ju Fan, Guoliang Li 104 Conference and Journal Notices ICDE 2019 Conference. 116 TCDE Membership Form . back cover Editorial Board TCDE Executive Committee Editor-in-Chief Chair David B. Lomet Xiaofang Zhou Microsoft Research The University of Queensland One Microsoft Way Brisbane, QLD 4072, Australia Redmond, WA 98052, USA [email protected] [email protected] Executive Vice-Chair Associate Editors Masaru Kitsuregawa Philippe Bonnet The University of Tokyo Department of Computer Science Tokyo, Japan IT University of Copenhagen 2300 Copenhagen, Denmark Secretary/Treasurer Joseph Gonzalez Thomas Risse EECS at UC Berkeley L3S Research Center 773 Soda Hall, MC-1776 Hanover, Germany Berkeley, CA 94720-1776 Committee Members Guoliang Li Amr El Abbadi Department of Computer Science University of California Tsinghua University Santa Barbara, California 93106 Beijing, China Malu Castellanos Alexandra Meliou Teradata College of Information & Computer Sciences Santa Clara, CA 95054 University of Massachusetts Xiaoyong Du Amherst, MA 01003 Renmin University of China Beijing 100872, China Distribution Brookes Little Wookey Lee IEEE Computer Society Inha University 10662 Los Vaqueros Circle Inchon, Korea Los Alamitos, CA 90720 Ren´ee J. Miller [email protected] University of Toronto Toronto ON M5S 2E4, Canada The TC on Data Engineering Membership in the TC on Data Engineering is open to Erich Neuhold all current members of the IEEE Computer Society who University of Vienna are interested in database systems. The TCDE web page is A 1080 Vienna, Austria http://tab.computer.org/tcde/index.html. Kyu-Young Whang The Data Engineering Bulletin Computer Science Dept., KAIST The Bulletin of the Technical Committee on Data Engi- neering is published quarterly and is distributed to all TC Daejeon 305-701, Korea members. Its scope includes the design, implementation, modelling, theory and application of database systems and Liaison to SIGMOD and VLDB their technology. Ihab Ilyas Letters, conference information, and news should be sent to the Editor-in-Chief. Papers for each issue are solicited University of Waterloo by and should be sent to the Associate Editor responsible Waterloo, Canada N2L3G1 for the issue. Opinions expressed in contributions are those of the au- thors and do not necessarily reflect the positions of the TC on Data Engineering, the IEEE Computer Society, or the authors’ organizations. The Data Engineering Bulletin web site is at http://tab.computer.org/tcde/bull_about.html. i Letter from the Editor-in-Chief About the Bulletin This June’s issue is the start of the editorial tenure of four new editors. I am both proud and humbled by my continued success in recruiting top notch technical people when I invite them to be Bulletin editors. This success is, indeed, the ”secret sauce” responsible for the Bulletin’s continued success. With the above as prelude, I am proud to announce that the new Bulletin editors are, in alphabetical order, Philippe Bonnet of ITU Copenhagen, Joseph Gonzalez of UC Berkeley, Guoliang Li of Tsinghua University, and Alexandra Meliou of UMASS Amherst. Their contact information appears on the inside cover of this issue. This talented team will be bringing you Bulletin issues over the next two years. The Current Issue There is a bias among some in the research community to focus on solvable problems. You create a program or system and it either solves the problem or it does not. Meanwhile, in the real user community, technical people struggle at tasks that have to be done where it is either impractically difficult or impossible to provide a technical approach that simply solves ”the problem”. Data integration is such a problem. It is important because there are 10’s, 100’s, even 1000’s of data sources that all too frequently need to be “integrated” in some way to solve a real world problem. This problem is sufficiently hard and costly that incremental improvements can be very important. For example, reducing the amount of human involvement in the data integration activity can produce important cost savings and even lead to better results. Data integration is the subject of the current issue. A quick look at the table of contents will show you that data integration has become a “first class” problem for our field. Authors come from universities, large companies, former start-ups, etc., with the divisions between these homes in flux. This illustrates the excitement in the area. Guoliang Li has reached out to these communities in his effort to give you this survey of the field. The issue is a snapshot in time of this challenging area, where the state of the art is improving continually. I want to thank Guoliang for a very successful issue on an important topic. ICDE 2019 The current issue includes an ICDE 2019 call for papers on page 116. ICDE is a premier database conference and the flagship conference of the Technical Committee on Data Engineering. This year’s submission deadlines are earlier than in the past, and there is a two round submission process. The intent is to provide authors earlier feedback and a better opportunity to publish their work earlier in the year. ICDE 2019 is in Macau, which has, via adjacency to Hong Kong, great air connections to the rest of the world. I hope to see you there. David Lomet Microsoft Corporation 1 Letter from the Special Issue Editor Data integration is a long-standing problem over the past few decades. As more than 80% time of a data science project is spent on data integration, it becomes an indispensable part in data analysis. In recent few years, tremendous progress on data integration has been made from systems to algorithms. In this issue, we review the challenges of data integration, survey data integration systems (e.g., Tamr, BigGorilla, Trifacta, PyData), report recent progresses (e.g., explaining data integration, data discovery on open data, big data integration pipeline for product specifications, integrated querying of table data and S3 data, crowd-based entity resolution and human-in-the-loop rule learning), and discuss the future of this filed. The first paper, “Data Integration: The Current Status and the Way Forward” by Michael Stonebraker and Ihab F. Ilyas, discusses scalable data integration challenges based on the experience at Tamr, and highlights future directions on building end-to-end scalable data integration systems, such as data discovery, human in- volvement, data exploration, and model reusability. The second paper, “BigGorilla: An Open-Source Ecosystem for Data Preparation and Integration” by Alon Halevy et al, presents BIGGORILLA, an open-source resource for data preparation and integration. The paper describes four packages – an information extraction tool, a schema matching tool, and two entity matching tools. The third paper, “Self-Service Data Preparation: Research to Practice” by Joseph M. Hellerstein et al, re- views self-service data preparation, which aims to enable the people who know the data best to prepare it. The paper discusses the key tasks in this problem and reviews the Trifacta system on how to handle these tasks. The fourth paper, “Toward a System Building Agenda for Data Integration (and Data Science)” by Anhai Doan et al, advocates to build data integration systems by extending the PyData system and developing more Python packages to solve data integration problems. The paper provides an integrated agenda of research, system building, education, and outreach. The paper also describes ongoing work at Wisconsin. The fifth paper, “Explaining Data Integration”, by Laura Haas et al, reviews existing data integration systems with respect to their ability to derive explanations. The paper presents a new classification of data integration systems by their explainability and discusses the characteristics of systems within these classes. The authors also present a vision of the desired properties of future data integration systems with respect to explanations. The sixth paper, “Making Open Data Transparent: Data Discovery on Open Data” by Renee´ J. Miller et al, discusses the problem of data discovery on open data, e.g., open government data. The paper considers three im- portant data discovery problems: finding joinable tables, finding unionable tables, and creating an organization over a massive collection of tables. The seventh paper, “Big Data Integration for Product Specifications” by Divesh Srivastava et al, presents an end-to-end big data integration pipeline for product specifications.