Deep Learning with TensorFlow 2 and Keras Second Edition Regression, ConvNets, GANs, RNNs, NLP, and more with TensorFlow 2 and the Keras API Antonio Gulli Amita Kapoor Sujit Pal BIRMINGHAM - MUMBAI Deep Learning with TensorFlow 2 and Keras Second Edition Copyright © 2019 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. Commissioning Editor: Amey Varangaonkar Acquisition Editors: Yogesh Deokar, Ben Renow-Clarke Acquisition Editor – Peer Reviews: Suresh Jain Content Development Editor: Ian Hough Technical Editor: Gaurav Gavas Project Editor: Janice Gonsalves Proofreader: Safs Editing Indexer: Rekha Nair Presentation Designer: Sandip Tadge First published: April 2017 Second edition: December 2019 Production reference: 2130320 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-83882-341-2 www.packt.com packt.com Subscribe to our online digital library for full access to over 7,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website. Why subscribe? • Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals • Learn better with Skill Plans built especially for you • Get a free eBook or video every month • Fully searchable for easy access to vital information • Copy and paste, print, and bookmark content Did you know that Packt offers eBook versions of every book published, with PDF and ePub fles available? You can upgrade to the eBook version at www.Packt.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details. At www.Packt.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks. Contributors About the authors Antonio Gulli has a passion for establishing and managing global technological talent, for innovation and execution. His core expertise is in cloud computing, deep learning, and search engines. Currently, he serves as Engineering Director for the Offce of the CTO, Google Cloud. Previously, he served as Google Warsaw Site leader, doubling the size of the engineering site. So far, Antonio has been lucky enough to gain professional experience in 4 countries in Europe and has managed teams in 6 countries in EMEA and the US: in Amsterdam, as Vice President for Elsevier, a leading scientifc publisher; in London, as Engineering Site Lead for Microsoft working on Bing Search as CTO for Ask.com; and in several co-funded start-ups including one of the frst web search companies in Europe. Antonio has co-invented a number of technologies for search, smart energy, the environment, and AI, with 20+ patents issued/applied, and he has published several books about coding and machine learning, also translated into Japanese and Chinese. Antonio speaks Spanish, English, and Italian, and he is currently learning Polish and French. Antonio is a proud father of 2 boys, Lorenzo, 18, and Leonardo, 13, and a little queen, Aurora, 9. I want to thank my kids, Aurora, Leonardo, and Lorenzo, for motivating and supporting me during all the moments of my life. Special thanks to my parents, Elio and Maria, for being there when I need it. I'm particularly grateful to the important people in my life: Eric, Francesco, Antonello, Antonella, Ettore, Emanuela, Laura, Magda, and Nina. I want to thank all my colleagues at Google for their encouragement in writing this and previous books, for the precious time we've spent together, and for their advice: Behshad, Wieland, Andrei, Brad, Eyal, Becky, Rachel, Emanuel, Chris, Eva, Fabio, Jerzy, David, Dawid, Piotr, Alan, and many others. I'm especially appreciative of all my colleagues at OCTO, at the Offce of the CTO at Google, and I'm humbled to be part of a formidable and very talented team. Thanks, Jonathan and Will. Thanks to my high school friends and professors who inspired me over many years (D'africa and Ferragina in particular). Thanks to the reviewer for their thoughtful comments and efforts toward improving this book, and my co-authors for their passion and energy. This book has been written in six different nations: Warsaw, Charlotte Bar; Amsterdam, Cafe de Jaren; Pisa, La Petite; Pisa, Caffe i Miracoli; Lucca, Piazza Anfteatro, Tosco; London, Said; London, Nespresso, and Paris, Laduree. Lots of travel and lots of good coffee in a united Europe! Amita Kapoor is an associate professor in the Department of Electronics, SRCASW, University of Delhi, and has been actively teaching neural networks and artifcial intelligence for the last 20 years. Coding and teaching are her two passions, and she enjoys solving challenging problems. She is a recipient of the DAAD Sandwich fellowship 2008, and the Best Presentation Award at an international conference, Photonics 2008. She is an avid reader and learner. She has co-authored books on Deep Learning and has more than 50 publications in international journals and conferences. Her present research areas include machine learning, deep reinforcement learning, quantum computers, and robotics. To my grandmother the late Kailashwati Maini for her unconditional love and affection; and my grandmother the late Kesar Kapoor for her marvelous stories that fueled my imagination; my mother, the late Swarnlata Kapoor, for having trust in my abilities and dreaming for me; and my stepmother, the late Anjali Kapoor, for teaching me every struggle can be a stepping stone. I am grateful to my teachers throughout life, who inspired me, encouraged me, and most importantly taught me: Prof. Parogmna Sen, Prof. Wolfgang Freude, Prof. Enakshi Khullar Sharma, Dr. S Lakshmi Devi, Dr. Rashmi Saxena and Dr. Rekha Gupta. I am extremely thankful to the entire Packt team for the work and effort they put in since the inception of this book, the reviewers who painstakingly went through the content and verifed the codes; their comments and suggestions helped improve the book. I am particularly thankful to my co- authors Antonio Gulli and Sujit Pal for sharing their vast experience with me in the writing of this book. I would like to thank my college administration, governing body and Principal Dr. Payal Mago for sanctioning my Sabbatical leave so that I can concentrate on the book. I would also like to thank my colleagues for the support and encouragement they have provided, with a special mention of Dr. Punita Saxena, Dr. Jasjeet Kaur, Dr. Ratnesh Saxena, Dr. Daya Bhardwaj, Dr. Sneha Kabra, Dr. Sadhna Jain, Mr. Projes Roy, Ms. Venika Gupta and Ms. Preeti Singhal. I want to thank my family members and friends my extended family Krishna Maini, Suraksha Maini, the late HCD Maini, Rita Maini, Nirjara Jain, Geetika Jain, Rashmi Singh and my father Anil Mohan Kapoor. And last but not the least I would like to thank Narotam Singh for his invaluable discussions, inspiration and unconditional support through all phases of my life. A part of the royalties of the book will go to smilefoundation.org. Sujit Pal is a Technology Research Director at Elsevier Labs, an advanced technology group within the Reed-Elsevier Group of companies. His areas of interest include Semantic Search, Natural Language Processing, Machine Learning, and Deep Learning. At Elsevier, he has worked on several machine learning initiatives involving large image and text corpora, and other initiatives around recommendation systems and knowledge graph development. He has previously co-authored another book on Deep Learning with Antonio Gulli and writes about technology on his blog Salmon Run. I would like to thank both my co-authors for their support and for making this authoring experience a productive and pleasant one, the editorial team at Packt who were constantly there for us with constructive help and support, and my family for their patience. It has truly taken a village, and this book would not have been possible without the passion and hard work from everyone on the team. About the reviewers Haesun Park is a machine learning Google Developer Expert. He has been a software engineer for more than 15 years. He has written and translated several books on machine learning. He is an entrepreneur, and currently runs his own business. Other books Haesun has worked on include the translation of Hands-On Machine Learning with Scikit-Learn and TensorFlow, Python Machine Learning, and Deep Learning with Python. I would like to thank Suresh Jain who proposed this work to me, and extend my sincere gratitude to Janice Gonsalves, who provided me with a great deal of support in the undertaking of reviewing this book. Dr. Simeon Bamford has a background in AI. He is specialized in neural and neuromorphic engineering, including neural prosthetics, mixed-signal CMOS design for spike-based learning, and machine vision with event-based sensors.