MEAP Edition Manning Early Access Program The Programmer’s Guide to Apache Thrift Version 5 Copyright 2013 Manning Publications For more information on this and other Manning titles go to www.manning.com ©Manning Publications Co. We welcome reader comments about anything in the manuscript - other than typos and other simple mistakes. These will be cleaned up during production of the book by copyeditors and proofreaders. http://www.manning-sandbox.com/forum.jspa?forumID=873 Licensed to Daniel Gavrila <[email protected]> Welcome Hello and welcome to the third MEAP update for The Programmer’s Guide to Apache Thrift. This update adds Chapter 7, Designing and Serializing User Defined Types. This latest chapter is the first of the application layer chapters in Part 2. Chapters 3, 4 and 5 cover transports, error handling and protocols respectively. These chapters describe the foundational elements of Apache Thrift. Chapter 6 describes Apache Thrift IDL in depth, introducing the tools which enable us to describe data types and services in IDL. Chapters 7 through 9 bring these concepts into action, covering the three key applications areas of Apache Thrift in turn: User Defined Types (UDTs), Services and Servers. Chapter 7 introduces Apache Thrift IDL UDTs and provides insight into the critical role played by interface evolution in quality type design. Using IDL to effectively describe cross language types greatly simplifies the transmission of common data structures over messaging systems and other generic communications interfaces. Chapter 7 demonstrates the process of serializing types for use with external interfaces, disk I/O and in combination with Apache Thrift transport layer compression. Chapter 8 will add Apache Thrift service coverage and Chapter 9 will round out Part 2 with a full treatment of Apache Thrift servers. Part 3 is also taking shape and will serve as an introduction to several of the other key languages used with Apache Thrift including JavaScript (browser and Node implementations), C#, Ruby and more. Please feel free to leave any questions of comments on the book’s forum. I will be sure to respond to any and all. Happy coding, --Randy Abernethy ©Manning Publications Co. We welcome reader comments about anything in the manuscript - other than typos and other simple mistakes. These will be cleaned up during production of the book by copyeditors and proofreaders. http://www.manning-sandbox.com/forum.jspa?forumID=873 Licensed to Daniel Gavrila <[email protected]> brief contents PART 1: APACHE THRIFT OVERVIEW 1 Introduction to Apache Thrift 2 Apache Thrift Architecture PART 2: PROGRAMMING APACHE THRIFT 3 Moving Bytes with Transports 4 Handling Exceptions 5 Serializing Data with Protocols 6 Apache Thrift IDL 7 User Defined Types 8 Implementing Services 9 Servers PART 3: POLYGLOT APPLICATION DEVELOPMENT 10 A Thrift based Enterprise 11 The C++ Live Feed Service 12 The Java Transaction Processing Service 13 The Python/PHP Web Tier 14 The JavaScript Browser Client 15 The C# Update Service 16 The Ruby Log Processor 17 The iOS and Android Mobile Clients 18 The Big Picture ©Manning Publications Co. We welcome reader comments about anything in the manuscript - other than typos and other simple mistakes. These will be cleaned up during production of the book by copyeditors and proofreaders. http://www.manning-sandbox.com/forum.jspa?forumID=873 Licensed to Daniel Gavrila <[email protected]> APPENDIXES: appendix A Apache Thrift Setup on Ubuntu/Debian Linux appendix B Apache Thrift Setup on Centos/RHEL Linux appendix C Apache Thrift Setup on Windows appendix D Apache Thrift Setup on OS X appendix E Apache Thrift C++ Dependencies appendix F Apache Thrift Java Dependencies appendix G Apache Thrift Python Dependencies ©Manning Publications Co. We welcome reader comments about anything in the manuscript - other than typos and other simple mistakes. These will be cleaned up during production of the book by copyeditors and proofreaders. http://www.manning-sandbox.com/forum.jspa?forumID=873 Licensed to Daniel Gavrila <[email protected]> 1 Part 1 Apache Thrift Overview Apache Thrift is an open source cross language serialization and RPC framework. With support for over 15 programming languages, Apache Thrift can play an important role in a range of distributed application development environments. As a serialization platform Apache Thrift enables efficient cross language storage and retrieval of a wide range of data structures. As an RPC framework, Apache Thrift enables rapid development of complete polyglot services in a few lines of code. Part 1 of this book takes you on a guided tour through the range of distributed development solutions empowered by Apache Thrift. You’ll see how the Apache Thrift framework fits into various communications schemes and also get a high level picture of the overall Apache Thrift architecture. ©Manning Publications Co. We welcome reader comments about anything in the manuscript - other than typos and other simple mistakes. These will be cleaned up during production of the book by copyeditors and proofreaders. http://www.manning-sandbox.com/forum.jspa?forumID=873 Licensed to Daniel Gavrila <[email protected]> 2 1 Introduction to Apache Thrift This chapter covers • How Apache Thrift supports polyglot system development • How Apache Thrift simplifies the creation of networked services • An introduction to the Apache Thrift modular serialization system • How to create a simple Apache Thrift multilanguage application This chapter introduces the Apache Thrift framework. We take a look at why Apache Thrift was created and how it helps programmers build high performance cross language services. We begin with a look at the growing need for multilanguage integration and examine the role Apache Thrift plays in distributed application development. The chapter also includes a tutorial walk through of a simple Apache Thrift application, demonstrating how easily cross language networked services can be created with Apache Thrift. 1.1 Polyglotism, the pleasure and the pain The number of programming languages in common commercial use has grown considerably in recent years. In 2003 80% of the Tiobe Index was attributed to six programming languages: Java, C, C++, Perl, Visual Basic and PHP. In 2013 it took twice as many languages to capture the same 80%, adding Objective-C, C#, Python, JavaScript and Ruby to the list. Increasingly developers and architects choose the programming language most suitable for the task at hand. A developer working on a Big Data project might decide Clojure is the best language to use, meanwhile folks down the hall may be doing front end work in JavaScript, while programmers upstairs might be working in C++ to improve I/O performance. Years ago this type of diversity would be rare at a single company, now it can be found within a single team. ©Manning Publications Co. We welcome reader comments about anything in the manuscript - other than typos and other simple mistakes. These will be cleaned up during production of the book by copyeditors and proofreaders. http://www.manning-sandbox.com/forum.jspa?forumID=873 Licensed to Daniel Gavrila <[email protected]> 3 Choosing a programming language uniquely suited to solving a particular problem can lead to productivity gains and better quality software. When the language fits the problem, friction is reduced, programming becomes more direct and code becomes Figure 1.1 - The Tiobe Index uses web search results to track programming simpler and easier to language popularity (www.tiobe.com) maintain. For example, in large scale data analysis, horizontal scaling is instrumental in achieving performance. Functional programming languages like Haskell, Scala and Clojure tend to fit naturally here, allowing analytic systems to scale without complex concurrency concerns. New platforms drive language adoption as well. Objective-C exploded in popularity when Apple released the iPhone, most programming for Android will be biased toward Java and the Windows Phone folks will likely be using C#. Those coding for the browser will have teams competent with JavaScript. Embedded systems shops are going to have strong C programming groups and high performance GUI applications will often be written in C++. These choices are driven by history as well as compelling technology underpinnings. Even when such groups are internally monoglots, languages mix and mingle as they collaborate across business boundaries. Many otherwise monoglot environments make use of a range of support languages for testing and prototyping. Dynamic programming languages such as Groovy and Ruby are often used for test and behavioral driven development solutions, while Perl and Python are popular for prototyping and PHP has a long history on the server side of the web. Platforms such as the Groovy based Gradle and the Ruby based Rake provide innovative build capabilities. Even firms that think they are monoglots may not be, given the proliferation of innovative language driven tools around the periphery or core application development. The Polyglot story is not all wine and song, however. Mastering a programming language is no small feat, not to mention the tools and support libraries that come with it. As this burden is multiplied with each new language, firms may experience diminishing returns. Introducing multiple languages into a product initiative can have numerous costs associated with cross language integration, developer training, and complexity in build and