Python & XML Christopher A. Jones Fred L. Drake, Jr. Publisher: O'Reilly First Edition January 2002 ISBN: 0-596-00128-2, 384 pages Python is an ideal language for manipulating XML, and this new volume gives you a solid foundation for using these two languages together. Complete with practical examples that highlight common application tasks, the book starts with the basics then quickly progresses to complex topics like transforming XML with XSLT and querying XML with XPath. It also explores more advanced subjects, such as SOAP and distributed web services. Copyright © 2002 O'Reilly & Associates, Inc. All rights reserved. Printed in the United States of America. Published by O'Reilly & Associates, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O'Reilly & Associates books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://safari.oreilly.com). For more information contact our corporate/institutional sales department: 800-998-9938 or [email protected]. Nutshell Handbook, the Nutshell Handbook logo, and the O'Reilly logo are registered trademarks of O'Reilly & Associates, Inc. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and O'Reilly & Associates, Inc. was aware of a trademark claim, the designations have been printed in caps or initial caps. The association between the image of elephant shrews and Python and XML is a trademark of O'Reilly & Associates, Inc. While every precaution has been taken in the preparation of this book, the publisher assumes no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein. IT-SC book 1 Dedication Preface Audience Organization Conventions Used in This Book How to Contact Us Acknowledgments 1. Python and XML 1.1 Key Advantages of XML 1.2 The XML Specifications 1.3 The Power of Python and XML 1.4 What Can We Do with It? 2. XML Fundamentals 2.1 XML Structure in a Nutshell 2.2 Document Types and Schemas 2.3 Types of Conformance 2.4 Physical Structures 2.5 Constructing XML Documents 2.6 Document Type Definitions 2.7 Canonical XML 2.8 Going Beyond the XML Specification 3. The Simple API for XML 3.1 The Birth of SAX 3.2 Understanding SAX 3.3 Reading an Article 3.4 Searching File Information 3.5 Building an Image Index 3.6 Converting XML to HTML 3.7 Advanced Parser Factory Usage 3.8 Native Parser Interfaces 4. The Document Object Model 4.1 The DOM Specifications 4.2 Understanding the DOM 4.3 Python DOM Offerings 4.4 Retrieving Information 4.5 Changing Documents 4.6 Building a Web Application 4.7 Going Beyond SAX and DOM 5. Querying XML with XPath 5.1 XPath at a Glance 5.2 Where Is XPath Used? 5.3 Location Paths 5.4 XPath Arithmetic Operators 5.5 XPath Functions 5.6 Compiling XPath Expressions IT-SC book 2 6. Transforming XML with XSLT 6.1 The XSLT Specification 6.2 XSLT Processors 6.3 Defining Stylesheets 6.4 Using XSLT from the Command Line 6.5 XSLT Elements 6.6 A More Complex Example 6.7 Embedding XSLT Transformations in Python 6.8 Choosing a Technique 7. XML Validation and Dialects 7.1 Working with DTDs 7.2 Validation at Runtime 7.3 The BillSummary Example 7.4 Dialects, Frameworks, and Workflow 7.5 What Does ebXML Offer? 8. Python Internet APIs 8.1 Connecting Web Sites 8.2 Working with URLs 8.3 Opening URLs 8.4 Connecting with HTTP 8.5 Using the Server Classes 9. Python, Web Services, and SOAP 9.1 Python Web Services Support 9.2 The Emerging SOAP Standard 9.3 Python SOAP Options 9.4 Example SOAP Server and Client 9.5 What About XML-RPC? 10. Python and Distributed Systems Design 10.1 Sample Application and Flow Analysis 10.2 Understanding the Scope 10.3 Building the Database 10.4 Building the Profiles Access Class 10.5 Creating an XML Data Store 10.6 The XML Switch 10.7 Running the XML Switch 10.8 A Web Application A. Installing Python and XML Tools A.1 Installing Python A.2 Installing PyXML A.3 Installing 4Suite B. XML Definitions B.1 XML Definitions C. Python SAX API D. Python DOM API D.1 4DOM Extensions IT-SC book 3 E. Working with MSXML3.0 E.1 Setting Up MSXML3.0 E.2 Basic DOM Operations E.3 MSXML3.0 Support for XSLT E.4 Handling Parsing Errors E.5 MSXML3.0 Reference F. Additional Python XML Tools F.1 Pyxie F.2 Python XML Tools F.3 XML Schema Validator F.4 Sab-pyth F.5 Redfoot F.6 XML Components for Zope F.7 Online Resources Colophon IT-SC book 4 Dedication We would like to dedicate this book to Frank Willison, O'Reilly Editor- in-Chief and Python Champion ——Christopher A. Jones and Fred L. Drake, Jr. Frank will be remembered in the Python community for the several great Python books that he made possible, memories of his participation in many Python conferences, and his Frankly Speaking columns. The Python world (and the world at large) won't be the same without Frank. ——Guido van Rossum, Python creator IT-SC book 5 Preface This book comes to you as a result of the collaboration of two authors who became interested in the topic in very different ways. Hopefully our motivations will help you understand what we each bring to the book, and perhaps prove to be at least a little entertaining as well. Chris Jones started using XML several years ago, and began using Python more recently. As a consultant for major companies in the Seattle area, he first used XML as the core data format for web site content in a home-grown publishing system in 1997. But he really became an XML devotee when developing an open source engine, which eventually became the key technology for Planet 7 Technologies. As a consultant, he continues to use XML on an almost daily basis for everything from configuration files to document formats. Chris began dabbling in Python because he thought it was a clean, object-oriented alternative to Perl. A long-time Unix user (but one who frequently finds himself working with Windows in Seattle), he has grown accustomed to scripting languages that place the full Unix API in the hands of developers. Having used far too much Java and ASP in web development over the years, he found Python a refreshing way to keep object-orientation while still accessing Unix sockets and threads—all with the convenience of a scripting language. The combination of Python and XML brings great power to the developer. While XML is a potent technology, it requires the programmer to use objects, interfaces, and strings. Python does so as well, and therefore provides an excellent playpen for XML development. The number of XML tools for Python is growing all the time, and Chris can produce an XML solution in far less time using Python than he can with Java or C++. Of course, the cross-platform nature of Python keeps our work consistently usable whether we're developing on Windows, Linux, or a Unix variant—the combination of which we both seem to find powerful. Fred Drake came to Python and XML from a different avenue, arriving at Python before XML. He discovered Python while in graduate school experimenting with a number of programming languages. After recognizing Python as an excellent language for rapid development, he convinced his advisors that he should be able to write his masters project using Python. In the course of developing the project, he became increasingly interested in the Python community. He then made his first contributions to the Python standard library, and in so doing became noticed by a group of Python programmers working on distributed systems projects at the research organization of CNRI. The group was led by Guido van Rossum, the creator of Python. Fred joined the team and learned more about distributed systems and gluing systems together than he ever expected possible, and he loved it. While still in graduate school, Fred argued that Python's documentation should be converted to a more structured language called SGML. After a few years at CNRI, he began to do just that, and was able to sink his teeth into the documentation more vigorously. The SGML migration path eventually changed to an XML migration path as XML acceptance grew. Though that goal has not yet been achieved (he is still working on it), Fred has substantially changed the way the documentation is maintained, and it now represents one of the most structured applications of the typesetting and document markup system developed by Donald Knuth and Leslie Lamport. Over time, the team from CNRI became increasingly focused on the development of Python, and moved on to form PythonLabs. Fred remained active in XML initiatives around Python and IT-SC book 6 pushed to add XML support to the standard library. Once this was achieved, he returned to the task of migrating the Python documentation to XML, and hopes to complete this project soon. Audience This book is for anyone interested in learning about using Python to build XML applications. The bulk of the material is suited for programmers interested in using XML as a data interchange format or as a transformable format for web content, but the first half of the book is also useful to those interested in building more document-oriented applications.