Pro DLR in .NET 4
Chaur Wu
Pro DLR in .NET 4 Copyright © 2010 by Chaur Wu All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and the publisher. ISBN 978-1-4302-3066-3 ISBN 978-1-4302-3067-0 (eBook) Printed and bound in the United States of America 9 8 7 6 5 4 3 2 1 Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. President and Publisher: Paul Manning Lead Editor: Jonathan Gennick Technical Reviewer: Scott Isaacs Editorial Board: Steve Anglin, Mark Beckner, Ewan Buckingham, Gary Cornell, Jonathan Gennick, Jonathan Hassell, Michelle Lowman, Matthew Moodie, Duncan Parkes, Jeffrey Pepper, Frank Pohlmann, Douglas Pundick, Ben Renow-Clarke, Dominic Shakeshaft, Matt Wade, Tom Welsh Coordinating Editor: Jennifer L. Blackwell Copy Editor: Sharon Terdeman Compositor: Bytheway Publishing Services Indexer: Brenda Miller Artist: Integra Software Services Pvt. Ltd. Cover Designer: Anna Ishchenko Distributed to the book trade worldwide by Springer Science+Business Media, LLC., 233 Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail [email protected], or visit www.springeronline.com. For information on translations, please e-mail [email protected], or visit www.apress.com. Apress and friends of ED books may be purchased in bulk for academic, corporate, or promotional use. eBook versions and licenses are also available for most titles. For more information, reference our Special Bulk Sales–eBook Licensing web page at www.apress.com/info/bulksales. The information in this book is distributed on an “as is” basis, without warranty. Although every precaution has been taken in the preparation of this work, neither the author(s) nor Apress shall have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the information contained in this work. I want to dedicate this book to Sarah, Everett, Cedric, and Chiachi
CONTENTS AT A GLANCE
Contents at a Glance
About the Author...... xiv About the Technical Reviewer ...... xv Acknowledgments ...... xvi Introduction ...... xvii PART 1 DLR Fundamentals ...... 1 Chapter 1: Introduction to DLR ...... 3 Chapter 2: DLR Expression...... 27 Chapter 3: Late Binding and Caching...... 65 Chapter 4: Late Binding and Interoperability...... 87 Chapter 5: Dynamic Objects...... 109 Chapter 6: DLR Hosting API...... 133 PART 2 Applying the DLR...... 163 Chapter 7: DLR and Aspect-Oriented Programming ...... 165 Chapter 8: Metaprogramming...... 185 Chapter 9: Stitch — A DSL for Hosting Languages ...... 211 Chapter 10: Application Scripting...... 239 Chapter 11: DLR in Silverlight...... 251 Chapter 12: Dynamic Languages on JVM ...... 275 Index ...... 297
iv CONTENTS
Contents
About the Author...... xiv About the Technical Reviewer ...... xv Acknowledgments ...... xvi Introduction ...... xvii PART 1 DLR Fundamentals ...... 1 Chapter 1: Introduction to DLR ...... 3 Setting Up Code Examples...... 4 Software Requirements...... 4 Installing the DLR, IronPython, and IronRuby ...... 5 Hello World Examples...... 6 Implementing REPL for the Hello Language ...... 9 Practical Uses for the DLR ...... 9 Application Scripting and DSL ...... 10 XML Builder ...... 10 Aspect-Oriented Programming...... 11 Runtime ...... 13 Runtime vs. Run Time...... 14 Run Time vs. Compile Time...... 14 Dynamic vs. Static...... 16 Dynamic Typing...... 17 Dynamic Dispatch...... 20 Language...... 21 Programming Languages in Practice...... 22
v CONTENTS
Putting It Together...... 24 Summary ...... 25 Chapter 2: DLR Expression...... 27 DLR Expression as a Language...... 27 Code as Data ...... 28 A Common Denominator like CLR...... 28 Concrete Syntax and Serialization...... 29 Expressions vs. Statements...... 29 Expression Type and Kind...... 31 Binary Expressions ...... 33 Flow Control Expressions ...... 36 If-Then-Else Expressions...... 37 Switch Expressions ...... 38 Scopes and Name Binding...... 39 Lexical vs. Dynamic Scope...... 40 BlockExpression and Lexical Scoping ...... 41 Lambda Expressions and Closure...... 44 The GotoExpression Class...... 46 While Loops ...... 48 Dynamic Expressions...... 49 Index Expressions...... 52 Expression Abstraction, Reduction and Extension...... 53 Immutability and the Visitor Pattern ...... 55 Visitor Pattern in General...... 56 Visitor Pattern in DLR Expression ...... 57 Expression Visitor Examples...... 59 Summary ...... 63
vi CONTENTS
Chapter 3: Late Binding and Caching...... 65 Binding...... 65 Call Sites and Early Binding...... 66 Call Sites and Late Binding...... 67 DLR Binders ...... 68 Set Up Code Examples...... 69 Making a Debug Build of the DLR for .NET 2.0 ...... 69 Developing for Both .NET 2.0 and .NET 4.0 ...... 69 The CallSiteBinder Class...... 71 DLR Call Sites ...... 72 Binding Restrictions and Rules...... 73 Checking Binding Rules in Debug Mode...... 75 Caching...... 78 Three Cache Levels ...... 78 Late-Binding Context...... 80 L0 Cache Example ...... 82 L1 Cache Example ...... 83 L2 Cache Example ...... 84 Creating Canonical Binders ...... 86 Summary ...... 86 Chapter 4: Late Binding and Interoperability...... 87 Language Interoperability...... 87 Static and Dynamic Objects...... 91 Late-Binding Logic in Two Places...... 92 Late Binding Logic in Binders...... 92 Late-Binding Logic in Dynamic Objects...... 94 Late-Bound Actions ...... 94
vii CONTENTS
Examples ...... 94 Common Type System...... 96 Class Hierarchy of Binders ...... 99 Implement a Custom Binder Class...... 100 Interoperability Protocol ...... 102 Summary ...... 106 Chapter 5: Dynamic Objects...... 109 What is a Dynamic Object?...... 109 IDynamicMetaObjectProvider Interface ...... 111 Dynamic Meta Objects...... 112 DynamicMetaObject and Binding Logic...... 113 DynamicMetaObject and Binding Result ...... 114 Interoperability...... 115 DynamicObject Class...... 119 XML Builder ...... 122 Summary ...... 130 Chapter 6: DLR Hosting API...... 133 Life Without the DLR Hosting API...... 133 Using a Static Language’s Code in Another Static Language...... 134 Using a Static Language’s Code in a Dynamic Language...... 134 Using a Dynamic Language’s Code in a Static Language...... 135 Using a Dynamic Language’s Code in Another Dynamic Language ...... 136 Overview of the DLR Hosting API...... 136 Major Classes in the API...... 137 The Tale of Two APIs ...... 138 The DLR Hosting API in Relation to Binders and Dynamic Objects...... 139 Using Script Runtime to Execute Code ...... 139
viii CONTENTS
Configuring the Languages You Want to Speak...... 140 Configuring Script Runtime Declaratively...... 141 Configuring Script Runtime Programmatically ...... 142 Scripting an Object Model ...... 143 Script Scopes...... 145 Value and Reference Variables...... 146 Global Scope and Variable Passing ...... 147 Language Neutral Scope and Variable Passing...... 148 Level Two Use of the DLR Hosting API...... 150 Compiling Code...... 150 Loading Assemblies into Script Runtime...... 152 Creating Python Class Instances Using Object Operations...... 153 Level Three Use of the DLR Hosting API ...... 154 Script Host...... 154 Object Operations ...... 156 Remote Script Runtime ...... 156 .NET Remoting Quick Tour...... 158 Running Script Runtime in a Separate Process...... 159 Summary ...... 161 PART 2 Applying the DLR...... 163 Chapter 7: DLR and Aspect-Oriented Programming ...... 165 Aspect-Oriented Programming ...... 165 Cross-Cutting Concerns...... 165 Advice, Join Points, and Pointcuts ...... 166 An Example...... 167 A Test Run ...... 170 AOP for Dynamic Objects...... 171 Understanding the Framework...... 171
ix CONTENTS
Implementing the Framework ...... 172 Integration with Spring.NET AOP ...... 174 Getting the AOP Advisors...... 175 Implementing Advice...... 177 Applying Advice ...... 178 Cutting Across Dynamic and Static Objects ...... 180 Summary ...... 183 Chapter 8: Metaprogramming...... 185 Overview of Metaprogramming ...... 185 Changing Class Definitions...... 186 Ruby ...... 187 Python...... 189 DLR ...... 191 LINQ Query Provider...... 196 Understanding the End Goal ...... 196 Implementing the Query Class...... 196 Implementing the QueryProvider Class ...... 198 Implementing QueryExpressionVisitor...... 200 Data Access...... 201 Static Data Access...... 202 Dynamic Data Access...... 204 Generated Data Access ...... 206 Summary ...... 210 Chapter 9: Stitch — A DSL for Hosting Languages ...... 211 The Need for Stitch...... 211 Syntax of the Stitch Language...... 212 Requirements for the Example ...... 214 Software Requirements...... 214
x CONTENTS
Organization of the Code ...... 214 Stitch in Use...... 215 Being Declarative ...... 215 Hosting DLR- and Non-DLR-Based Languages...... 216 Hosting Stitch Itself ...... 217 Executing in Parallel...... 217 Stitch Language Grammar...... 218 Setting Up Eclipse and ANTLR...... 219 Defining the Grammar ...... 220 Test-Driving the Grammar...... 224 The Stitch Runtime ...... 225 Overview of the Runtime ...... 226 The Script Engine ...... 227 Function Execution Coordinator...... 228 Parallel Extensions for .NET...... 230 Script Runner...... 231 Running DLR-based Language Code ...... 233 Language Plug-In...... 234 Develop a Stitch Plug-In for PowerShell...... 234 Configuring a Plug-In...... 237 Summary ...... 238 Chapter 10: Application Scripting...... 239 Ball World ...... 239 Software Requirements...... 240 Application Architecture ...... 241 Application Object Model...... 241 Application Scripting...... 244 The Physics Engine...... 246
xi CONTENTS
User Interface ...... 248 Summary ...... 250 Chapter 11: DLR in Silverlight...... 251 Different Client Side Web Scripting Approaches ...... 251 Apache HTTP Server Configurations...... 252 Using the Hosted Gestalt Components ...... 253 Hosting the Gestalt Components ...... 254 Dissecting the Gestalt Components...... 255 Scripting HTML ...... 257 Scripting XAML ...... 260 DLR Settings ...... 263 Speak Your Own Language in Silverlight...... 264 Software Requirements...... 264 Build DLR for Silverlight...... 265 The Hello Language ...... 265 Hello Console in Silverlight...... 266 Gestalt-like Hello Console on Silverlight...... 270 Summary ...... 273 Chapter 12: Dynamic Languages on JVM ...... 275 Quick Comparisons...... 275 Python and Ruby on JVM...... 276 Hosting Python Code in Java Programs...... 277 Hosting Ruby Code in Java Programs...... 279 Overview of the BoolScript Example...... 280 BoolScript Language...... 281 Script Engine Factory...... 282
xii CONTENTS
Script Engine Discovery Mechanism ...... 284 Bindings, Scope, and Context...... 286 BoolScript Engine ...... 289 Compile BoolScript Code ...... 291 Run BoolScript Code as Invocable Function ...... 294 Summary ...... 295 Index ...... 297
xiii About the Author
Chaur Wu is an author and developer with a passion for model-based, language-oriented software development. He works extensively with .NET, with experience going back to the initial beta release in 2000. He has two successful books to his credit: Professional Design Patterns in VB.NET and Professional UML with Visual Studio .NET, both published by Wrox Press. Wu has implemented a number of domain-specific and general-purpose languages—for work, for study, and for fun. All of the projects he developed for Pro DLR in .NET 4 are collectively hosted on an open source project site at http://code.google.com/p/dpier/. You are welcome to visit the web site and check out the latest development there.
xiv INTRODUCTION
About the Technical Reviewer
Scott Isaacs spent the first 25 years of his life in California's Central Valley. He has long since moved to southeast Wisconsin, where he designs software. He also runs the WI .NET Users Group, one of the oldest and largest .NET community groups around. Isaacs lives in a Milwaukee suburb with his wife and children. He occasionally blogs at http://tapmymind.com/, and can also be found online at http://twitter.com/daughtkom/.
xv INTRODUCTION
Acknowledgments
I'd like to thank the fabulous folks at Apress, especially Jonathan Gennick and Jennifer Blackwell for their support throughout the writing of the book. There were times when it was impossible for me to make any progress on the writing and I really appreciated Jonathan and Jennifer's understanding and encouragement during those tough periods. I'd also like to thank Scott Isaacs for reviewing the book and providing valuable feedback. Throughout my career as a software engineer, I’ve continually benefited from the many people I work with, and from open source communities. For this book, I'd like to express my thanks to the DLR forum and the wonderful folks who provided prompt answers to the questions I posted on the forum. A great part of this book was written when I was in Taiwan at my parents' place. I enjoyed my time with them and I want them to know that I love them very much. The rest of the book was written in Fremont, California with the love and support of my wife. Without that love and support, this book would not be what it is.
xvi INTRODUCTION
Introduction
The book you’re holding focuses on Microsoft’s Dynamic Language Runtime (DLR) and what it can do for you in your day-to-day programming work. Many think the DLR is an esoteric platform that matters only if you happen to be one of the very few who are implementing languages such as Python and Ruby atop the .NET Framework. That belief is far from the truth. The DLR puts a number of exciting capabilities at your disposal. Implementing languages is actually pretty far down on that list. One of the most obvious things to do with the DLR is to mix and match code and objects from different languages. Do you have an object in Python that does what you need? Use the DLR to make that object usable from your C# code. Going further, you can mix and match dynamic and compiled languages in ways that are convenient, that allow you to choose the best tool for the job at hand within your overall application. This ability to mix and match leads directly to using dynamic languages as scripting languages within your applications. Going even further, you can dive in and make the DLR your basis for implementing application- and domain-specific scripting languages. Aspect-oriented programming and runtime code generation are two other techniques made possible by the DLR. You’ll find examples of both in this book. You’ll also find clear examples that show the details of using the DLR. You’ll learn about core components such as LINQ expressions, call sites, binders, and dynamic objects. You’ll see how to apply those components to the problem of combining dynamic and compiled languages into a single application. You’ll end up with the ability to apply whatever language or language-library is most productive given the programming problem you’re trying to solve at any given moment. You’ll truly be able to apply the best tool to the job at hand.
Prerequisites Chapter 1 describes the prerequisites in detail, explaining the software you need to install in order to mimic my own configuration so you can run the examples in this book. In general, though, you should be comfortable programming in C#. You should also know at least one of the common scripting languages, such as Python or Ruby. If you can compile and run a C# program and you can execute Python or Ruby code, you have what it takes to get the most out of this book.
Structure of the Book (or How this Book is Organized…) This book consists of two parts:
Part I deals with the fundamentals. You are introduced to the DLR, and to the core functionality that the API provides. This is where you’ll learn the mechanics of using the DLR.
xvii INTRODUCTION
Part II explores applying the DLR to various ends. You’ll find chapters devoted to such topics as aspect-oriented programming, application scripting, domain-specific languages, meta- programming, and more.
Part I consists of six chapters. Chapter 1 stands out in that it gives you a whirlwind tour — code included! — showing all you can accomplish using the DLR. If you want to get the lay of the land, to know what the possibilities are, Chapter 1 is what you should read. Chapters 2-6 then go into detail on the various mechanics of using the DLR. Part II also consists of six chapters. Here the emphasis is on applying the DLR to specific programming techniques. You’ll begin, for example, by learning how the DLR helps enable aspect- oriented programming. You’ll be introduced to STITCH, a domain-specific language implemented atop the DLR that makes it easier to host languages such as Python and Ruby from within your C# programs. And you’ll learn about metaprogramming, application scripting, and how to run DLR applications on the Silverlight platform.
Obtaining the Source Code Source code is available for the examples shown in this book. You can download that source code from the book’s catalog page on the Apress web site. Here is the URL for that page: http://apress.com/book/view/1430230665 Once there, look under the book’s cover image for the catalog page section entitled “Book Resources,” where you’ll see a link for “Source Code.” Click that link to download a zip archive containing the example code for this book. When you have the download, refer to Chapter 1. There you’ll find instructions on setting up the code examples. You’ll also find a description of the directory structure used in the example archive.
xviii