in .NET Kevin Hazzard Jason Bock FOREWORD BY Rockford Lhotka MANNING www.it-ebooks.info Metaprogramming in .NET www.it-ebooks.info www.it-ebooks.info Metaprogramming in .NET KEVIN HAZZARD JASON BOCK MANNING SHELTER ISLAND www.it-ebooks.info For online information and ordering of this and other Manning books, please visit www.manning.com. The publisher offers discounts on this book when ordered in quantity. For more information, please contact Special Sales Department Manning Publications Co. 20 Baldwin Road PO Box 261 Shelter Island, NY 11964 Email: [email protected] ©2013 by Manning Publications Co. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps. Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books we publish printed on acid-free paper, and we exert our best efforts to that end. Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine. Manning Publications Co. Development editor: Cynthia Kane 20 Baldwin Road Copyeditor: Corbin Collins PO Box 261 Technical proofreader: Justin Chase Shelter Island, NY 11964 Proofreader: Elizabeth Martin Typesetter: Dennis Dalinnik Cover designer: Marija Tudor ISBN: 9781617290268 Printed in the United States of America 12345678910–MAL–18171615141312 www.it-ebooks.info brief contents PART 1DEMYSTIFYING METAPROGRAMMING ..............................1 1 ■ Metaprogramming concepts 3 2 ■ Exploring code and metadata with reflection 41 PART 2TECHNIQUES FOR GENERATING CODE ..........................63 3 ■ The Text Template Transformation Toolkit (T4) 65 4 ■ Generating code with the CodeDOM 101 5 ■ Generating code with Reflection.Emit 139 6 ■ Generating code with expressions 171 7 ■ Generating code with IL rewriting 199 PART 3LANGUAGES AND TOOLS ............................................221 8 ■ The Dynamic Language Runtime 223 9 ■ Languages and tools 267 10 ■ Managing the .NET Compiler 287 v www.it-ebooks.info www.it-ebooks.info contents foreword xiii preface xv acknowledgments xvii about this book xix about the cover illustration xxiii PART 1DEMYSTIFYING METAPROGRAMMING...................1 Metaprogramming concepts 3 1 1.1 Definitions of metaprogramming 6 1.2 Examples of metaprogramming 8 Metaprogramming via scripting 8 ■ Metaprogramming via reflection 11 ■ Metaprogramming via code generation 14 Metaprogramming via dynamic objects 29 1.3 Summary 39 Exploring code and metadata with reflection 41 2 2.1 The need for reflection 42 Creating extensible applications 42 ■ Manipulating code members at runtime 42 vii www.it-ebooks.info viii CONTENTS 2.2 Reading metadata and executing code 43 Obtaining the starting point 44 ■ Finding member information 46 ■ Gathering attribute data 47 Executing code 48 2.3 Impractical uses of reflection 49 Performance concerns with reflection 49 Brittleness and reflection 50 2.4 Practical uses of reflection 51 Automatically registering known types in WCF 52 Dynamic implementation of ToString 55 Invoking arbitrary methods on objects 58 Quick summary of reflection examples 61 2.5 Summary 62 PART 2TECHNIQUES FOR GENERATING CODE...............63 The Text Template Transformation Toolkit (T4) 65 3 3.1 Thinking of generics as templates 66 3.2 Introducing T4 69 T4 syntax basics 71 ■ Understanding T4’s block types 73 How T4 stitches together template blocks 74 ■ T4’s expression control block 75 ■ A brief history of T4 75 3.3 More useful T4 examples 77 Templates should be beautiful 83 3.4 T4 fundamentals 84 Directives and text blocks 84 ■ Control blocks 85 Handling indentation 88 3.5 Using T4 inside Visual Studio 93 How T4 uses the single file generator extension point 93 Creating a T4 template from Visual Studio 95 ■ More on the template directive 96 ■ Using the output directive 97 Using T4 to generate Visual Basic dynamically 98 3.6 Summary 100 Generating code with the CodeDOM 101 4 4.1 Understanding the CodeDOM 102 CodeDOM organization and types 103 ■ How statements and expressions fit together 105 www.it-ebooks.info CONTENTS ix 4.2 The code provider classes 106 Code provider instantiation 106 ■ Code generator supportable options 110 ■ Code provider services 112 4.3 Adding objects to a code graph 113 Creating a namespace with imports 113 ■ Adding a class to a namespace 115 ■ Adding a constructor to a class 116 Adding statements to a member 117 ■ Adding a property to a class 120 4.4 Metaprogramming with the CodeDOM 121 Using branching logic 121 ■ Referencing a member 124 Invoking methods 126 ■ Compiling assemblies 133 Dynamic invocation 135 4.5 Summary 137 Generating code with Reflection.Emit 139 5 5.1 Why Emitter classes? 140 Support for DSLs 140 ■ Moving reflection code into IL 141 Using .NET functionality not supported in your language 142 5.2 An overview of assembly internals 144 Transforming high-level languages 144 ■ Member layouts in assemblies and keywords 147 5.3 A lightning tour of opcodes 148 The mnemonic patterns for opcodes 148 ■ Using local variables 150 ■ Accessing fields 151 ■ Creating objects 152 Calling methods 152 ■ Controlling code flow 153 Exception handling 154 5.4 Creating dynamic assemblies 155 Building a dynamic version of ToString() 155 Adding debugging support 160 ■ Verifying results with peverify 162 ■ Using ILDasm to cheat 164 5.5 Lightweight code generation with dynamic methods 165 When creating an assembly is too much 165 ■ Creating shim methods 166 ■ Using caching for performance 167 Disadvantages of DynamicMethod 168 5.6 Summary 170 Generating code with expressions 171 6 6.1 Expression-oriented programming 172 Understanding code as data 172 ■ Expressions take metaprogramming mainstream 174 www.it-ebooks.info x CONTENTS 6.2 Making dynamic methods with LINQ Expressions 176 Understanding LINQ Expressions 176 ■ Generating expressions at runtime 178 ■ Comparison with dynamic methods 182 6.3 Using expressions effectively 184 Debugging expressions 184 ■ Mutating expression trees 187 6.4 Evolving expression trees 189 The essence of genetic programming 189 ■ Applying GAs to expressions 191 6.5 Summary 198 Generating code with IL rewriting 199 7 7.1 The case for code injection 200 Repeated implementations of coding patterns 200 Code restructuring (Code Contracts) 201 7.2 Creating an injection framework 203 What’s Cecil? 203 ■ Weaving code with Cecil 203 Creating an MSBuild task 210 7.3 Debugging injected code 212 Clearing up debugging confusion 212 ■ Loading and saving debug information 212 ■ Issues with adding debugging information 213 ■ Adding debugging information for injected code 214 7.4 Summary 218 PART 3LANGUAGES AND TOOLS.................................221 The Dynamic Language Runtime 223 8 8.1 The simplest dynamic classes 224 The ExpandoObject class 224 ■ The DynamicObject class 227 Parsing the Open Data Protocol dynamically 231 8.2 The DLR hosting model 239 Runtimes, engines, and scopes 241 ■ Adding a rules engine to your application 252 8.3 Summary 264 www.it-ebooks.info CONTENTS xi Languages and tools 267 9 9.1 A survey of languages 268 C# and expression limitations 268 ■ Boo and metaprogramming 269 ■ Nemerle and metaprogrammg 275 9.2 A survey of tools 277 What is Spring.NET? 277 ■ Intercepting property usage with Spring.NET 278 ■ What is PostSharp? 280 Intercepting object creation with PostSharp 280 Implementing Equals() and GetHashCode() 282 A quick dive into the internals of PostSharp 285 9.3 Summary 286 Managing the .NET Compiler 287 10 10.1 Opening up the compiler 288 The current state of affairs: a black box 288 ■ Limitations for metaprogramming 288 ■ What Roslyn provides: a white box 289 ■ What’s in (and not in) the CTP 290 10.2 Understanding the basics of Roslyn 291 Running code snippets with the script engine 291 Creating dynamic assemblies with Roslyn 294 ■ What is a mock? 294 ■ Generating the mock code 295 ■ Compiling the mock code 300 ■ Understanding trees 302 10.3 Interacting with code in Visual Studio 303 Creating a IsOneWay warning 303 ■ Defining the Code Issue 303 ■ Defining the OneWayOperation code actions 306 ■ Viewing the results 307 Autoarrange code 309 ■ Specifying the algorithm to reformat the code 309 ■ Defining the core parts of the refactoring project 310 ■ Creating a code action 311 Viewing the results 314 10.4 Summary 315 appendix A Metaprogramming in Windows 8 316 appendix B Usage guide 319 index 321 www.it-ebooks.info www.it-ebooks.info foreword When I think about metaprogramming I view it through three sets of experience: as a computer scientist, a business developer, and a .NET framework author. From a computer science perspective, it is clear that our industry has been largely stagnant from a language perspective for an extremely long time. The slow evolution of 3GLs (third-generation languages) from C to C++ to Java to C# has resulted in incremental improvements, but no major leaps in terms of developer productivity, maintainability of code, reduction of complexity, or other meaningful metrics. (I chose the C language progression in my example because it is perhaps the most widely known. Comparable progressions exist for BASIC, Pascal, and many other lan- guage families.) Metaprogramming offers interesting possibilities around the creation of domain- specific languages and other abstraction concepts that could eventually break us out of the 3GL world we’ve lived in for the past 20-30 years. Although this book doesn’t focus on such a long-term goal, I think you can use Metaprogramming in .NET as a start- ing point to gain valuable perspective on myriad core ideas that might inspire you to think more about the future of our industry.