Imagine. Create. Deploy. Inspired? So Are We.
Inspiration is all around us. From beautiful screens on the web to well-designed reports. New devices push the development envelope and ask that we consider new technologies. The latest release, DevExpress 12.2, delivers the tools you need to build the multi-channel solutions you can imagine: Windows 8-inspired applications with live tiles perfect for Microsoft Surface, multi-screen iOS and Android apps. It’s all possible. Let’s see what develops.
Download your 30-day trial at www.DevExpress.com Copyright 1998-2013 Developer Express, Inc. All rights reserved. All trademarks are property of their respective owners.
Untitled-9 1 1/8/13 2:10 PM THE MICROSOFT JOURNAL FOR DEVELOPERS FEBRUARY 2013 VOL 28 NO 2
magazine
JavaScript API for Offi ce...... 20
Exploring the New JavaScript API for Offi ce COLUMNS Stephen Oliver and Eric Schmidt ...... 20 CUTTING EDGE Essential Facebook Async Causality Chain Tracking Programming: Andrew Stasyuk ...... 32 The JavaScript SDK Dino Esposito, page 6
Building a Simple Comet Application WINDOWS WITH C++ in the Microsoft .NET Framework Creating Desktop Apps Derrick Lau ...... 42 with Visual C++ 2012 Kenny Kerr, page 12
Detecting Abnormal Data Using TEST RUN k-Means Clustering Naive Bayes Classifi cation with C# ...... James McCaffrey 54 James McCaffrey, page 70 Taming the Event Stream: THE WORKING PROGRAMMER Fast Approximate Counting .NET Collections, Part 2: Michael Meijer ...... 64 Working with C5 Ted Neward, page 76
MODERN APPS Create Windows Store Apps with HTML5 and JavaScript Rachel Appel, page 80
DIRECTX FACTOR Constructing Audio Oscillators for Windows 8 Charles Petzold, page 84
DON’T GET ME STARTED What’s Up, Doc? David Platt, page 88 Compatible with Microsoft® Visual Studio® 2012
At Your Fingertips
Untitled-1 2 12/5/12 11:08 AM Download your free trial infragistics.com/
Infragistics Sales US 800 231 8588 • Europe +44 (0) 800 298 9055 • India +91 80 4151 8042 • APAC +61 3 9982 4545
Copyright 1996-2013 Infragistics, Inc. All rights reserved. Infragistics and NetAdvantage are registered trademarks of Infragistics, Inc. The Infragistics logo is a trademark of Infragistics, Inc. All other trademarks or registered trademarks are the respective property of their owners.
Untitled-1 3 12/5/12 11:08 AM FEBRUARY 2013 VOLUME 28 NUMBER 2 magazine
BJÖRN RETTIG Director MOHAMMAD AL-SABT Editorial Director/[email protected] PATRICK O’NEILL Site Manager
MICHAEL DESMOND Editor in Chief/[email protected] DAVID RAMEL Technical Editor SHARON TERDEMAN Features Editor WENDY HERNANDEZ Group Managing Editor KATRINA CARRASCO Associate Managing Editor
SCOTT SHULTZ Creative Director JOSHUA GOULD Art Director SENIOR CONTRIBUTING EDITOR Dr. James McCaffrey CONTRIBUTING EDITORS Rachel Appel, Dino Esposito, Kenny Kerr, Julie Lerman, Ted Neward, Charles Petzold, David S. Platt, Bruno Terkaly, Ricardo Villalobos
Henry Allain President, Redmond Media Group Michele Imgrund Sr. Director of Marketing & Audience Engagement Tracy Cook Director of Online Marketing Irene Fincher Audience Development Manager ADVERTISING SALES: 818-674-3416/[email protected] Dan LaBianca Group Publisher Chris Kourtoglou Regional Sales Manager Danna Vedder Regional Sales Manager/Microsoft Account Manager Jenny Hernandez-Asandas Director, Print Production Serena Barnes Production Coordinator/[email protected]
Neal Vitale President & Chief Executive Offi cer Richard Vitale Senior Vice President & Chief Financial Offi cer Michael J. Valenti Executive Vice President Christopher M. Coates Vice President, Finance & Administration Erik A. Lindgren Vice President, Information Technology & Application Development David F. Myers Vice President, Event Operations Jeffrey S. Klein Chairman of the Board
MSDN Magazine (ISSN 1528-4859) is published monthly by 1105 Media, Inc., 9201 Oakdale Avenue, Ste. 101, Chatsworth, CA 91311. Periodicals postage paid at Chatsworth, CA 91311-9998, and at additional mailing offi ces. Annual subscription rates payable in US funds are: U.S. $35.00, International $60.00. Annual digital subscription rates payable in U.S. funds are: U.S. $25.00, International $25.00. Single copies/back issues: U.S. $10, all others $12. Send orders with payment to: MSDN Magazine, P.O. Box 3167, Carol Stream, IL 60132, email [email protected] or call (847) 763-9560. POSTMASTER: Send address changes to MSDN Magazine, P.O. Box 2166, Skokie, IL 60076. Canada Publications Mail Agreement No: 40612608. Return Undeliverable Canadian Addresses to Circulation Dept. or XPO Returns: P.O. Box 201, Richmond Hill, ON L4B 4R5, Canada. Printed in the U.S.A. Reproductions in whole or part prohibited except by written permission. Mail requests to “Permissions Editor,” c/o MSDN Magazine, 4 Venture, Suite 150, Irvine, CA 92618.
Legal Disclaimer: The information in this magazine has not undergone any formal testing by 1105 Media, Inc. and is distributed without any warranty expressed or implied. Implementation or use of any information contained herein is the reader’s sole responsibility. While the information has been reviewed for accuracy, there is no guarantee that the same or similar results may be achieved in all environments. Technical inaccuracies may result from printing errors and/or new developments in the industry.
Corporate Address: 1105 Media, Inc., 9201 Oakdale Ave., Ste 101, Chatsworth, CA 91311, www.1105media.com Media Kits: Direct your Media Kit requests to Matt Morollo, VP Publishing, 508-532-1418 (phone), 508-875-6622 (fax), [email protected]
Reprints: For single article reprints (in minimum quantities of 250-500), e-prints, plaques and posters contact: PARS International, Phone: 212-221-9595, E-mail: [email protected], www.magreprints.com/ QuickQuote.asp
List Rental: This publication’s subscriber list, as well as other lists from 1105 Media, Inc., is available for rental. For more information, please contact our list manager, Merit Direct. Phone: 914-368-1000; E-mail: [email protected]; Web: www.meritdirect.com/1105 All customer service inquiries should be sent to [email protected] or call 847-763-9560.
Printed in the USA Untitled-13 1 1/7/13 12:56 PM EDITOR’S NOTE MICHAEL DESMOND
Inside Modern Apps
This month MSDN Magazine welcomes Rachel Appel and her writing back-end enterprise applications in COBOL. She struck Modern Apps column. Readers of the magazine and Web site might out on her own as an independent consultant, trainer and mentor, be familiar with Appel’s work. She wrote the popular Web Dev earning MVP recognition from Microsoft , before joining Microsoft Report column on the MSDN Magazine Web site up until Sep- as a technical evangelist. Today, she spends a great deal of her time tember of last year. And she appeared in our Windows 8 Special giving talks at conferences, working with customers and blogging Edition, published in October, with a look at the unique aspects about development issues. of the Windows Store application lifecycle. When I asked Appel how a former COBOL programmer ends Th is month, Modern Apps debuts with a quick rundown of what up helping lead the charge on Windows Store app development, you need to get started with Windows Store app development, and she couldn’t resist a joke—“Th e short answer? It’s new and shiny”— explores the new features and capabilities that enable developers before providing a more serious response. to create powerful applications for Windows 8 and the Windows “I enjoy learning about new technologies and the latest in soft - Runtime. Going forward, you can expect Appel to dig deeper into ware development,” Appel says. “As a tech evangelist, being out in the Windows Store app dev experience. In the March issue, look the public allows me to see and work with all kinds of awesome for the column to explore the important topic of data access and ideas and code.” storage for Windows Store apps. She goes on to praise some of the built-in features of Windows 8, Appel’s column arrives a few short months aft er Bruno Terkaly including Search and Share contracts, the rich sensor platforms and and Ricardo Villalobos came on board as authors of the new the device APIs. “You can access it all through open, standard HTML5 Windows Azure Insider column. And just last month Charles and ES5 [ECMAScript 5] if you want, or you can use C#/Visual Petzold re-branded his column as DirectX Factor, refl ecting his Basic/C++ and XAML, as all languages have parity.” focus on the powerful DirectX development infrastructure in the In her community engagements, Appel says she sometimes fi elds Windows Runtime. questions about the Windows Runtime and its relationship to the The changes reflect the significant progress we’ve seen from Microsoft .NET Framework. Her message: Th e .NET Framework Microsoft in updating its developer infrastructure. Windows 8 is not going anywhere. and the Windows Runtime present a compelling target for an “Windows Runtime is not a .NET replacement. Rather, many of the incredibly broad range of software developers, and the new WinRT APIs are wrappers around classic .NET or Win32 libraries, columns from Appel and Petzold aim to empower those people— so those underlying frameworks are still available,” Appel says. “Yes, from business devs working with C# to Web programmers working you can write WinRT apps in HTML5. And, yes, it’s real HTML5.” with JavaScript to native coders working with C++. Similarly, major Appel urges readers of her Modern Apps column to also check updates in developer tooling for Windows Azure have changed the out the GenerationApp site (bit.ly/W8GenAppDev), which features game in the cloud development space, and Terkaly and Villalobos articles and tutorials aimed at developers building Windows 8 and are here to help guide you through it. Windows Phone 8 apps. Th ere’s a whole lot going on with the emergence of Windows 8 ‘It’s New and Shiny’ and the Windows Runtime, and our new columns are designed As for Rachel Appel, she’s been writing for MSDN Magazine and its to address that activity. Is there something you want to see Appel Web site for more than a year now, and was a Microsoft technical cover in her Modern Apps evangelist for years before that. A veteran programmer, Appel has column? E-mail her at been in the soft ware development racket since the late 1980s, fi rst [email protected].
Visit us at msdn.microsoft.com/magazine. Questions, comments or suggestions for MSDN Magazine? Send them to the editor: [email protected].
© 2013 Microsoft Corporation. All rights reserved. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, you are not permitted to reproduce, store, or introduce into a retrieval system MSDN Magazine or any part of MSDN Magazine. If you have purchased or have otherwise properly acquired a copy of MSDN Magazine in paper format, you are permitted to physically transfer this paper copy in unmodifi ed form. Otherwise, you are not permitted to transmit copies of MSDN Magazine (or any part of MSDN Magazine) in any form or by any means without the express written permission of Microsoft Corporation. A listing of Microsoft Corporation trademarks can be found at microsoft.com/library/toolbar/3.0/trademarks/en-us.mspx. Other trademarks or trade names mentioned herein are the property of their respective owners. MSDN Magazine is published by 1105 Media, Inc. 1105 Media, Inc. is an independent company not affi liated with Microsoft Corporation. Microsoft Corporation is solely responsible for the editorial contents of this magazine. The recommendations and technical guidelines in MSDN Magazine are based on specifi c environments and confi gurations. These recommendations or guidelines may not apply to dissimilar confi gurations. Microsoft Corporation does not make any representation or warranty, express or implied, with respect to any code or other information herein and disclaims any liability whatsoever for any use of such code or other information. MSDN Magazine, MSDN, and Microsoft logos are used by 1105 Media, Inc. under license from owner.
4 msdn magazine Untitled-8 1 12/6/12 2:37 PM CUTTING EDGE DINO ESPOSITO
Essential Facebook Programming: The JavaScript SDK
I’ll be honest: I don’t know exactly what a social strategy might look like and I have no more than a faint idea about the tricks that social experts can play to make your content more popular. Likewise, I never paid much attention to search engine optimization (SEO) in Web sites, but I do believe that social optimization is today much more relevant and rewarding than SEO ever was. In the past two installments of this column, I discussed how to authenticate users of a Web or Windows application using Facebook and how to post to the social network on behalf of a consenting Figure 1 The Standard Interface of the Like Button user. (Th ose two columns can be found at msdn.microsoft.com/magazine/jj863128 and msdn.microsoft.com/magazine/jj883950.) Th ere are a few diff erent ways to embed a Like button. eTh simplest As far as Facebook is concerned, there’s a lot more you can do and most direct way is using an iframe element: to add a “social layer” to a Web site. A social layer results from the interaction. In those previous columns, I addressed identity and The href query string parameter refers to the URL you want a bit of interaction. Here, I’m going to explore the principal tools to like. The URL must be expressed in a fully qualified manner, you can leverage to add a true social layer onto your existing Web something like http://www.contoso.com/cool. site. According to Facebook, these tools are collectively known Most of the markup is aimed at styling the iframe. Usually, you as social plug-ins. A social plug-in is a powerful mix of HTML don’t want the iframe to scroll or be framed. You also want it to markup, CSS and JavaScript code. It’s your responsibility as a Web take up an appropriate area. Th e preceding markup produces the developer to fuse these elements into the UX. output in Figure 1. If the height of the iframe is too small (less than 25 pixels or so), you The Ubiquitous Like Button lose the panel containing the button to post an additional comment. The primary purpose of social plug-ins is to create a bridge If you’re using the Like button within a horizontal menu bar, then between the content of the Web site visited by a user and the user’s the height is a critical issue. Otherwise, giving users a chance to also Facebook page. By hosting one or more social plug-ins, the Web post their own comment augments the penetration of the feature. site enables users to share specific content with friends. The Th ere are several parameters you can leverage to customize the look, most immediate way for a Web site to let users share content via feel and functionality of the button. Figure 2 lists the options available. Facebook is the Like button. All will be assigned through an additional query string parameter. By simply clicking the Like button, a user can let friends know While Figure 1 shows the standard layout, Figure 3 shows the that she likes something at a given URL. Hosting the Like button button_count and box_count layouts. on a page couldn’t be simpler; fusing the button to the existing UI Another parameter you should look into is the ref parameter. It may require a few extra steps. helps you track use of the Like button in your Web site. By giving each Like button (even in diff erent pages of the site) a diff erent ref value—a plain alphanumeric string—you can easily track referrers Code download available at archive.msdn.microsoft.com/ mag201302CuttingEdge. in the server log that can be traced back to that specifi c Like button. In particular, for any click back from Facebook to your site, you’ll
6 msdn magazine Untitled-1 1 1/4/13 10:51 AM receive a referrer URL with two extra parameters. The fb_ref see the comments on a given post. Other query string parameter is just your original ref string; the fb _source plug-ins such as Like, the Like box and the query string parameter is a string that gives you information about Activity feed (the small list of notifi cations the context from within Facebook where the click originated. from all friends you usually have on the top- right corner of your page) also can be quickly Localizing the Like Button confi gured and embedded via an iframe. Even though the Like button can be considered universal, there still Plug-ins not implemented through iframes might be situations in which you want to translate the Like plug-in or direct URLs require the use of the Face- to a given language. As shown in Figure 2, all you need to do is book JavaScript SDK. As you can imagine, add the locale parameter to the query string and set it to a custom the SDK is a client front-end for a number Figure 3 Additional string that indicates the desired language and culture. of Facebook endpoints for you to perform Layout Formats To make things a bit trickier, you can’t express culture using the tasks such as authentication, posting and (Button Count and canonical xx-XX format where xx indicates the language and XX retrieving comments, likes, and other actions. Box Count) for the the culture. In the Microsoft .NET Framework, you get this string In order to use the JavaScript SDK, even Like Button from the following expression: before you download it, you must have an var name = Thread.CurrentThread.CurrentUICulture.Name; app ID. I thoroughly discussed this point in previous columns, For this string to be usable with Facebook, you need to replace but in a nutshell, for any interaction with the Facebook site that the dash with an underscore. Also note that the sole language token goes beyond basic operations such as Like, you need to register an isn’t suffi cient in itself and the whole locale setting will be ignored. app and get a unique ID. Th e app plays the role of the connector Th is point leads me to another interesting consideration: What’s the between your client (for example, a Web site) and the Facebook default behavior of the Like button as far as the language is concerned? back end. Th e Facebook App Dashboard for registering an app is Facebook, as well as Twitter, defaults to the language the user available at bit.ly/mly4xs. has chosen as the primary language in her profi le. If the user isn’t The second step consists of adding some code to your currently logged in, then the UI is based on the language settings Web pages that downloads the SDK. The URL to invoke is: detected on the browser. /con nect.facebook.net/xx_XX/all.js. Introducing the JavaScript SDK In general, you can confi gure Facebook’s plug-ins in a few diff erent Most sophisticated social plug-ins ways: using a plain URL from a custom hyperlink; using an iframe (as shown in this article); using HTML5 markup; and using the are only available through eXtended Facebook Markup Language (XFBML). HTML5 and XFBML are equivalent in many ways; HTML5 is just a more- HTML5 and XFBML markup. recent syntax supported for completeness. Both HTML5 and XFBML require the use of the Facebook JavaScript SDK. Th e xx_XX in the URL is a placeholder for the desired locale. An Most sophisticated social plug-ins are only available through option is linking this URL from within a static script tag. Because the HTML5 and XFBML markup. Th is is the case for the Send button size of the fi le is far more than 180KB, it might not be a good idea to for sending content directly to friends and the Comments box to download it synchronously through a static script tag. Th e file gets cached soon and most of the successive requests for it will receive an Figure 2 Customizing Look, Feel and Functionality HTTP 304 “not modifi ed” status code; however, Facebook recommends Parameter Description you download the fi le asynchronously. Th is is a pattern common to all action Indicates the text of the button and the subsequent action script blocks required for social interactions—for example, it works to be taken. It can be Like or Recommend. Default is Like. the same for Twitter plug-ins. Here’s the code you need: colorscheme Styles the button differently. It can be light or dark. format xx_XX. Note the use of the underscore to separate Th e code is defi ned as a JavaScript immediate function and runs as the two parts of a culture name. soon as it’s found. Note that you place this code preferably right aft er show_faces Boolean fl ag to indicate whether you want the picture of the opening body tag and you should fi ll the appId parameter with the user rendered once a user has liked the content. your app ID. (Note that many developers advocate placing scripts width Indicates the width of the plug-in. The minimum width at the bottom of the body section in order to prevent any blocking also depends on the layout used. of the rendering of the page.) 8 msdn magazine Cutting Edge Telerik DevCraft The all-in-one toolset for professional developers targeting Microsoft platforms.
&UHDWHZHEPRELOHDQGGHVNWRSDSSOLFDWLRQVWKDWLPSUHVV &RYHUDQ\1(7SODWIRUPDQGDQ\GHYLFH &RGHIDVWHUDQGVPDUWHUZLWKRXWFXWWLQJFRUQHUV
www.telerik.com/all-in-one
Untitled-2 1 1/8/13 10:24 AM At this point, you’re ready to use HTML5 and XFBML tags to log out from Facebook through your app. If data-show-faces is integrate Facebook plug-ins in your pages. Let’s start with a look false, then the login button stays visible all the time and it doesn’t at the Login button. react if clicked. As a Web developer, you should see the profound diff erence The Login Button between using the JavaScript SDK or the Login plug-in and server- Th e SDK contains a method on the root FB object through which side code and the Facebook C# SDK. If you work the client side, you can programmatically trigger the login process. Th e simplest then you’re essentially targeting Facebook, and login is prob- option consists of adding your own link or button and binding its ably the necessary first step in order to do more specific work. click handler to the following code: Facebook is the goal here. FB.login(function(response) { if (response.authResponse) { // Display some wait message // ... Provided that it’s not FB.api('/me', function(response) { $("#username").html('Welcome, ' + response.name + '.'); }); already so, authentication via } }); Th is code is far simpler than any other analogous code in the past social networks will become a two columns. In this way, authenticating users against Facebook is a breeze (see Figure 4). must-have feature for all sites Th e Login plug-in can make the login process even easier. In addition to linking the SDK, all you need is the following HTML5 markup (or needing authentication. analogous XFBML markup as demonstrated on the Facebook site):
tion cookie of ASP.NET and the standard membership system. The data-* attributes let you configure the appearance and Facebook is the means here. behavior of the button. The blue button in the page of Figure 4 gives an idea of the standard look and feel. In particular, the Coming Up data-show-faces attribute enables you to display the pictures of Provided that it’s not already so, authentication via social networks users (and some of their friends) that used your app to connect will become a must-have feature for all sites needing authentica- to Facebook. Th e data-max-rows attribute determines the num- tion. Th is means the C# SDK probably remains the most fl exible ber of rows (given the width of the plug-in) to be fi lled with faces. approach. In this regard, the new social classes in ASP.NET MVC It should also be noted that if data-show-faces is on and the user 4 are just icing on the cake. In terms of Web site development, I is already logged in, then no login button is shown. Th e user can’t likewise see no reason for not populating page layouts with social buttons and plug-ins to tweet or post immediate content. Next time, I’ll be back with Facebook program- ming covering more advanced social plug-ins such as Comments and the Like box. Meanwhile, for more information on Facebook social plug-ins, you can have a look at bit.ly/fAU7Xe. QDINO ESPOSITO is the author of “Architect- ing Mobile Solutions for the Enterprise” (Microsoft Press, 2012) and “Program- ming ASP.NET MVC 3” (Microsoft Press, 2011), and coauthor of “Microsoft .NET: Architecting Applications for the Enter- prise” (Microsoft Press, 2008). Based in Italy, Esposito is a frequent speaker at industry events worldwide. Follow him on Twitter at twitter.com/despos.
THANKS to the following technical experts for reviewing this article: Figure 4 Authenticating Users via JavaScript Christopher Bennage and Scott Densmore 10 msdn magazine Cutting Edge Untitled-4 1 11/30/12 11:37 AM WINDOWS WITH C++ KENNY KERR
Creating Desktop Apps with Visual C++ 2012
With all the hype over Windows 8 and what are now known as that you don’t want a console box popping up every time your app Windows Store apps, I’ve received some questions about the rele- starts. Th e wWinMain function is steeped in history. eTh __stdcall vance of desktop apps and whether Standard C++ is still a viable calling convention clarifi es matters on the confusing x86 archi- choice going forward. These questions are sometimes hard to tecture, which provides a handful of calling conventions. If you’re answer, but what I can tell you is that the Visual C++ 2012 compil- targeting x64 or ARM, then it doesn’t matter because the Visual er is more committed than ever to Standard C++ and it remains C++ compiler only implements a single calling convention on those the best toolchain, in my humble opinion, for building great architectures—but it doesn’t hurt, either. desktop apps for Windows whether you’re targeting Windows 7, Windows 8 or even Windows XP. A follow-up question I inevitably receive is how best to approach I’ll show you that creating a desktop app development on Windows and where to begin. Well, in this month’s column, I’m going to explore the fundamentals of desktop window without any creating desktop apps with Visual C++. When I was fi rst introduced to Windows programming by Jeff Prosise (bit.ly/WmoRuR), Microsoft library at all isn’t actually as Foundation Classes (MFC) was a promising new way to build apps. While MFC is still available, it really is showing its age, and a need daunting as it might seem at fi rst. for modern and fl exible alternatives has driven programmers to search for new approaches. Th is issue has been compounded by a Th e two HINSTANCE parameters are particularly shrouded in shift away from USER and GDI (msdn.com/library/ms724515) resources history. In the 16-bit days of Windows, the second HINSTANCE was and toward Direct3D as the primary foundation by which content the handle to any previous instance of the app. Th is allowed an app is rendered on the screen. to communicate with any previous instance of itself or even to switch For years I’ve been promoting the Active Template Library (ATL) back to the previous instance if the user had accidentally started it and its extension, the Windows Template Library (WTL), as great again. Today, this second parameter is always a nullptr. You may also choices for building apps. However, even these libraries are now have noticed that I named the fi rst parameter “module” rather than showing signs of aging. With the shift away from USER and GDI “instance.” Again, in 16-bit Windows, instances and modules were resources, there’s even less reason to use them. So where to begin? two separate things. All apps would share the module containing With the Windows API, of course. I’ll show you that creating a code segments but would be given unique instances containing the desktop window without any library at all isn’t actually as daunting data segments. Th e current and previous HINSTANCE parameters as it might seem at fi rst. I’ll then show you how you can give it a bit should now make more sense. 32-bit Windows introduced separate more of a C++ fl avor, if you so desire, with a little help from ATL address spaces and along with that the necessity for each process to and WTL. ATL and WTL make a lot more sense once you have a map its own instance/module, now one and the same. Today, this good idea of how it all works behind the templates and macros. is just the base address of the executable. Th e Visual C++ linker actually exposes this address through a pseudo variable, which you The Windows API can access by declaring it as follows: Th e trouble with using the Windows API to create a desktop win- extern "C" IMAGE_DOS_HEADER __ImageBase; dow is that there are myriad ways you could go about writing it—far The address of __ImageBase will be the same value as the too many choices, really. Still, there’s a straightforward way to create HINSTANCE parameter. Th is is in fact the way that the C Run- a window, and it starts with the master include fi le for Windows: Time Library (CRT) gets the address of the module to pass to your #include
12 msdn magazine Untitled-1 1 1/4/13 11:10 AM Th e next parameter provides any command-line arguments, and as I mentioned, together represent the key that the RegisterClass the last parameter is a value that should be passed to the Show- function creates to let CreateWindow fi nd the window class infor- Window function for the app’s main window, assuming you’re mation. Th e second parameter indicates the text that will be displayed initially calling ShowWindow. Th e irony is that it will almost always in the window’s title bar. Th e third indicates the window’s style. eTh be ignored. Th is goes back to the way in which an app is launched WS_OVERLAPPEDWINDOW constant is a commonly used style via CreateProcess and friends to allow a shortcut—or some other describing a regular top-level window with a title bar with buttons, app—to defi ne whether an app’s main window is initially minimized, resizable borders and so on. Combining this with the WS_VISIBLE maximized or shown normally. constant instructs CreateWindow to go ahead and show the window. Inside the wWinMain function, the app needs to register a If you omit WS_VISIBLE, then you’ll need to call the ShowWindow window class. Th e window class is described by a WNDCLASS function before your window will make its debut on the desktop. structure and registered with the RegisterClass function. Th is reg- Th e next four parameters indicate the window’s initial position istration is stored in a table using a pair made up of the module and size, and the CW_USEDEFAULT constant used in each case pointer and class name, allowing the CreateWindow function to just tells Windows to choose appropriate defaults. The next two look up the class information when it’s time to create the window: parameters provide the handle to the window’s parent window and WNDCLASS wc = {}; menu, respectively (and neither are needed). Th e fi nal parameter wc.hCursor = LoadCursor(nullptr, IDC_ARROW); wc.hInstance = module; provides the option of passing a pointer-sized value to the window wc.lpszClassName = L"window"; procedure during creation. If all goes well, a window appears on wc.lpfnWndProc = [] the desktop and a window handle is returned. If things go south, (HWND window, UINT message, WPARAM wparam, LPARAM lparam) -> LRESULT then nullptr is returned instead and the GetLastError function { ... may be called to fi nd out why. With all the talk about the hardships }; of using the Windows API, it turns out that creating a window is VERIFY(RegisterClass(&wc)); actually quite simple and boils down to this: To keep the examples brief, I’ll just use the common VERIFY WNDCLASS wc = { ... }; RegisterClass(&wc); macro as a placeholder to indicate where you’ll need to add some CreateWindow( ... ); error handling to manage any failures reported by the various API Once the window appears, it’s important that your app starts functions. Just consider these as placeholders for your preferred dispatching messages as soon as possible—otherwise your app will error-handling policy. appear unresponsive. Windows is fundamentally an event-driven, message-based OS. Th is is particularly true of the desktop. While Windows creates and manages the queue of messages, it’s the app’s Once the window appears, it’s responsibility to dequeue and dispatch them, because messages are sent to a window’s thread rather than directly to the window. Th is important that your app starts provides a great deal of fl exibility, but a simple message loop need not be complicated, as shown here: dispatching messages as soon as MSG message; BOOL result;
possible—otherwise your app while (result = GetMessage(&message, 0, 0, 0)) { if (-1 != result) will appear unresponsive. { DispatchMessage(&message); } Th e earlier code is the minimum that’s required to describe a stan- } dard window. Th e WNDCLASS structure is initialized with an empty Perhaps not surprisingly, this seemingly simple message loop is pair of curly brackets. Th is ensures that all the structure’s members oft en implemented incorrectly. Th is stems from the fact that the are initialized to zero or nullptr. Th e only members that must be GetMessage function is prototyped to return a BOOL value, but in set are hCursor to indicate which mouse pointer, or cursor, to use fact, this is really just an int. GetMessage dequeues, or retrieves, a when the mouse is over the window; hInstance and lpszClassName message from the calling thread’s message queue. Th is may be for to identify the window class within the process; and lpfnWndProc any window or no window at all, but in our case, the thread is only to point to the window procedure that will process messages sent pumping messages for a single window. If the WM_QUIT message to the window. In this case, I’m using a lambda expression to keep is dequeued, then GetMessage will return zero, indicating that the everything inline, so to speak. I’ll get back to the window procedure window has disappeared and is done processing messages and that in a moment. Th e next step is to create the window: the app should terminate. If something goes terribly wrong, then VERIFY(CreateWindow(wc.lpszClassName, L"Title", GetMessage might return -1 and you can again call GetLastError to WS_OVERLAPPEDWINDOW | WS_VISIBLE, CW_USEDEFAULT, CW_USEDEFAULT, CW_USEDEFAULT, CW_USEDEFAULT, get more information. Otherwise, any nonzero return value from nullptr, nullptr, module, nullptr)); GetMessage indicates that a message was dequeued and is ready Th e CreateWindow function expects quite a few parameters, but to be dispatched to the window. Naturally, this is the purpose of most of them are just defaults. Th e fi rst and second-to-last parameters, the DispatchMessage function. Of course, there are many variants
14 msdn magazine Windows with C++ Untitled-1 1 1/4/13 11:12 AM Figure 1 A Static Table of Message Handlers Without these calls, Windows wouldn’t consider the WM_PAINT
static message_handler s_handlers[] = message answered and your window would receive a steady stream { of WM_PAINT messages unnecessarily. { WM_PAINT, [] (HWND window, WPARAM, LPARAM) -> LRESULT The WM_DESTROY message arrives after the window has { disappeared, letting you know that the window is being destroyed. PAINTSTRUCT ps; VERIFY(BeginPaint(window, &ps)); Th is is usually an indicator that the app should terminate, but the GetMessage function inside the message loop is still waiting for // Dress up some pixels here! the WM_QUIT message. Queuing this message is the job of the EndPaint(window, &ps); return 0; PostQuitMessage function. Its single parameter accepts a value } that’s passed along via WM_QUIT’s WPARAM, as a way to return }, { diff erent exit codes when terminating the app. WM_DESTROY, [] (HWND, WPARAM, LPARAM) -> LRESULT Th e fi nal piece of the puzzle is to implement the actual window { PostQuitMessage(0); procedure. I omitted the body of the lambda that I used to prepare return 0; the WNDCLASS structure previously, but given what you now } } know, it shouldn’t be hard to fi gure out what it might look like: }; wc.lpfnWndProc = [] (HWND window, UINT message, WPARAM wparam, LPARAM lparam) -> LRESULT { to the message loop, and having the ability to construct your own for (auto h = s_handlers; h != s_handlers + aff ords you many choices for how your app will behave, what input _countof(s_handlers); ++h) { it will accept and how it will be translated. Apart from the MSG if (message == h->message) pointer, the remaining parameters to GetMessage can be used to { return h->handler(window, wparam, lparam); optionally fi lter messages. } Th e window procedure will start receiving messages before the }
CreateWindow function even returns, so it had better be ready and return DefWindowProc(window, message, wparam, lparam); waiting. But what does that look like? A window requires a message }; map or table. Th is could literally be a chain of if-else statements or a Th e for loop looks for a matching handler. Fortunately, Windows big switch statement inside the window procedure. Th is does, how- provides default handling for messages that you choose not to ever, quickly become unwieldy, and much eff ort has been spent in process yourself. Th is is the job of the DefWindowProc function. diff erent libraries and frameworks to try to manage this somehow. And that’s it—if you’ve gotten this far, you’ve successfully created In reality, it doesn’t have to be anything fancy, and a simple static a desktop window using the Windows API! table will suffi ce in many cases. First, it helps to know what a window message consists of. Most importantly, there’s a constant—such as The ATL Way WM_PAINT or WM_SIZE—that uniquely identifi es the message. Th e trouble with these Windows API functions is that they were designed Two arguments, so to speak, are provided for every message, and long before C++ became the smash hit that it is today, and thus weren’t these are called WPARAM and LPARAM, respectively. Depending designed to easily accommodate an object-oriented view of the world. on the message, these might not provide any information. Finally, Still, with enough clever coding, this C-style API can be transformed Windows expects the handling of certain messages to return a value, and this is called the LRESULT. Most messages that your app handles, Figure 2 Expressing a Window in ATL however, won’t return a value and should instead return zero. class Window : public CWindowImpl
of Windows thanks to advances in rendering and composition of LRESULT DestroyHandler(UINT, WPARAM, LPARAM, BOOL &) the desktop. Th e BeginPaint and EndPaint functions are relics of { PostQuitMessage(0); the GDI but are still needed even if you’re drawing with an entirely return 0; diff erent rendering engine. Th is is because they tell Windows that } }; you’re done painting by validating the window’s drawing surface.
16 msdn magazine Windows with C++ Untitled-1 1 11/26/12 3:03 PM into something a little more suited to the average C++ programmer. new app using a modern rendering engine, it doesn’t provide a great ATL provides a library of class templates and macros that do deal of value. Still, there are a handful of useful helpers. From the just that, so if you need to manage more than a handful of WTL atlapp.h header, you can use its message loop implementation window classes or still rely on USER and GDI resources for your to replace the hand-rolled version I described earlier: window’s implementation, there’s really no reason not to use ATL. Th e CMessageLoop loop; window from the previous section can be expressed with ATL as loop.Run(); Although it’s simple to drop into your app and use, WTL packs shown in Figure 2. a lot of power if you have sophisticated message fi ltering and The CWindowImpl class provides the necessary routing of routing needs. WTL also provides atlcrack.h with macros designed to messages. CWindow is a base class that provides a great many replace the generic MESSAGE_HANDLER macro provided by member function wrappers, mainly so you don’t need to provide ATL. Th ese are merely conveniences, but they do make it easier to the window handle explicitly on every function call. You can see get up and running with a new message because they take care of this in action with the BeginPaint and EndPaint function calls in cracking open the message, so to speak, and avoid any guesswork this example. Th e CWinTraits template provides the window style in fi guring out how to interpret WPARAM and LPARAM. A good constants that will be used during creation. example is WM_SIZE, which packs the window’s new client area as the low- and high-order words of its LPARAM. With ATL, this Although it’s simple to drop might look as follows: BEGIN_MSG_MAP(Window) ... into your app and use, WTL MESSAGE_HANDLER(WM_SIZE, SizeHandler) END_MSG_MAP() packs a lot of power if you have LRESULT SizeHandler(UINT, WPARAM, LPARAM lparam, BOOL &) { auto width = LOWORD(lparam); sophisticated message fi ltering auto height = HIWORD(lparam); and routing needs. // Handle the new size here ... return 0; } Th e macros harken back to MFC and work with CWindowImpl With the help of WTL, this is a little simpler: to match incoming messages to the appropriate member functions BEGIN_MSG_MAP(Window) for handling. Each handler is provided with the message constant as ... MSG_WM_SIZE(SizeHandler) its fi rst argument. Th is can be useful if you need to handle a variety END_MSG_MAP() of messages with a single member function. Th e fi nal parameter void SizeHandler(UINT, SIZE size) defaults to TRUE and lets the handler decide at run time whether { it actually wants to process the message or let Windows—or even auto width = size.cx; auto height = size.cy; some other handler—take care of it. These macros, along with CWindowImpl, are quite powerful and let you handle refl ected // Handle the new size here ... } messages, chain message maps together and so on. Notice the new MSG_WM_SIZE macro that replaced the To create the window, you must use the Create member function generic MESSAGE_HANDLER macro in the original message map. that your window inherits from CWindowImpl, and this in turn Th e member function handling the message is also simpler. As you will call the good old RegisterClass and CreateWindow functions can see, there aren’t any unnecessary parameters or a return value. on your behalf: Th e fi rst parameter is just the WPARAM, which you can inspect Window window; VERIFY(window.Create(nullptr, 0, L"Title")); if you need to know what caused the change in size. At this point, the thread again needs to quickly begin dispatching Th e beauty of ATL and WTL is that they’re just provided as a set messages, and the Windows API message loop from the previous of header fi les that you can include at your discretion. You use what section will suffi ce. Th e ATL approach certainly comes in handy if you need and ignore the rest. However, as I’ve shown you here, you you need to manage multiple windows on a single thread, but with can get quite far without relying on any of these libraries and just a single top-level window, it’s much the same as the Windows API write your app using the Windows API. Join me next time, when approach from the previous section. I’ll show you a modern approach for actually rendering the pixels in your app’s window. Q WTL: An Extra Dose of ATL While ATL was designed primarily to simplify the development of KENNY KERR is a computer programmer based in Canada, an author for Plural- COM servers and only provides a simple—yet extremely eff ective— sight and a Microsoft MVP. He blogs at kennykerr.ca and you can follow him on window-handling model, WTL consists of a slew of additional class Twitter at twitter.com/kennykerr. templates and macros specifi cally designed to support the creation of more-complex windows based on USER and GDI resources. THANKS to the following technical expert for reviewing this article: WTL is now available on SourceForge (wtl.sourceforge.net), but for a Worachai Chaoweeraprasit 18 msdn magazine Windows with C++ Untitled-1 1 1/4/13 11:27 AM MICROSOFT OFFICE
Exploring the New JavaScript API for Offi ce
Stephen Oliver and Eric Schmidt
This article is the fi rst in a series of in-depth looks at the code samples and community resources at the Apps for Offi ce and JavaScript API for Offi ce, newly introduced in Microsoft Offi ce SharePoint Developer Center on MSDN (dev.offi ce.com). 2013. It presupposes that you’re familiar with apps for Office. If not, the MSDN documentation page, “Overview of apps for Overview of the JavaScript API for Offi ce Offi ce” (bit.ly/12nBWHG), provides a broad overview of and general Th e JavaScript API for Offi ce comprises a complete object model. introduction to the API. Th e API is contained within a set of JavaScript fi les, starting with Th is article and the others in this series, while not exhaustive, the offi ce.js fi le. An app must include a reference to the offi ce.js fi le go deep into the API, touching on key aspects that will give you a to use the JavaScript API for Offi ce. On load, the ce.jsoffi fi le loads solid, richer understanding of how the apps for Offi ce API works. the other required scripts that it needs to operate, including the In this fi rst article, we review the apps for Offi ce object model. scripts needed for the host environment and the locale strings. For- Part 2 will focus on the core task of how to access Offi ce fi le content tunately, you can add a reference to the offi ce.js fi le using a content and will review the event model. Part 3 will consider the concept delivery network (CDN), so you don’t need to deploy a copy of the of data binding and examine the basics of working with custom offi ce.js fi le along with your app. Here’s an example: XML parts. Finally, Part 4 will close the series with a focused look at mail apps. Th e object model was designed around several goals: 1. “Write once, run everywhere.” It had to be extensible— This article discusses: not tied to a specifi c host application, but built around capabilities available in multiple host applications. Apps • Overview of the JavaScript API for Offi ce access host-specifi c functionality in a consistent way. • The asynchronous programming pattern 2. Cross-platform. Compatibility ranked high on this list, • Object model hierarchy too; thus, the object model isn’t tied to a specifi c version of • Testing for host application support Offi ce. As well, the same code works on the Web App ver- Technologies discussed: sions of the Offi ce client applications, where supported. For example, an app for Excel can work on the Excel Web App JavaScript API for Offi ce just as well as in the Excel client application. 20 msdn magazine Untitled-1 1 1/4/13 11:37 AM Th e signature for all asynchro- nous functions adheres to the following basic pattern: functionNameAsync( requiredParameters, [, options], [callback]); Th e required parameters are fol- lowed by two other parameters: an object that holds optional param- eters and a callback function, both of which are always optional. Optional Parameters in Asyn- chronous Functions Th e optional JavaScript object in the signature of asynchronous functions is a collection of key/value pairs, sep- arated by a colon, where the key is the name of the parameter and the value is the data that you want to Figure 1 The Object Model Hierarchy in the JavaScript API for Offi ce use for that parameter. Th e order of the key/value pairs doesn’t matter 3. Performance and security. It needed to be maximized as long as the parameter name is correct. Th e MSDN documenta- for performance, so that apps can be as unobtrusive to tion for each asynchronous function details what parameters are users as possible. Also, the JavaScript API was designed to available to use in the options object for that particular function. interact directly with document content without having to For example, the Document.setSelectedDataAsync method has automate the Offi ce applications, improving the stability the same basic signature common to all asynchronous functions and security of the solutions. in the apps for Offi ce: Another key goal for the JavaScript API was to attract Web devel- Office.context.document.setSelectedDataAsync( opers to the Offi ce platform. us,Th the object model was built with data [, options], callback); modern Web programming in mind. You can leverage your current skills and knowledge of other JavaScript libraries, such as jQuery, when creating apps in conjunction with the JavaScript API for Offi ce. One of the ways that the
The Asynchronous Programming Pattern designers enhanced the As mentioned, performance was a key goal in the design of the apps for Offi ce API. One of the ways that the designers enhanced performance of the API was the performance of the API was through the heavy use of asyn- chronous functions. through the heavy use of Th e use of asynchronous functions avoids blocking an app during execution in the event a function takes a while to return. Th e asyn- asynchronous functions. chronous function is invoked, but program execution doesn’t wait for the function to return. Instead, it continues while the asynchronous Like all asynchronous functions in the API, Document.set- function is still executing. Th is allows the user to continue to use the SelectedDataAsync has an options object that holds optional Offi ce document while the app is potentially still working. parameters, but the parameters for its options object are diff erent Some key points for understanding the asynchronous design in from those for other asynchronous functions in the API, because the apps for Offi ce API covered in this section are: the point of this function is to set data. So the optional parameters • Th e common signature of asynchronous functions in the for Document.setSelectedDataAsync are related to setting data: apps for Offi ce API • coercionType: A CoercionType enumeration that specifi es • Th e use of optional parameters in asynchronous functions the format for the data you insert (text, HTML, OOXML, • Th e role of the AsyncResult object in asynchronous functions table or matrix) We’ll discuss each in turn. • asyncContext: A user-defi ned object that’s returned Common Signature of Asynchronous Functions in the Apps unchanged in the AsyncResult object that’s passed in to for Offi ce API All asynchronous functions in the apps for Offi ce the callback function as its only parameter API have the same naming convention and the same basic sig- Th at same concept applies to all the other asynchronous functions. nature. Every asynchronous function name ends in “Async,” for You can either supply the object that contains the optional param- example, like this: Document.getSelectedDataAsync. eters as an object literal inline in the asynchronous function call, or
22 msdn magazine Microsoft Offi ce You used to think "Impossible" Your Apps, Any Device
Now you think—game on!! The new tools in 12.2 help you envision and create engaging applications for the Web that can be accessed by mobile users on the go. And, with our Windows 8 XAML and JS tools you will begin to create highly interactive applications that address your customer needs today and build next generation touch enabled solutions for tomorrow.
The next generation of inspiring tools. Today.
Download your 30-day trial at www.DevExpress.com Copyright 1998-2013 Developer Express Inc. All rights reserved. All trademarks are property of their respective owners.
Untitled-13 1 1/7/13 12:52 PM Figure 2 Availability of Capabilities in the JavaScript API for Offi ce by Host Application anonymous function callback to get a refer- ence to the File object. Th en it displays the Excel/Excel Outlook/Outlook Capability Word Web App PowerPoint Web App Project size of the fi le in the specifi ed element using the size property of the File object. In a similar Get/set data as All All Text only Text only text, table, matrix way, you’ll use the AsyncResult.value property Settings All All All (RoamingSettings) to get the return value of any asynchronous function in the apps for Offi ce API. Get fi le All Compressed only You can read more about the Docu- Bindings All All ment.getFileAsync method in the next Custom XML Parts All article of this series. HTML and OOXML All Mailbox All Object Model Hierarchy The JavaScript API for Office aims to create an object fi rst and then pass that object in for the parameter. provide compatibility across versions of Office and symmetry Following are two code samples that show both ways of supplying the across different host applications. To support these goals, the options object using the Document.setSelectedDataAsync function. JavaScript API has a lean object model with a distinct hierarchy Passing the options parameter inline: that isn’t directly tied to any specifi c host application. Instead, the function setData(data) { object model hosts a targeted set of capabilities for interacting with Office.context.document.setSelectedDataAsync(data, { coercionType: Office.CoercionType.Text } Offi ce documents, scoped to the type of app (task pane, content ); or mail app) using them. } Figure 1 provides an abbreviated overview of the top-level Passing the options parameter in a JavaScript object: function setData(data) { hierarchy of objects in the JavaScript API for Offi ce (note that the var options = { coercionType: Office.CoercionType.Text }; entire object model isn’t shown). Specifi cally, the diagram demon- Office.context.document.setSelectedDataAsync(data, options ); } strates the relationships between the Offi ce, Context, Document, The Role of the AsyncResult Object in Asynchronous Functions Settings, Mailbox and RoamingSettings objects. Th e third parameter in the common signature for asynchronous Each host application (Word, Excel, Excel Web App, PowerPoint, functions in the JavaScript API for Offi ce is the optional callback Project, Outlook and Outlook Web App) can use a subset of the parameter. Th e callback parameter is exactly as it sounds: a func- capabilities included in the API. For example, roughly 40 percent tion you provide that’s invoked when the asynchronous operation of the object model pertains solely to mail apps that can only be completes. Of course, you can provide either a named function used in Outlook and the Outlook Web App. Another portion of the or an anonymous function inline in the call to the asynchronous object model allows interaction with Custom XML Parts, which is function. Th e key thing to note here is the role of the AsyncResult only available in Word 2013. object with respect to the callback function. Figure 2 shows the capabilities available to specifi c host applications. When the runtime invokes your callback, it passes in an Async- Shared Objects in the Object Model The JavaScript API Result object as the only argument for the callback. Th e AsyncResult for Offi ce has a defi nitive entry point, the Offi ce object, which is object contains information about the asynchronous operation, available to all types of apps and in all of the host applications. Th e such as: whether or not the operation succeeded; what errors, if any, Figure 3 Storing a Reference to the occurred; and the return value, if any, of the asynchronous func- Document Object When the App Initializes tion. In fact, in all asynchronous functions that return some kind // Add a handler to the initialize event of the Office object of data or object, the AsyncResult is the only way you can get at the Office.initialize = function (reason) { returned value. You do this using the AsyncResult.value property. $(document).ready(function () { app.get_Document(Office.context.document); For example, the following code snippet gets the size of the // Other initialization logic goes here document and displays it in the specifi ed HTML element on the }) app UI. In order to get the file size, you first get the file object }
that the Document.getFileAsync method returns through the // Use a self-executing anonymous function to encapsulate the AsyncResult.value property. Here’s how to do this: // functionality that the app uses var app = (function () { function getFileData(elementId) { Office.context.document.getFileAsync(Office.FileType.Text, var _document; function (asyncResult) { function get_Document(officeDocument) { if (asyncResult.status === 'succeeded') { _document = officeDocument; var myFile = asyncResult.value; } $(elementId).val(myFile.size); } // Other fields and functions associated with the app }); } return { get_Document: get_Document Th e getFileData function calls the Document.getFileAsync method, // Other exposed members specifying that it should return the file content as text. It then }; })() uses the value property of the AsyncResult object passed in to the 24 msdn magazine Microsoft Offi ce Offi ce object represents a specifi c instance of an app inserted into Also exposed by the Document object is the Settings object, which a document, workbook, presentation, project, e-mail message or represents the “property bag” for an app. An app can store and per- appointment. It can access bindings between the app and the doc- sist custom properties across app sessions in the same document ument using the select method. (We’ll discuss bindings in greater using the Settings object. Th e properties travel with the document: depth in a future article.) Most importantly, the Offi ce object exposes If you share an Offi ce fi le that contains an app with someone else, the initialize event for the app, which allows you to build initializa- the custom properties stored in the app will be available when the tion logic for the app (more on that in a future article). Finally, the other person reads the fi le. Offi ce object contains a reference to the Context object for the app. Storing and retrieving settings using the property bag is simple. Th e Context object, which is also available to all types of apps and Th e Settings.set method creates the setting in memory as a key/ in all of the host applications, exposes infor- mation about the runtime environment that’s hosting the app. In addition to storing the lan- guage settings for the app, the Context object provides the entry point to runtime capabilities in the JavaScript API for Offi ce that are specifi c to the host in which the app was activated. For example, you can access the document (Document object) associated with the app through the Context.document property. However, this property returns a value only when called from within a host application that supports it, that is, from within a task pane or content app. If we attempt to access the Context.document property from a mail app, we’ll get an “undefi ned object” error. Like- wise with the Context.mailbox property: In a mail app, it returns the mailbox (Mailbox object) opened in the host application. In a task pane app, it’s undefi ned. Support for Task Pane and Content Apps in the Object Model For task pane and content apps, the Document object represents the document, workbook, presen- tation or project into which the app has been inserted. Th e Document object provides the highest degree of access to the fi le’s content— in essence, it’s the primary point of contact between an app and an Offi ce document. Almost all of the techniques for accessing the content in the Offi ce document require use of the Document object. For this reason, you might want to capture a reference to the Document object when the app initializes as shown in Figure 3. When an app is activated within a Project file, the Document object exposes addi- tional, specific capabilities targeted for Project fi les. Th rough the Document object, an app can get data for specifi c tasks, views, fi elds and resources in the project. An app can also add event listeners to monitor when the user changes the selected view, task or resource selected in the project. (We’ll talk more about using the Document object in an app for Project in the next article.) msdnmagazine.com February 2013 25 value pair. To get the properties out of the property bag, we use Testing Whether a Capability the Settings.get method, passing in the setting’s name (key), to get Can Be Used in a Host Application the value. Both the set and get methods are synchronous. To store As we alluded to earlier, one of the strengths of the JavaScript API the settings across sessions, we need to call the Settings.saveAsync for Offi ce is the “develop once, host many places” nature of apps method, which saves all of the custom properties contained in the for Offi ce. For example, the same task pane app can be activated app when the document is saved. within Word, Excel, Project and PowerPoint (provided that its Th e code sample, “Apps for Offi ce: Persist custom settings” (bit.ly/ manifest allows all of those capabilities). UEiZff), provides additional examples of how to use the Settings object Yet, because not all apps have access to the exact same list of and how to store data in an app. capabilities, an app could be inserted into a host application that Support for Mail Apps in the Object Model For mail apps, doesn’t allow the capabilities that the app requires. For example, the Mailbox object provides the entry point of data access for Project currently doesn’t provide access to the Settings object. mail-app-specifi c functionality. As the name implies, the Mailbox An app that tries to access the Settings object when inserted into object corresponds to the mailbox of the current user and travels Project will raise an “undefi ned object” error. to wherever users read their e-mail—either in the Outlook client Thus, developers must include logic in their apps for testing application or in the Outlook Web App. In addition to providing the availability of the capabilities they need. In the example with access to individual e-mail messages and appointments (through Project, the best technique for detecting capabilities in a host the Mailbox.item property), the Mailbox object allows the app to application is through a simple if block: create new appointments, access the profi le of the local user and // Test for Settings object in host application even get the user’s local time. if (Office.context.document.settings) { Like the Document object for content and task pane apps, you // Provide implementation that uses the Settings object might want to capture a reference to the Mailbox object when the } app initializes, as shown in Figure 4. else { The RoamingSettings object, which is also available only in // Use some other technique for saving custom properties, mail apps, is similar to the Settings object for document-centric // like localStorage, sessionStorage or cookies apps (task pane and content apps). It allows apps to persist custom } properties as name/value pairs across sessions. However, unlike the For more information about how to detect whether a member Settings object, which saves the custom properties within the host is available in the host application, see the MSDN documentation Offi ce fi le, the RoamingSettings object saves the custom settings page, “How to: Determine host application support for specifi c API to the current user’s mailbox. Th is makes the custom properties members,” at bit.ly/TR5ZlB. available to the app no matter what message the user is looking To summarize this article, we’ve discussed the meat and at or how the user has accessed his mailbox (in Outlook or the potatoes of the JavaScript API for Offi ce. We described the object Outlook Web App). model hierarchy at a high level and discussed the asynchronous For more information about the object model hierarchy in the pattern as it’s implemented in the object model. We also described JavaScript API for Office, see the MSDN documentation page, how to test whether a capability is supported in a host application. “Understanding the JavaScript API for Offi ce” (bit.ly/UV2POY). In the next article in this series, we’ll take a closer look at the Figure 4 Storing a Reference to the Mailbox Object simplest, yet most powerful ways for working with data in an in a Global Variable When the App Initializes app for Offi ce. We’ll describe how to get and set selected data in more depth. We’ll look at getting all of the fi le content and how to // Add a handler to the initialize event of the Office object Office.initialize = function (reason) { parse it. Also, we’ll discuss apps in Project and how to read task, $(document).ready(function () { resource and view data. Finally, we’ll review the event model in the app.get_Mailbox(Office.context.mailbox); JavaScript API for Offi ce: what events you can code against and // Other initialization logic goes here how to handle the results. Q }) } TEPHEN LIVER is a programming writer in the Offi ce Division and a Microsoft // Use a self-executing anonymous function to encapsulate the S O // functionality that the app uses Certified Professional Developer (SharePoint 2010). He writes the developer var app = (function () { documentation for the Excel Services and Word Automation Services, along with PowerPoint Automation Services developer documentation. He helped curate and var _mailbox; design the Excel Mashup site at ExcelMashup.com. function get_Mailbox(mailbox) { _mailbox = mailbox; } ERIC SCHMIDT is a programming writer in the Offi ce Division. He has created // Other fields and functions associated with the app several code samples for apps for Offi ce, including the popular “Persist custom settings” code sample. In addition, he has written articles and created videos about return { other products and technologies within Offi ce programmability. get_Mailbox: get_Mailbox // Other exposed members }; HANKS to the following technical experts for reviewing this article: })() T Mark Brewster, Shilpa Kothari and Juan Balmori Labra
26 msdn magazine Microsoft Offi ce emement1);ent1); a areaSeriesreaSerieies.A Add(seriesElement2);Add(se(s rriesElement2t2);) areaSeriesareaSeries.Ad Add(seriesElement3);A d(seriesElement3); //// AddAdd seriesseries to the plot areaarea plotAreaplotArea.S Serieseries.Ad Add(areaSeries);d(areaSeries); //page//page.El Elementsemenem ts.A Add(dd(d newne LayoutGrid()LaLayyoutGrid() ); /// AddAdd thethe pagepage elementselements to the page AddEAAddEA s, 240, 0); AddEAN1AN 3SupSup5(pa5(p ge.Elemeentnts, 480, 0); AdddUPCVersionA(page.Elemene ts, 0, 135); AddUPCVersionASup2(page.Elements, 240, 135); AdddUPCd CVerssionAo Sup5((page.Elemennts,t 480, 135); AddEAN8(page.Elements, 0, .Elements, 480, 270);; AddddUUPCVersionE(papage.Elementts, 0, 405); AddUPCVersionESuE p2(page.Elements, 240, 405); AddUPCVersionESup5(pageage.Elelemments, 48800, 4405); // AAdd the page tot the document document.Pages.Add(pa CaptionAndRectanga lee(elemeements, “EAN/JA/JAN 13 Bar Codde”, x, y, 204, 99); BarCode barCode = new Ean13(“123456789012”, x, y + 21); barCode.XX+ +=+ ((2004 - baarCoode.GettSymbolWidth()h ) / 2; elements.Add(barCode); } private vo dRectangle(elemente s,, “EANEAN/JAN 13 Bar Code, 2 digit supplement”, x, y, 204, 99); BarCode barCode = new Ean13Sup2(“12 234556789678 0121212”, x, y + 211); barCoode.X += (204 - barCode.GetSymbolWidth()) / 2; elements.Add((barC ts, float x, float y)y { AddCaCaptionAndRectanangle(elements, “EAN/JAN 13 Bar Code, 5 digit supplement”, x, y, 204, 99); ( g[ ] g) BaB rrCode barrCode = new Ean13SupS 5(“12345678901212345”, x, y + 21); ba s.Add(barCode); } pprivatete vooid AddUPCVPCVersiers onA(Group elements, float x, float y) {Add{ CaptionAndRectangle(elements,{ “UPC Versionio A Baar CCoode”, x, y, 204, 99);; BarCoode barCode = new UpcVersionA(“12345678901”, xx, y + s.Add(barCode); } pprivatete vooid AddUPCVPCVersiers onASup2(Group elements, float x, floato y) { AddCaptionAndRectangle(elementm p ““UPCC Version E Bar Code, 2 digit supplement”, x, y, 204,4 99); BarCoodeod 21) ; barCode.X += (2( 04 - babarCode.GeetSymbymbolWidth()) / 2; elements.Add(barCode);e } private void AddUPCVVersionASSuup5( g g ( p p ) Group elements,n float x, flfloaflo t digit supplement”,t” xx, y, 2004, 99); BarCode bbarCode = new UpcVep rsionASuAS p5(“12343 567890112345”, x, y + 21); barCarCode.Xde.X [ ] += (204 - barCode.GetSymbom olWidth()) / 22; ele t x, float y) { AddCaptionnAndRAnd ectangle(eleelemments, “EAN/JANN 8 Bar Code”, x,x y, 204,0 99); BarCode barCode = new Ean8(n8 “1 [ ] 2345670”, x, y + 21); OpenFileileDiala og fileDie aloloog = g.Filter = “Adobeob PDPDF filees (*.pdf)|*.pdf|Alll Files (*.*)|*.*”; if (fileDieD alog.SShowDiialog() == DialogResult.OK) { pdfVf ieweewer.OpOpen p y y yyp (fifileDialog.FilleName, “”); } SaveveFileDialog saveFileileDDialoog = neneww Sav Dialog.Filter = “AdoAdobe PDF files (*.pdff)|*.pdpdf|A|All Files (*.*)|*.*”;”; iff (saveFFileDialolog.Showh Dialog() ==DialoalogResRee ult..OK) { pdfVView High h SecuSec rity sec urit y new w High h Secu u rity y (“O Ow nerP assw ord” , “U“ serPassw ord” ) .SSaveAsve (saveFieFileDia printer = pdfViewer.Per rinterr; prip nter.PriintWiW thDialog(); } elsee { MesssageBox.SShoww(“Please open a fifile to printrin ”)”); }OpenFin le y ppy DDialoog fileDieD alog = new OpenFileDDialog(); fileDiaDialog.og Tittle = “Opepen Filel D les (*.*)|*.*|Addobee PDF fileses (*.p*.pdf)|*.pddf”; ifi (fifileDialog.ShowwDialalog() === DialogResult.ult OK) { DynamicPDFViewewerClass testees = y y neew DynammiccPDFViewerClass((); PPDFPrinter pprintererr = tet st.OpenFFileFo = File.ReadAAllBytes(@”C:\M:\MyDyDoc.pdff”); //u// sing System.Runntimme.IntntteropServicces;GCHandled gch=GCH= CHaananddle.Aloc(conteents, p GCHandleTTypee.Pinnedd);) IIntntPtr coconton entsIntPtr = g tents.Lengthh, “”); AddCaCaptioonAndReectanglan ee(pageElementts, “BBBookkmark Pagee Elemeent:”, x, y);; pagpaa eEeEleemeents.Add(newtesw t ( p p ) Bookmaark(“Bookmarked Textt”, x + 5, y ++2 20, pparentre OutlO ine));) pa 215, 10, Fonnt.TTimemesRomanma , 10)); } prriivate vooid AddCiircle(GGroup paageElemeennts,nts float x,x floafloatyt y) { //Add/A s a circlt to the papageg p ( p p ) Eleemenmentenents AddCapCa tiont AndRdRectae Add(new Cirrcle(x + 112.5f5f, y + 50f, 1007.5f, 300f, RgbColoolor.RRRed, RgbbColor.Bluee, 2, LineL eStyylyle.Dae DashLas L rge))); } pprivate void AdA d p () FormatteeddTexxtAreea(Groupp paageElements, flfloaat x, flfloat y) { // Adds a formatted textte a micPDFPDF/b>&tm; GGenerator v66.0 for ..NETNET has a forrmattted tetext t arrea paage “ + “eleemment. This prroviddes rich fororm } atting suppupport for texext that appap eears in the doocumment. You have “ + “completlete cocontrolol ovever 8 paragraph left indentattionn, righht indenentation, aligliggnnment, allollowing orphaan linnes, aand white “ + “spaace prereservata ion; 6 foont propropeerties:e Thiss is a very simplee HTMLHT strs ingin inclinc udinud ng a T ;peXd`ZG;=Ç:fdgi\_\ej`m\G;=Jfclk`fej]fi%E KIPFLIG;=JFCLK@FEJ=I<<KF;8PK nnn%;peXd`ZG;=%Zfd&\mXcn fiZXcc/''%-*(%,''-s"(+('%..)%/-)' Untitled-1 1 9/8/11 11:56 AM YOURR BAACKKSTAGGE PAASS TTO THHE MICROOSOFFT PLLATFFORM Intense Take-Home Training for Developers, Software Architects LETand Designers US HEAR YOU CODE SUPPORTED BY PRODUCED BY magazine Untitled-9 2 1/2/13 1:46 PM MARCH las vegas 25-29, 2013 MGM Grand Hotel & Casino EARLY BIRD SPECIAL Register before February 27 and save $300! Use Promo Code LVFEB4 Everyone knows all the *really* cool stuff happens behind the scenes. Get an all-access look at the Microsoft Platform and practical, unbiased, developer training at Visual Studio Live! Las Vegas. Topics will include: ³ ASP.NET ³ Azure / Cloud Computing ³ Cross-Platform Mobile ³ Data Management ³ HTML5 / JavaScript ³ Developer Deep Dive: SharePoint / OfÀ ce 365 ³ Developer Deep Dive: SQL Server ³ Windows 8 / WinRT ³ WPF / Silverlight ³ Visual Studio 2012 / .NET 4.5 ¬TURN THE PAGE FOR MORE EVENT DETAILS vslive.com/lasvegas Untitled-9 3 1/2/13 1:46 PM YOURR BAACKKSTAGGE PAASS TTO THHE MICROOSOFFT PLLATFFORM CODE WITH .NET ROCKSTARS AND AGENDA AT-A-GLANCE LEARN HOW TO MAXIMIZE THE Windows 8/ WinRT WPF/ Silverlight ASP.NET Visual Studio 2012/ .NET 4.5 SharePoint / OfÀ ce A DEVELOPMENT CAPABILITIES OF START TIME END TIME Visual Studio Live! Pre-Conference Workshops: Monday, March 25, 2 7:30 AM 9:00 AM Pre-Conference Workshop Registration - Coffee and Morning Pastries VISUAL STUDIO AND .NET DURING 9:00 AM 6:00 PM MW01 - Workshop: Build a Windows 8 MW02 - Workshop: Services - Using WCF 5 ACTION-PACKED DAYS OF Application in a Day - Rockford Lhotka and ASP.NET Web API - Miguel Castro PRE- AND POST-CONFERENCE START TIME END TIME Visual Studio Live! Day 1: Tuesday, March 26, 2013 7:00 AM 8:00 AM Registration - Coffee and Morning Pastries WORKSHOPS, 70+ SESSIONS LED BY 8:00 AM 9:00 AM Keynote: To Be Announced 9:15 AM 10:30 AM T01 - A Primer in Windows 8 Development T02 - jQuery Fundamentals - with WinJS - Robert Boedigheimer EXPERT INSTRUCTORS AND KEYNOTES Philip Japikse 10:45 AM 12:00 PM T06 - Windows 8 Style Apps - Design T07 - Hate JavaScript? Try TypeScript. - BY INDUSTRY HEAVYWEIGHTS. Essentials - Billy Hollis Ben Hoelting 12:00 PM 2:30 PM Lunch & Expo Hall 1:15 PM 2:15 PM T11 - Chalk Talk: MVVM in Practice aka T12 - Chalk Talk: Neural Networks for ""Code behind""-free XAML - Developers - James McCaffrey Tiberiu Covaci 2:30 PM 3:45 PM T15 - New XAML controls in Windows 8 - T16 - Tips for building Multi-Touch Billy Hollis Enabled Web Sites - Ben Hoelting 3:45 PM 4:15 PM Networking Break 4:15 PM 5:30 PM T20 - Make your App Alive with Tiles and T21 - Build Speedy Azure Applications NotiÀ cations - Ben Dewey with HTML 5 and Web Sockets Today - Rick Garibay 5:30 PM 7:00 PM Welcome Reception START TIME END TIME Visual Studio Live! Day 2: Wednesday, March 27, 2013 7:00 AM 8:00 AM Registration - Coffee and Morning Pastries 8:00 AM 9:00 AM Keynote: To Be Announced 9:15 AM 10:30 AM W01 - Building Your First Windows Phone W02 - Azure - Vishwas Lele 8 Application - Brian Peek 10:45 AM 12:00 PM W06 - Cross Win8/WP8 Apps - W07 - Azure - Vishwas Lele Ben Dewey 12:00 PM 2:30 PM Birds-of-a-Feather Lunch & Expo Hall 1:15 PM 2:15 PM W11 - Chalk Talk: Moving Web Apps to the W12 - Chalk Talk: Improving Web Cloud - Eric D. Boyd Performance - Robert Boedigheimer 2:30 PM 3:45 PM W15 - Designing Your Windows Phone W16 - IaaS in Windows Azure with Apps for Multitasking and Background Virtual Machines - Eric D. Boyd Processing - Nick Landry Bonus Las Vegas 3:45 PM 4:15 PM Sponsored Break - Exhibitor RafÁ e 4:15 PM 5:30 PM W20 - Building a Windows Runtime W21 - Bringing Open Source to Component with C# - Brian Peek Windows Azure: A Match Made in Content! Heaven - Jesus Rodriguez 6:30 PM 8:30 PM Evening Event START TIME END TIME Visual Studio Live! Day 3: Thursday, March 28, 2013 Developer Deep Dives 7:00 AM 8:00 AM Registration - Coffee and Morning Pastries 8:00 AM 9:15 AM TH01 - Building Extensible XAML Client TH02 - JavaScript, Meet Cloud: Node.js on sharepoint and Apps - Brian Noyes on Windows Azure - Sasha Goldshtein 9:30 AM 10:45 AM TH06 - Migrating from WPF or Silverlight TH07 - Using Windows Azure to Build SQL server – brought to WinRT - Rockford Lhotka the Next Generation of Mobile Applications - Jesus Rodriguez to you by: 11:00 AM 12:15 PM TH11 - Managing the .NET Compiler - TH12 - Cloud Backends for Your Mobile Jason Bock Apps: Windows Azure Mobile Services and Parse - Sasha Goldshtein 12:15 PM 1:30 PM Lunch 1:30 PM 2:45 PM TH16 - Understanding Dependency TH17 - Using Windows Azure for Injection and Those Pesky Containers - Solving Identity Management Miguel Castro Challenges - Michael Collier 3:00 PM 4:15 PM TH21 - Static Analysis in .NET - TH22 - Elevating Windows Azure Jason Bock Deployments - Michael Collier START TIME END TIME Visual Studio Live! Post-Conference Workshops: Friday, March 29, 2 7:30 AM 8:00 AM Post-Conference Workshop Registration - Coffee and Morning Pastries 8:00 AM 5:00 PM FW02 - Workshop: Happy ALM with Visual Studio 2012 and Team Foundation Server 2012 - Brian Randell Speakers and sessions subject to change Untitled-9 4 1/2/13 1:53 PM MARCH las vegas 25-29, 2013 CONNECT WITH Azure / Cloud Computing Data Management HTML5 / JavaScript Cross-Platform Mobile SQL Server VISUAL STUDIO LIVE! 5, 2013 (Separate entry fee required) twitter.com/vslive – @VSLive MW03 - Workshop: HTML5 MW04 - Workshop: SharePoint 2013 Developer Boot Camp - + Cloud - Reach Everyone, Andrew Connell Everywhere - Eric D. Boyd facebook.com – Search “VSLive” T03 - Busy Developer's Guide to T04 - Mastering Visual Studio T05 - Building Your First SharePoint linkedin.com – Join the “VSLive” group! MongoDB - Ted Neward 2012 - Deborah Kurata 2013 Application Using Visual Studio 2012 - Darrin Bishop T08 - Busy Developer's Guide to T09 - IntelliTrace, What is it and T10 - Getting Started with Microsoft Cassandra - Ted Neward How Can I Use it to My BeneÀ ts? - OfÀ ce 365 SharePoint Online Marcel de Vries Development T13 - Chalk Talk: Building a T14 - Chalk Talk: Acing Application Lifecycle Management URL Shortening Service with in SharePoint Node.js - Rick Garibay T17 - What's New in T18 - Team Foundation Server T19 - Unit Testing in SharePoint - ASP.NET 4.5 - Adam Tuliper 2012 Builds: Understand, Jim Wooley ConÀ gure, and Customize - Benjamin Day T22 - 25 Tips and Tricks for the T23 - Design for Testability: T24 - Better Together - ASP.NET Developer - Mocks, Stubs, Refactoring, and SharePoint 2013 and Mobile Adam Tuliper User Interfaces - Benjamin Day Development - Darrin Bishop W03 - Controlling ASP.NET W04 - Modern ALM and the W05 - Developing and MVC4 - Philip Japikse DevOps Story - Brian Randell Extending Enterprise Content Management Features with SharePoint 2013 - Paul Swider W08 - MVC For WebForms W09 - OData - Sergey Barskiy W10 - Use 2012 (and Beyond) Developers: Comparing and Technology with SharePoint Contrasting - Miguel Castro 2010 - Ryan McIntyre W13 - Chalk Talk: NoSQL for the W14 - Demystifying the Microsoft UI Technology Roadmap - SQL Guy - Ted Neward Brian Noyes W17 - ASP.NET MVC - AJAX in W18 - Code First and Entity W19 - Build Modern your Views - Walt Ritscher Framework - Sergey Barskiy Collaborative Solutions with OfÀ ce 2013, "Napa" OfÀ ce 365 Development Tools, and SharePoint 2013 - Brian Randell W22 - Patterns for Parallel W23 - LINQ performance and W24 - Intro to Windows Azure Programming - Tiberiu Covaci Scalability - Jim Wooley SQL Database and What's New - Eric D. Boyd TH03 - To Be Announced TH04 - Sharing up to 80% of TH05 - SQL Server Data Tools - code building Mobile apps for Leonard Lobel iOS, Android, WP 8 and Windows 8 - Marcel de Vries TH08 - ASP.NET MVC - Routing TH09 - iOS Development TH10 - Programming the T-SQL in the spotlight - Walt Ritscher Survival Guide for the .NET Guy - Enhancements in SQL Server Scan the QR Nick Landry 2012 - Leonard Lobel code to register TH13 - From 0 to Web Site in TH14 - To Be Announced TH15 - Getting to know the BI or for more 60 Minutes with Web Matrix - Semantic Model - Andrew Brust event details. Mark Rosenberg TH18 - Creating Web Sites TH19 - Building Multi-Platform TH20 - Big Data-BI Fusion: Using Visual Studio Mobile Apps with Push Microsoft HDInsight & MS BI - LightSwitch - NotiÀ cations - Nick Landry Andrew Brust Michael Washington TH23 - Building Single Page TH24 - Create HTML 5 Mobile TH25 - Optimizing Stored Web Applications with HTML5, websites with Visual Studio Procedures - Mark Rosenberg ASP.NET MVC4 and Web API - LightSwitch - Register at vslive.com/ Marcel de Vries Michael Washington 9, 2013 (Separate entry fee required) lasvegas FW02 - Workshop: SQL Server 2012 - Andrew Brust & Leonard Lobel Use Promo Code LVFEB4 Untitled-9 5 1/2/13 1:47 PM ASYNCHRONOUS PROGRAMMING Async Causality Chain Tracking Andrew Stasyuk With the advent of C# 5, Visual Basic .NET 11, the Microsoft to provide means to bridge this gap and improve the asynchronous .NET Framework 4.5 and .NET for Windows Store apps, the asyn- debugging experience in your .NET 4.5 or Windows Store app. chronous programming experience has been streamlined greatly. Let’s settle on essential terminology fi rst. New async and await keywords (Async and Await in Visual Basic) allow developers to maintain the same abstraction they were used Defi nition of a Call Stack to when writing synchronous code. MSDN documentation (bit.ly/Tukvkm) used to defi ne call stack as “the A lot of eff ort was put into Visual Studio 2012 to improve asyn- series of method calls leading from the beginning of the program chronous debugging with tools such as Parallel Stacks, Parallel to the statement currently being executed at run time.” Th is notion Tasks, Parallel Watch and the Concurrency Visualizer. However, was perfectly valid for the single-threaded, synchronous program- in terms of being on par with the synchronous code debugging ming model, but now that parallelism and asynchrony are gaining experience, we’re not quite there yet. momentum, more precise taxonomy is necessary. One of the more prominent issues that breaks the abstraction and For the purpose of this article, it’s important to distinguish the reveals internal plumbing behind the async/await façade is the lack causality chain from the return stack. Within the synchronous of call stack information in the debugger. In this article, I’m going paradigm, these two terms are mostly identical (I’ll mention the exceptional case later). In asynchronous code, the aforementioned This article discusses: defi nition describes a causality chain. • Causality chains versus return stacks On the other hand, the statement currently being executed, when fi nished, will lead to a series of methods continuing their execution. • Asynchronous debugging with existing tools Th is series constitutes the return stack. Alternatively, for readers • Preserving causality chains in classic and Windows Store apps familiar with the continuation passing style (Eric Lippert has a fab- • Using EventSource and EventListener ulous series on this topic, starting at bit.ly/d9V0Dc), the return stack • Emulating async-local storage might be defi ned as a series of continuations that are registered to • Comparison of causality tracking approaches and caveats execute, should the currently executing method complete. Technologies discussed: In a nutshell, the causality chain answers the question, “How did I get here?” while return stack is the answer for, “Where do I Visual Studio 2012, Microsoft .NET Framework 4.5, .NET for go next?” For example, if you’ve got a deadlock in your application, Windows Store Apps you might be able to fi nd out what caused it from the former, while Code download available at: the latter would let you know what the consequences are. Note archive.msdn.microsoft.com/mag201302Causality that while a causality chain always tracks back to the program entry point, the return stack is cut off at the point where the result 32 msdn magazine of asynchronous operation is not observed (for example, async void causality chain/return stack when execution is paused at the methods or work scheduled via Th readPool.QueueUserWorkItem). Console.WriteLine method: Th ere’s also a notion of stack trace being a copy of a synchronous call ConsoleSample.exe!ConsoleSample.Program.OperationAsync() Line 19 ConsoleSample.exe!ConsoleSample.Program.Main() Line 13 stack preserved for diagnostics; I’ll use these two terms interchangeably. But if you try this, you’ll fi nd that in the Call Stack window the Be aware that there are several unspoken assumptions in the Main method is missing, while the stack trace starts directly in the preceding defi nitions: OperationAsync method preceded by [Resuming Async Method]. • “Method calls” referred to in the fi rst defi nition generally Parallel Stacks has both methods; however, it doesn’t show that Main imply “methods that have not completed yet,” which bear calls OperationAsync. Parallel Tasks doesn’t help either, showing the physical meaning of “being on stack” in the synchro- “No tasks to display.” nous programming model. However, while we’re generally Note: At this point the debugger is aware of the Main method being not interested in methods that have already returned, it’s part of the call stack—you might have noticed that by the gray back- not always possible to distinguish them during asynchro- ground behind the call to OperationAsync. Th e CLR and Windows nous debugging. In this case, there’s no physical notion of Runtime (WinRT) have to know where to continue execution aft er “being on stack” and all continuations are equally valid the topmost stack frame returns; thus, they do indeed store return elements of a causality chain. stacks. In this article, though, I’ll only delve into causality tracking, • Even in synchronous code, a causality chain and return leaving return stacks as a topic for another article. stack aren’t always identical. One particular case when a method might be present in one, but missing from the other, Preserving Causality Chains is a tail call. Though not directly expressible in C# and In fact, causality chains are never stored by the runtime. Even Visual Basic .NET, it may be coded in Intermediate Lan- call stacks that you see when debugging synchronous code are, in guage (IL) (“tail.” prefi x) or produced by the just-in-time essence, return stacks—as was just said, they’re necessary for the (JIT) compiler (especially in a 64-bit process). CLR and Windows Runtime to know which methods to execute • Last, but not least, causality chains and return stacks can be aft er the topmost frame returns. Th e runtime doesn’t need to know nonlinear. Th at is, in the most general case, they’re directed what caused a particular method to execute. graphs having current statement as a sink (causality graph) To be able to view causality chains during live and post-mortem or source (return graph). Nonlinearity in asynchronous debugging, you have to explicitly preserve them along the way. code is due to forks (parallel asynchronous operations Presumably, this would require storing (synchronous) stack trace originating from one) and joins (continuation scheduled information at every point where continuation is scheduled and to run upon completion of a set of parallel asynchronous restoring this data when continuation starts to execute. These operations). For the purpose of this article, and due to plat- stack trace segments could then be stitched together to form a form limitations (explained later), I’ll consider only linear causality chain. causality chains and return stacks, which are subsets of We’re more interested in transferring causality information corresponding graphs. across await constructs, as this is where abstraction of similarity Luckily, if asynchrony is introduced into a program by using async with synchronous code breaks. Let’s see how and when this data and await keywords with no forks or joins, and all async methods can be captured. are awaited, the causality chain is still identical to the return stack, As Stephen Toub points out (bit.ly/yF8eGu), provided that FooAsync just as in synchronous code. In this case, both of them are equally returns a Task, the following code: useful in orienting yourself in the control fl ow. await FooAsync(); On the other hand, causality chains are rarely equal to return RestOfMethod(); stacks in programs employing explicitly scheduled continuations, is transformed by the compiler to a rough equivalent of this: var t = FooAsync(); a notable example being Task Parallel Library (TPL) datafl ow. This var currentContext = SynchronizationContext.Current; is due to the nature of data fl owing from a source block to a target t.ContinueWith(delegate { block, never returning to the former. if (currentContext == null) RestOfMethod(); else Existing Tools currentContext.Post(delegate { RestOfMethod(); }, null); Consider a quick example: }, TaskScheduler.Current); static void Main() From looking at the expanded code, it appears there are at least { OperationAsync().Wait(); two extension points that might allow for capturing causality infor- } mation: TaskScheduler and SynchronizationContext. Indeed, both async static Task OperationAsync() off er similar pairs of virtual methods where it should be possible { to capture call stack segments at the right moments: QueueTask/ await Task.Delay(1000); Console.WriteLine("Where is my call stack?"); TryDequeue on TaskScheduler and Post/OperationStarted on } SynchronizationContext. By extrapolating the abstraction developers were used to in Unfortunately, you can only substitute default TaskScheduler synchronous debugging, they would expect to see the following when explicitly scheduling a delegate via the TPL API, such msdnmagazine.com February 2013 33 as Task.Run, Task.ContinueWith, TaskFactory.StartNew and Introducing EventListener so on. This means that whenever continuation is scheduled In the .NET Framework 4, in order to capture ETW events, you had outside a running task, the default TaskScheduler will be in force. to be running an out-of-process ETW listener, such as Windows Th us, the TaskScheduler-based approach won’t be able to capture Performance Recorder or Vance Morrison’s PerfView, and then cor- necessary information. relate captured data with the state you observed in the debugger. As for SynchronizationContext, although it’s possible to override Th is posed additional problems, as data was stored outside process the default instance of this class for the current thread by calling memory space and crash dumps didn’t include it, which made this the SynchronizationContext.SetSynchronizationContext method, solution less suitable for post-mortem debugging. For example, if this has to be done for every thread in the application. Th us, you’d you rely on Windows Error Reporting to provide dumps, you won’t have to be able to control thread lifetime, which is infeasible if get any ETW traces and thus causality information will be missing. you aren’t planning to re-implement a thread pool. Moreover, However, starting in the .NET Framework 4.5, it’s possible to Windows Forms, Windows Presentation Foundation (WPF) and subscribe to TPL events (and other events fi red by EventSource ASP.NET all provide their own implementations of Synchroniza- inheritors) via System.Diagnostics.Tracing.EventListener (bit.ly/XJelwF). tionContext in addition to SynchronizationContext.Default, which Th is allows the capture and preservation of stack trace segments in the schedules work to the thread pool. Hence, your implementation process memory space. Th erefore, a mini-dump with heap should be would have to behave diff erently depending on the origin of the enough to extract causality information. In this article, I’ll only detail thread in which it’s working. EventListener-based subscriptions. Also note that when awaiting a custom awaitable, it’s entirely It’s worth mentioning that the advantage of an out-of-process up to implementation whether to use SynchronizationContext to listener is that you can always get the call stacks by listening to the Stack schedule a continuation. ETW Events (either relying on an existing tool or doing tedious stack Luckily, there are two extension points suitable for our scenario: walking and module address tracking yourself). When subscribing subscribing to TPL events without having to modify the existing to the events using EventListener, you can’t get call stack information codebase, or explicitly opting in by slightly modifying every await in Windows Store apps, because the StackTrace API is prohibited. expression in the application. The first approach only works in (An approach that works for Windows Store apps is described later.) desktop .NET applications, while the second can accommodate In order to subscribe to events, you have to inherit from Event- Windows Store apps. I’ll detail both in the following sections. Listener, override the OnEventSourceCreated method and make sure that an instance of your listener gets created in every Introducing EventSource AppDomain of your program (subscription is per application Th e .NET Framework supports Event Tracing for Windows (ETW), domain). Aft er EventListener is instantiated, this method will be having defined event providers for practically every aspect of called to notify the listener of event sources that are being created. the runtime (bit.ly/VDfrtP). Particularly, TPL fi res events that allow It will also provide notifi cations for all event sources that existed you to track Task lifetime. Although not all of these events are before the listener was created. Aft er fi ltering event sources either documented, you can obtain their defi nitions yourself by delving by name or GUID (performance-wise, comparing GUIDs is a better into mscorlib.dll with a tool such as ILSpy or Refl ector or peeking idea), a call to EnableEvents subscribes the listener to the source: bit.ly/HRU3 into framework reference source ( ) and searching for the private static readonly Guid tplGuid = TplEtwProvider class. Of course, the usual refl ection disclaimer new Guid("2e5dba47-a3d2-4d16-8ee0-6671ffdcd7b5"); applies: If the API isn’t documented, there’s no guarantee that protected override void OnEventSourceCreated(EventSource eventSource) empirically observed behavior will be retained in the next release. { if (eventSource.Guid == tplGuid) TplEtwProvider inherits from System.Diagnostics.Tracing.Event- EnableEvents(eventSource, EventLevel.LogAlways); Source, which was introduced in the .NET Framework 4.5 and is } now a recommended way to fi re ETW events in your application To process events, you need to implement abstract method (previously you had to deal with manual ETW manifest genera- OnEventWritten. For the purpose of preserving and restoring tion). In addition, EventSource allows for consumption of events stack trace segments, you need to capture the call stack right in process, by subscribing to them via EventListener, also new in before an asynchronous operation is scheduled, and then, when it the .NET Framework 4.5 (more on this momentarily). starts execution, associate a stored stack trace segment with it. To Th e event provider can be identifi ed by either a name or GUID. Each correlate these two events, you can use the TaskID parameter. particular event type is in turn identifi ed by event ID and, optionally, Parameters passed to a corresponding event-fi ring method in an a keyword to distinguish from other unrelated types of events fi red event source are boxed into a read-only object collection and passed by this provider (TplEtwProvider doesn’t use keywords). Th ere are in as the Payload property of EventWrittenEventArgs. optional Task and Opcode parameters that you might fi nd useful Interestingly, there are special fast paths for EventSource events that for fi ltering, but I’ll rely solely on event ID. Each event also defi nes are consumed as ETW (not via EventListener), where boxing doesn’t the level of verbosity. occur for their arguments. Th is does provide a performance improve- TPL events have a variety of uses besides causality chains, such ment, but it’s mostly zeroed out due to cross-process machinery. as tracking of tasks in-fl ight, telemetry and so on. Th ey don’t fi re In the OnEventWritten method, you need to distinguish between for custom awaitables, though. event sources (in case you subscribe to more than one) and identify 34 msdn magazine Asynchronous Programming /update/2013/02 www.componentsource.com BESTBEST SELLER SELLER ComponentOne Studio Enterprise from $1,315.60 .NET Tools for the Smart Developer: Windows, Web, and XAML. • Hundreds of UI controls for all .NET platforms including grids, charts, reports and schedulers • Supports Visual Studio 2012 and Windows 8 • Now includes Windows 8 Studios for WinRT XAML and WinJS • New Cosmopolitan (Windows 8 UI) theme provides a modern look and feel • Royalty-free deployment and distribution BEST SELLER Help & Manual Professional from $583.10 Easily create documentation for Windows, the Web and iPad. • Powerful features in an easy accessible and intuitive user interface • As easy to use as a word processor, but with all the power of a true WYSIWYG XML editor • Single source, multi-channel publishing with conditional and customized output features • Output to HTML, WebHelp, CHM, PDF, ePUB, RTF, e-book or print • Styles and Templates give you full design control BEST SELLER Aspose.Total for .NET from $2,449.02 Every Aspose .NET component in one package. • Programmatically manage popular fi le formats including Word, Excel, PowerPoint and PDF • Add charting, email, spell checking, barcode creation, OCR, diagramming, imaging, project management and fi le format management to your .NET applications • Common uses also include mail merge, adding barcodes to documents, building dynamic Excel reports on the fl y and extracting text from PDF lesfi BEST SELLER Code Compare Pro from $48.95 An advanced visual fi le comparison tool with Visual Studio integration. • Code oriented comparison, including syntax highlighting, unique structure and lexical comparison algorithms, for the most popular programming languages • Smooth Visual Studio integration to develop and merge within one environment in the context of current solution, using native IDE editors • Three-way fi le merge, folder comparison and synchronization We accept purchase orders. © 1996-2013 ComponentSource. All Rights Reserved. All prices correct at the time of press. Online prices may vary from those shown due to daily fluctuations & online discounts. Contact us to apply for a credit account. US Headquarters European Headquarters Asia / Pacific Headquarters ComponentSource ComponentSource ComponentSource Sales Hotline - US & Canada: 650 Claremore Prof Way 30 Greyfriars Road 3F Kojimachi Square Bldg Suite 100 Reading 3-3 Kojimachi Chiyoda-ku Woodstock Berkshire Tokyo (888) 850-9911 GA 30188-5188 RG1 1PE Japan USA United Kingdom 102-0083 www.componentsource.com Untitled-8 1 1/2/13 1:15 PM Figure 1 Handling of TPL Events Bear in mind that CallContext (actually, internally there are two in the OnEventWritten Method of them: LogicalCallContext and IllogicalCallContext) is a heavy protected override void OnEventWritten(EventWrittenEventArgs eventData) object, designed to flow across remoting boundaries. When no { custom data is stored, the runtime doesn’t initialize the contexts, if (eventData.EventSource.Guid == tplGuid) { sparing the cost of maintaining them with the control fl ow. As soon int taskId; as you call the CallContext.LogicalSetData method, a mutable switch (eventData.EventId) { ExecutionContext and several Hashtables have to be created and case 7: // Task scheduled passed along or cloned from then on. taskId = (int)eventData.Payload[2]; stackStorage.StoreStack(taskId); Unfortunately, ExecutionContext (together with all its constitu- break; ents) is captured before the described TPL events fi re and restored case 10: // Task wait begin taskId = (int)eventData.Payload[2]; shortly aft erward. Th us, any custom data saved in CallContext in bool waitBehaviorIsSynchronous = between is discarded after ExecutionContext is restored, which (int)eventData.Payload[3] == 1; if (!waitBehaviorIsSynchronous) makes it unsuitable for our particular purpose. stackStorage.StoreStack(taskId); In addition, the CallContext class isn’t available in the .NET break; case 11: // Task wait end for Windows Store apps subset, so an alternative is needed for taskId = (int)eventData.Payload[2]; this scenario. stackStorage.RestoreStack(taskId); break; One way to build an async-local storage that would work around } these problems is to maintain the value in thread-local storage (TLS) } } while the synchronous portion of code is executing. Th en, when the TaskWaitStart event fi res, store the value in a shared (non-TLS) dictionary, keyed by the TaskID. When the counterpart event, the event itself. The stack trace will be captured (stored) when TaskWaitEnd, fi res, remove the preserved value from the dictionary TaskScheduled or TaskWaitBegin events fi re, and associated with and save it back to TLS, possibly on a diff erent thread. a newly started asynchronous operation (restored) in TaskWait- As you might know, values stored in TLS are preserved even End. You also need to pass in taskId as the correlation identifi er. aft er a thread is returned to the thread pool and gets new work to Figure 1 shows the outline of how the events will be handled. execute. So, at some point, the value has to be removed from TLS Note: Explicit values (“magic numbers”) in code are a bad program- (otherwise, some other asynchronous operation executing on this ming practice and are used here only for brevity. Th e accompanying thread later might access the value stored by the previous operation sample code project has them conveniently structured in constants as if it were its own). You can’t do this in the TaskWaitBegin event and enumerations to avoid duplication and risk of typos. handler because, in case of nested awaits, TaskWaitBegin and Task- Note that in TaskWaitBegin, I check for TaskWaitBehavior being WaitEnd events occur multiple times, once per await, and a stored synchronous, which happens when a task being awaited is executed value might be needed in between, such as in the following snippet: synchronously or has already completed. In this case, a synchronous async Task OuterAsync() call stack is still in place, so it doesn’t need to be stored explicitly. { await InnerAsync(); } Async-Local Storage async Task InnerAsync() { Whatever data structure you choose to preserve call stack segments await Task.Delay(1000); needs the following quality: Stored value (causality chain) should be } preserved for every asynchronous operation, following control fl ow Instead, it’s safe to consider that the value in TLS is eligible to along the way across await boundaries and continuations, bearing in be cleared when the current asynchronous operation is no longer mind that continuations may execute on diff erent threads. being executed on a thread. Because the CLR doesn’t have an in- Th is suggests a thread-local-like variable that would preserve its process event that would notify of a thread being recycled back to value pertaining to the current asynchronous operation (a chain the thread pool (there’s an ETW one—bit.ly/ZfAWrb), for this purpose of continuations), instead of a particular thread. It can be roughly I’ll use Th readPoolDequeueWork fi red by FrameworkEventSource named “async-local storage.” (also undocumented), which occurs when a new operation is Th e CLR already has a data structure called ExecutionContext started on a thread pool thread. Th is leaves out non-pooled threads, that’s captured on one thread and restored on the other (where for which you’d have to manually clean the TLS, such as when a UI continuation gets to execute), thus being passed along with con- thread returns to the message loop. trol fl ow. isTh is essentially a container that stores other contexts For a working implementation of this concept together with (SynchronizationContext, CallContext and so on) that might be stack segments capturing and concatenation, please refer to the needed to continue execution in exactly the same environment, StackStorage class in the accompanying source code download. where they were interrupted. Stephen Toub has the details at bit.ly/M0amHk. Th ere’s also a cleaner abstraction, AsyncLocal 36 msdn magazine Asynchronous Programming Untitled-1 1 10/13/11 11:25 AM Tracing Causality in Windows Store Apps Viewing Causality Chains Th e described approach would still hold up in a Windows Store sce- Both described approaches keep causality information in memory nario if the System.Diagnostics.StackTrace API were available. For as lists of call stack segments or frames. However, walking them better or for worse, it isn’t, which means you can’t get any information and concatenating into a single causality chain for display is about call stack frames above the current one from within your code. tedious to do by hand. Th us, even while TPL events are still supported, a call to TaskWaitStart Th e easiest option to automate this is to leverage the debugger eval- or TaskWaitEnd is buried deep in the framework method calls, so you uator. In this case, you author a public static property (or method) on have no information about your code that caused these events to fi re. a public class, which, when called, walks the list of stored segments Luckily, .NET for Windows Store apps (as well as the .NET and returns a concatenated causality chain. Th en you can evaluate this Framework 4.5) provides CallerMemberNameAttribute property during debugging and see the result in the text visualizer. (bit.ly/PsDH0p) and its peers CallerFilePathAttribute and CallerLine- Unfortunately, this approach doesn’t work in two situations. NumberAttribute. When optional method arguments are deco- One occurs when the topmost stack frame is in native code, which rated with these, the compiler will initialize the arguments with is quite a common scenario for debugging application hangs, as corresponding values at compile time. For example, the following kernel-based synchronization primitives do call into native code. code will output “Main() in c:\Full\Path\To\Program.cs at line 14”: The debugger evaluator would just display, “Cannot evaluate static void Main(string[] args) expression because the code of the current method is optimized” { bit.ly/SLlNuT LogCurrentFrame(); (Mike Stall describes these limitations in detail at ). } Th e other issue is with post-mortem debugging. You can actu- static void LogCurrentFrame([CallerMemberName] string name = null, ally open a mini-dump in Visual Studio and, surprisingly (given [CallerFilePath] string path = null, [CallerLineNumber] int line = 0) that there’s no process to debug, only its memory dump), you’re { Console.WriteLine("{0}() in {1} at line {2}", name, path, line); allowed to examine property values (run property getters) and } even call some methods! Th is amazing piece of functionality is Th is only allows the logging method to get information about built into the Visual Studio debugger and works by interpreting a the calling frame, which means you have to ensure it gets called watch expression and all methods that it calls into (in contrast to from all the methods you want captured in the causality chain. live debugging, where compiled code gets executed). One convenient location for this would be decorating each await Obviously, there are limitations. For example, while doing dump expression with a call to an extension method, like this: debugging, you can’t in any way call into native methods (meaning await WorkAsync().WithCausality(); that you can’t even execute a delegate, because its Invoke method is Here, the WithCausality method captures the current frame, generated in native code) or access some restricted APIs (such as appends it to causality chain and returns a Task or awaitable System.Refl ection). Interpreter-based evaluation is also expectedly (depending on what WorkAsync returns), which upon completion slow—and, sadly, due to a bug, the evaluation timeout for dump of the original one removes the frame from the causality chain. debugging is limited to 1 second in Visual Studio 2012, regardless of As multiple diff erent things can be awaited, there should be confi guration. Th is, given the number of method calls required to multiple overloads of WithCausality. Th is is straightforward for a traverse the list of stack trace segments and iterate over all frames, Task 38 msdn magazine Asynchronous Programming WHAT IF DEVELOPING FOR HUNDREDS OF SERVERS WAS AS EASY AS ONE? IT IS... MEET FATCLOUD The next generation cloud enabled application platform for .NET is here. For a free three-node developer download visit FATCLOUD.COM. Untitled-13 1 1/7/13 12:54 PM An additional benefi t of the TPL events-based tracker is that This issue doesn’t affect the caller-info-based tracker, as it existing asynchronous code doesn’t have to be modified, while removes a frame from the list immediately upon the start of a for the caller info attributes-based approach, you have to alter continuation. Th ere are two (combinable) approaches to fi x this every await statement in your program. But only the latter supports for the TPL events scenario. One is to cut older data based on the Windows Store apps. rolling maximum storage amount. Th e other is to represent loops Th e TPL events tracker also suff ers from a lot of boilerplate frame- effi ciently and avoid duplication. For both approaches, you might work code in stack trace segments, though it can be easily fi ltered also detect common infi nite loop patterns and cut the causality out by frame namespace or class name. See the sample code for a chain explicitly at these points. list of common fi lters. Feel free to refer to the accompanying sample project to see how Another caveat concerns loops in asynchronous code. Consider loop folding might be implemented. the following snippet: As stated, the TPL events API only lets you capture a cau- async static Task Loop() sality chain, not a graph. This is because the Task.WaitAll and { for (int i = 0; i < 10; i++) Task.WhenAll methods are implemented as countdowns, where { continuation is scheduled only when the last task comes in com- await FirstAsync(); await SecondAsync(); pleted and the counter reaches zero. Th us, only the last completed await ThirdAsync(); task forms a causality chain. } } By the end of the method, its causality chain would grow to Wrapping Up more than 30 segments, repeatedly alternating between FirstAsync, In this article, you’ve learned the diff erence between a call stack, SecondAsync and Th irdAsync frames. For a fi nite loop, this may a return stack and a causality chain. You should now be aware of exten- be tolerable, though it’s still a waste of memory to store duplicate sion points that the .NET Framework provides to track scheduling frames 10 times. However, in some cases, a program might intro- and execution of asynchronous operations and be able to leverage duce a valid infi nite loop, for example, in the case of a message these to capture and preserve causality chains. The approaches loop. Moreover, infi nite repetition might be introduced without described cover tracking causality in classic and Windows Store loop or await constructs—a timer rescheduling itself on every tick apps, both in live and post-mortem debugging scenarios. You also is a perfect example. Tracking an infi nite causality chain is a sure learned about the concept of async-local storage and its possible way to run out of memory, so the amount of data stored has to be implementation for Windows Store apps. reduced to a fi nite amount somehow. Now you can go ahead and incorporate causality tracking into your asynchronous codebase or use async-local storage in parallel calculations; explore the event sources that the .NET Framework 4.5 and .NET for Windows Store apps off er to build something new, such as a tracker for unfinished tasks in your program; or use this extension point to fi re your own events to fine-tune the perfor- mance of your application. Q ANDRIY (ANDREW) STASYUK is a software development engineer in test II on the Managed Languages team at Microsoft. He has seven years of experience as a participant, task author, jury member, and coach at various national and international programming contests. He worked in fi nancial soft ware development at Paladyne/ Broadridge Financial Solutions Inc. and Deutsche Bank AG before moving to Microsoft . His main interests in programming are algorithms, parallelism and brainteasers. THANKS to the following technical experts for reviewing this article: Figure 2 Causality Chain for an Asynchronous Method and “Parallel” Causality for All Threads Vance Morrison and Lucian Wischik 40 msdn magazine Asynchronous Programming Untitled-13 1 1/7/13 12:58 PM ASP.NET Building a Simple Comet Application in the Microsoft .NET Framework Derrick Lau Comet is a technique for pushing content from a Web solution. However, the browser must support AJAX by implementing server to a browser without an explicit request, using long-lived the xmlHttpRequest object; otherwise it won’t be able to support AJAX connections. It allows for a more interactive UX and uses Comet-style communication. less bandwidth than the typical server round-trip triggered by a page postback to retrieve more data. Although there are plenty The High-Level Architecture of Comet implementations available, most are Java-based. In this Figure 1 shows basic Comet-style communication, while article I’ll focus on building a C# service based on the cometbox Figure 2 depicts the architecture of my example. Comet uses the code sample available at code.google.com/p/cometbox. browser’s xmlHttpRequest object, which is essential for AJAX Th ere are newer methods for implementing the same behavior communication, to establish a long-lived HTTP connection using HTML5 features such as WebSockets and server-side events, to a server. The server holds the connection open, and pushes but these are available only in the latest browser versions. If you content to the browser when available. must support older browsers, Comet is the most-compatible Between the browser and the server is a proxy page, which resides in the same Web application path as the Web page containing the client code and does nothing except forward the messages from This article discusses: browser to server and from server to browser. Why do you need a • Combining .NET and Windows services technologies with AJAX proxy page? I’ll explain in a bit. to enable Comet-style communication The first step is to select a format for the messages exchanged • Pitfalls in implementing Comet-style architecture between the browser and server—JSON, XML or a custom format. For simplicity’s sake, I picked JSON because it’s naturally supported • Testing the application in JavaScript, jQuery and the Microsoft .NET Framework, and can Technologies discussed: transmit the same amount of data as XML using fewer bytes and, ASP.NET 4, Windows Services, C#, JavaScript, JSON, therefore, less bandwidth. Microsoft .NET Framework To set up Comet-style communication, you open an AJAX con- Code download available at: nection to the server. Th e easiest way to do this is to use jQuery because it supports multiple browsers and provides some nice archive.msdn.microsoft.com/mag201302Comet wrapper functions such as $.ajax. This function is essentially a 42 msdn magazine Browser Proxy Comet Server recursively call the AJAX function again and return; I’ve found that adding the return stops the code from continuing. If the message is AJAX Request #1 OK, you simply read the message and write the contents to the page: $("#_receivedMsgLabel").append(msg.Message + " }); Message to This creates a simple client that illustrates how Comet-style Comet Response #1 Broadcast Received communication works, as well as providing a means for run- ning performance and scalability tests. For my example, I put the Comet Response #1 getResponse JavaScript code in a Web user control and registered it in the codebehind so the AJAX connection opens immediately when the control is loaded onto the ASP.NET page: public partial class JqueryJsonCometClientControl : Figure 1 Comet-Style Communication System.Web.UI.UserControl { protected void Page_Load(object sender, EventArgs e) wrapper for each browser’s xmlHttpRequest object, and neatly pro- { string getResponseScript = vides event handlers that can be implemented to process incoming @""; messages from the server. Page.ClientScript.RegisterStartupScript(GetType(), "GetResponseKey", getResponseScript); Before starting the connection, you instantiate the message } to send. To do this, declare a variable and use JSON.stringify to } format the data as a JSON message, as shown in Figure 3. Next, initialize the function with the URL to connect to, the The Server HTTP method of communication to use, the communication Now that I have a client that can send and receive messages, I’ll style and the connection timeout parameter. JQuery supplies this build a service that can receive and respond to them. functionality in a library call named ajaxSetup. I set the timeout I tried implementing several diff erent techniques for Comet-style in this example to 10 minutes because I’m only building a proof communication, including the use of ASP.NET pages and HTTP of concept solution here; you can change the timeout setting to handlers, none of which were successful. What I couldn’t seem to whatever you want. do was get a single message to broadcast to multiple clients. Luckily, Now open a connection to the server using the jQuery $.ajax aft er a lot of research I stumbled across the cometbox project and method, with the defi nition of the success event handler as the found it to be the easiest approach. I did some tinkering to make only parameter: it run as a Windows service so it would be easier to use, then gave $.ajax({ it the ability to hold a long-lived connection and push content to success: function (msg) { // Alert("ajax.success()."); the browser. (Unfortunately, in doing so, I wrecked some of the if (msg == null || msg.Message == null) { cross-platform compatibility.) Finally, I added support for JSON getResponse(); return; and my own HTTP content message types. } To get started, create a Windows service project in your Visual The handler tests the message object returned to ensure it Studio solution and add a service installer component (you’ll contains valid information before parsing; this is necessary because fi nd the instructions at bit.ly/TrHQ8O) so you can turn your service if an error code is returned, jQuery will fail and display an unde- on and off in the Services applet of the Administrative Tools in fi ned message to the user. Upon a null message, the handler should Control Panel. Once this is done, you need to create two threads: one that will bind to the TCP port and receive as well Web Browser ASP.NET Proxy Windows Service as transmit messages; and one that will block on a message queue to ensure that content is transmitted ASP.NET Page TCP Listener (Holds only when a message is received. onto Connection) Web User Control First, you must create a class that listens on the TCP port for new messages and transmits the responses. $.ajax Now, there are several styles of Comet communi- (xmlHttpRequest) TCP Network Stream cation that can be implemented, and in the imple- $.ajax success mentation there’s a Server class (see the code file handler Comet_Win_Service HTTP\Server.cs in the sample Message Queue code) to abstract these. For simplicity’s sake, how- ever, I’ll focus on what’s required to do a very basic receive of a JSON message over HTTP, and to hold Figure 2 Architecture of the Comet Application the connection until there’s content to push back. msdnmagazine.com February 2013 43 Figure 3 Format the Data as a JSON Message it. Th en I’ll start a thread to receive and handle clients that connect function getResponse() { to the TCP listener. var currentDate = new Date(); Th e thread that listens for client connections contains a while var sendMessage = JSON.stringify({ SendTimestamp: currentDate, loop that continually resets a fl ag indicating whether the service Message: "Message 1" Stop event was raised (see Figure 4). I set the fi rst part of this loop }); $.ajaxSetup({ to a mutex to block on all listening threads to check whether the url: "CometProxy.aspx", service Stop event was raised. If so, the _isListenerShutDown prop- type: "POST", async: true, erty will be true. When the check completes, the mutex is released global: true, and if the service is still running, I call the TcpListener.Accept- timeout: 600000 }); TcpClient, which will return a TcpClient object. Optionally, I check existing TcpClients to ensure I don’t add an existing client. However, depending on the number of clients you expect, you In the Server class, I’ll create some protected members to hold might want to replace this with a system where the service generates objects I’ll need to access from the Server object. Th ese include a unique ID and sends it to the browser client, which remembers the thread that will bind to and listen on the TCP port for HTTP and resends the ID each time it communicates with the server to connections, some semaphores and a list of client objects, each of ensure it holds only a single connection. Th is can become problem- which will represent a single connection to the server. Of impor- atic, though, if the service fails; it resets the ID counter and could tance is _isListenerShutDown, which will be exposed as a public give new clients already-used IDs. property so it can be modifi ed in the service Stop event. Finally, the thread goes through the list of clients and removes Next, in the constructor, I’ll instantiate the TCP Listener object any that are no longer alive. For simplicity, I put this code in the against the port, set it for exclusive use of the port, and then start method that’s called when the TCP listener accepts a client connec- tion, but this can aff ect performance when the number of clients Figure 4 Listening for Client Connections gets into the hundreds of thousands. If you intend on using this in public-facing Web applications, I suggest adding a timer that fi res private void Loop() { every so oft en and doing the cleanup in that. try When a TcpClient object is returned in the Server class Loop { while (true) method, it’s used to create a client object that represents the browser { client. Because each client object is created in a unique thread, as TcpClient client = null; bool isServerStopped = false; with the server constructor, the client class constructor must wait _listenerMutex.WaitOne(); on a mutex to ensure the client hasn’t been closed before continu- isServerStopped = _isListenerShutDown; _listenerMutex.ReleaseMutex(); ing. Aft erward, I check the TCP stream and begin reading it, and if (!isServerStopped) initiate a callback handler to be executed once the read has been { client = listener.AcceptTcpClient(); completed. In the callback handler, I simply read the bytes and } parse them using the ParseInput method, which you can see in else { the sample code provided with this article. continue; } Figure 5 The Default XML Message Handler Trace.WriteLineIf(_traceSwitch.TraceInfo, "TCP client accepted.", "COMET Server"); if (request.Headers["Content-Type"].Contains("xml")) { bool addClientFlag = true; Trace.WriteLineIf(_traceSwitch.TraceVerbose, "Received XML content from client."); _messageFormat = MessageFormat.xml; Client dc = new Client(client, this, authconfig, _currentClientId); _currentClientId++; #region Process HTTP message as XML foreach (Client currentClient in clients) { try if (dc.TCPClient == currentClient.TCPClient) { { // Picks up message from HTTP lock (_lockObj) XmlSerializer s = new XmlSerializer(typeof(Derrick.Web.SIServer.SIRequest)); { addClientFlag = false; // Loads message into object for processing } Derrick.Web.SIServer.SIRequest data = } (Derrick.Web.SIServer.SIRequest)s.Deserialize(mem); } } catch (Exception ex) { if (addClientFlag) Trace.WriteLineIf(_traceSwitch.TraceVerbose, { "During parse of client XML request got this exception: " + ex.ToString()); lock (_lockObj) } { clients.Add(dc); #endregion Process HTTP message as XML } } } 44 msdn magazine ASP.NET Untitled-1 1 1/11/10 10:55 AM Figure 6 Deserializing a JSON Message HTTP request received from the client, I send out an appropriate else if (request.Headers["Content-Type"].Contains("json")) error response message. { I separated the method for parsing the HTTP request’s content Trace.WriteLineIf(_traceSwitch.TraceVerbose, "Received json content from client."); so I could add in support for diff erent message types, such as plain _messageFormat = MessageFormat.json; text, XML, JSON and so forth: public void ParseRequestContent() #region Process HTTP message as JSON { try if (String.IsNullOrEmpty(request.Body)) { { JavaScriptSerializer jsonSerializer = new JavaScriptSerializer(); Trace.WriteLineIf(_traceSwitch.TraceVerbose, ClientMessage3 clientMessage = "No content in the body of the request!"); jsonSerializer.Deserialize else if (request.Headers["Content-Type"].Contains("ping")) shown in Figure 8, that simply contains members to hold the { headers and other information sent in the HTTP request in a manner string msg = request.Body; Trace.WriteLineIf(_traceSwitch.TraceVerbose, "Ping received."); easily accessible within the service. If you wish, you can add helper if (msg.Equals("PING")) methods to determine if the request has any content or not, and { SendMessageEventArgs args = new SendMessageEventArgs(); authentication checks, too. However, I didn’t do this here to keep args.Client = this; this service simple and easy to implement. args.Message = "PING"; args.Request = request; Th e Response class, like the Request class, contains methods to args.Timestamp = DateTime.Now; store the HTTP response information in a manner easily accessible SendResponse(args); } by a C# Windows service. In the SendResponse method, I added logic to attach custom HTTP headers as required for cross-origin } resource sharing (CORS), and had those headers loaded from a 46 msdn magazine ASP.NET Untitled-2 1 9/13/12 12:51 PM Figure 8 The Request Class if (SendMessageEvent != null) { public class Request SendMessageEvent(this, args); { Trace.WriteLineIf(_traceSwitch.TraceVerbose, public string Method; "Message loop raised SendMessage event."); public string Url; } public string Version; public string Body; Next, I need a method that will build the actual HTTP response body— public int ContentLength; the contents of the message the server will broadcast to all the clients. public Dictionary public bool HasContent() Microsoft message queue and formats it as a JSON object for transmis- { sion to the clients via an HTTP response message, as shown in Figure 11. if (Headers.ContainsKey("Content-Length")) { Next, I need to instantiate an instance of the JavaScriptSerializer ContentLength = int.Parse(Headers["Content-Length"]); object to put the message contents into JSON format. I add the return true; } following try/catch error handling because sometimes there are return false; diffi culties instantiating an instance of a JavaScriptSerializer object: } try { jsonSerializer = new JavaScriptSerializer(); confi guration fi le so they can be easily modifi ed. Th e Response } catch (Exception ex) class also contains methods to output messages for some common { HTTP statuses, such as 200, 401, 404, 405 and 500. errorInSendResponse = true; Trace.WriteLine("Cannot instantiate JSON serializer: " + ex.ToString()); Th e SendResponse member of the Response class simply writes } the message to the HTTP response stream that should still be alive, as the timeout set by the client is quite long (10 minutes): Figure 9 Adding the CORS Headers public void SendResponse(NetworkStream stream, Client client) { if (client.Request.Headers.ContainsKey("Origin")) As shown in Figure 9, the appropriate headers are added to the { AddHeader("Access-Control-Allow-Origin", client.Request.Headers["Origin"]); HTTP response to fi t with the W3C specifi cation for CORS. For Trace.WriteLineIf(_traceSwitch.TraceVerbose, simplicity, the headers are read from the confi guration fi le so the "Access-Control-Allow-Origin from client: " + client.Request.Headers["Origin"]); header contents can be easily modifi ed. } Now I add the regular HTTP response headers and content, as else { shown in Figure 10. AddHeader("Access-Control-Allow-Origin", Here the entire HTTP response message, which was built as a ConfigurationManager.AppSettings["RequestOriginUrl"]); Trace.WriteLineIf(_traceSwitch.TraceVerbose, String, is now written to the HTTP response stream, which was "Access-Control-Allow-Origin from config: " + passed in as a parameter to the SendResponse method: ConfigurationManager.AppSettings["RequestOriginUrl"]); } byte[] htext = Encoding.ASCII.GetBytes(r.ToString()); stream.Write(htext, 0, htext.Length); AddHeader("Access-Control-Allow-Methods", "POST, GET, OPTIONS"); AddHeader("Access-Control-Max-Age", "1000"); // AddHeader("Access-Control-Allow-Headers", "Content-Type"); Transmitting Messages string allowHeaders = ConfigurationManager.AppSettings["AllowHeaders"]; Th e thread to transmit messages is essentially nothing more than // AddHeader("Access-Control-Allow-Headers", "Content-Type, x-requested-with"); AddHeader("Access-Control-Allow-Headers", allowHeaders); a While loop that blocks on a Microsoft message queue. It has a SendMessage event that’s raised when the thread picks up a mes- StringBuilder r = new StringBuilder(); sage from the queue. Th e event is handled by a method in the server object that basically calls the SendResponse method of each client, Figure 10 Adding the Regular HTTP Response Headers thus broadcasting the message to every browser connected to it. r.Append("HTTP/1.1 " + GetStatusString(Status) + "\r\n"); r.Append("Server: Derrick Comet\r\n"); Th e thread waits on the appropriate message queue until there’s r.Append("Date: " + DateTime.Now.ToUniversalTime().ToString( a message placed on it, indicating the server has some content it "ddd, dd MMM yyyy HH':'mm':'ss 'GMT'") + "\r\n"); wishes to broadcast to the clients: r.Append("Accept-Ranges: none\r\n"); Message msg = _intranetBannerQueue.Receive(); foreach (KeyValuePair SendMessageEventArgs args = new SendMessageEventArgs(); if (File != null) args.Timestamp = DateTime.Now.ToUniversalTime(); { r.Append("Content-Type: " + Mime + "\r\n"); When the message is received, it’s converted into the expected r.Append("Content-Length: " + File.Length + "\r\n"); object type: } else if (Body.Length > 0) msg.Formatter = new XmlMessageFormatter(new Type[] { typeof(string) }); { string cometMsg = msg.Body.ToString(); r.Append("Content-Type: " + Mime + "\r\n"); args.Message = cometMsg; r.Append("Content-Length: " + Body.Length + "\r\n"); Aft er determining what will be sent to the clients, I raise a Windows } r.Append("\r\n"); event on the server indicating there’s a message to be broadcast: 48 msdn magazine ASP.NET YOUR BAACKKSTAGGE PAASS TO THHE MMICRROSOOFTT PPLATTFORRM MARCH las vegas 25-29, 2013 MGM Grand Hotel & Casino Intense Take-Home 70+ Sessions Training for Developers, Software Architects and Workshops! and Designers Visual Studio 2012 / .NET 4.5 ASP.NET l SharePoint LET US SQL Server l Windows 8 / WinRT Register before HEAR February 27 YOU and save $300! CODE vslive.com/lasvegas Use Promo Code TIP2 FLIP OVER FOR MORE EVENT TOPICS ¬ YOUR BAACKSTAGEE PPASS TO THE MMICROSSOFFT PLAATFOORMM Bonus Las Vegas ➤ ASP.NET Content! ➤ Azure / Cloud Computing ➤ Cross-Platform Mobile Developer Deep Dives ➤ Data Management on sharepoint and ➤ HTML5 / JavaScript SQL server – brought ➤ Deep Dive: SharePoint / Offi ce 365 ➤ Deep Dive: SQL Server to you by: ➤ Windows 8 / WinRT ➤ WPF / Silverlight ➤ Visual Studio 2012 / .NET 4.5 Scan the QR code for more information on vslive.com/lasvegas Visual Studio Live! Use Promo Code TIP2 SUPPORTED BY PRODUCED BY magazine Figure 11 Building the HTTP Response Body At this point, I have a reasonably decent working Comet service, public void SendResponse(SendMessageEventArgs args) which I can install into the services applet from the command { prompt using the installutil command (for more information, see bit.ly/OtQCB7 Trace.WriteLineIf(_traceSwitch.TraceVerbose, ). You could also create your own Windows installer to "Client.SendResponse(args) called..."); deploy it, as you’ve already added the service installer components if (args == null || args.Timestamp == null) to the service project. { return; } Why Doesn’t It Work? The Problem with CORS if (_lastUpdate > args.Timestamp) Now, try setting the URL in the $.ajax call of the browser client { return; to point to the Comet service URL. Start the Comet service and } open the browser client in Firefox. Make sure you have the Firebug bool errorInSendResponse = false; extension installed in the Firefox browser. Start Firebug and refresh the page; you’ll notice you get an error in the console output JavaScriptSerializer jsonSerializer = null; area stating “Access denied.” Th is is due to CORS, where for security reasons, JavaScript can’t access resources outside the same Web Th en I create a string variable to hold the JSON-formatted message application and virtual directory its housing page resides in. For and an instance of the Response class to send the JSON message. example, if your browser client page is in http://www.somedomain.com/ I immediately do some basic error checking to make sure I’m somedir1/somedir2/client.aspx, then any AJAX call made on that working with a valid HTTP request. Because this Comet service page can go only to resources in the same virtual directory or a spawns a thread for each TCP client, as well as for the server subdirectory. Th is is great if you’re calling another page or HTTP objects, I felt it safest to include these safety checks every so oft en, handler within the Web application, but you don’t want pages and to make debugging easier. handlers to block on a message queue when transmitting the same Once I verify that it’s a valid request, I put together a JSON message message to all clients, so you need to use the Windows Comet ser- to send to the HTTP response stream. Note that I just create the JSON vice and you need a way of getting around the CORS restriction. message, serialize it and use it to create an HTML response message: if (request.HasContent()) Figure 12 Writing to the HttpWebRequest Stream { Stream stream = null; if (_messageFormat == MessageFormat.json) { if (cometRequest.ContentLength > 0 && !cometRequest.Method.Equals("OPTIONS")) ClientMessage3 jsonObjectToSend = new ClientMessage3(); { jsonObjectToSend.SendTimestamp = args.Timestamp; stream = cometRequest.GetRequestStream(); jsonObjectToSend.Message = args.Message; stream.Write(bytes, 0, bytes.Length); jsonMessageToSend = jsonSerializer.Serialize(jsonObjectToSend); } response = Response.GetHtmlResponse(jsonMessageToSend, args.Timestamp, _messageFormat); if (stream != null) response.SendResponse(stream, this); { } stream.Close(); To hook it all together, I fi rst create instances of the message loop } object and the server loop object during the service Start event. Note // Console.WriteLine(System.Text.Encoding.ASCII.GetString(bytes)); System.Diagnostics.Trace.WriteLineIf(_proxySwitch.TraceVerbose, that these objects should be protected members of the service class "Forwarding message: " + System.Text.Encoding.ASCII.GetString(bytes)); so that methods on them can be called during other service events. Now the message loop send message event should be handled by Figure 13 Writing the Server Message the server object BroadcastMessage method: to the HTTP Response Stream public override void BroadcastMessage(Object sender, SendMessageEventArgs args) { string msgSizeStr = ConfigurationManager.AppSettings["MessageSize"]; // Throw new NotImplementedException(); int messageSize = Convert.ToInt32(msgSizeStr); Trace.WriteLineIf(_traceSwitch.TraceVerbose, "Broadcasting message [" + args.Message + "] to all clients."); byte[] read = new byte[messageSize]; // Reads 256 characters at a time int numOfClients = clients.Count; int count = s.Read(read, 0, messageSize); for (int i = 0; i < numOfClients; i++) while (count > 0) { { clients[i].SendResponse(args); // Dumps the 256 characters on a string and displays the string to the console } byte[] actualBytes = new byte[count]; } Array.Copy(read, actualBytes, count); string cometResponseStream = Encoding.ASCII.GetString(actualBytes); Th e BroadcastMessage just sends the same message to all clients. Response.Write(cometResponseStream); If you wish, you can modify it to send the message only to the count = s.Read(read, 0, messageSize); clients you want; in this way you can use this service to handle, for } instance, multiple online chat rooms. Response.End(); Th e OnStop method is called when the service is stopped. It subse- System.Diagnostics.Trace.WriteLineIf(_proxySwitch.TraceVerbose, "Sent Message."); quently calls the Shutdown method of the server object, which goes s.Close(); } through the list of client objects that are still valid and shuts them down. msdnmagazine.com February 2013 49 Figure 14 Varying the Number of Users Th is creates a connection to the Comet server for each user, but since the same message is being broadcast to each user, you can just Message Size Response Time Users Repetitions (in Bytes) (in Milliseconds) encapsulate the cometRequest object in a double locking singleton to reduce the connection load on the Comet server, and let IIS do 1,000 10 5122.56 the connection load balancing for you. 5,000 10 5124.404 Th en I populate the HttpWebRequest headers with the same values 10,000 10 512 18.406 I received from the jQuery client, especially setting the KeepAlive 15,000 10 512 26.368 property to true so I maintain a long-lived HTTP connection, which 20,000 10 512 36.612 is the fundamental technique behind Comet-style communication. 25,000 10 512 48.674 Here I check for an Origin header, which is required by the W3C 30,000 10 512 64.016 specifi cation when dealing with CORS-related issues: 35,000 10 512 79.972 for (int i = 0; i < Request.Headers.Count; i++) { 40,000 10 512 99.49 if (Request.Headers.GetKey(i).Equals("Origin")) { 45,000 10 512 122.777 containsOriginHeader = true; 50,000 10 512 137.434 break; } } To do this, I recommend building a proxy page in the same I then pass the Origin header on to the HttpWebRequest so the virtual directory, whose only function is to intercept the HTTP Comet server will receive it: message from the browser client, extract all the relevant headers if (containsOriginHeader) and content, and build another HTTP request object that connects { // cometRequest.Headers["Origin"] = Request.Headers["Origin"]; to the Comet service. Because this connection is done on the cometRequest.Headers.Set("Origin", Request.Headers["Origin"]); server, it isn’t impacted by CORS. Th us, through a proxy, you can } else keep a long-lived connection between your browser client and { the Comet service. Moreover, you can now transmit a single mes- cometRequest.Headers.Add("Origin", Request.Url.AbsoluteUri); } sage when it arrives on a message queue to all connected browser clients simultaneously. System.Diagnostics.Trace.WriteLineIf(_proxySwitch.TraceVerbose, "Adding Origin header."); First, I take the HTTP request and stream it into an array Next, I take the bytes from the content of the HTTP request of bytes so I can pass it to a new HTTP request object that I’ll from the jQuery client and write them to the request stream of instantiate shortly: the HttpWebRequest, which will be sent to the Comet server, as byte[] bytes; using (Stream reader = Request.GetBufferlessInputStream()) shown in Figure 12. { After forwarding the message to the Comet server, I call the bytes = new byte[reader.Length]; reader.Read(bytes, 0, (int)reader.Length); GetResponse method of the HttpWebRequest object, which pro- } vides an HttpWebResponse object that allows me to process the Next, I create a new HttpWebRequest object and point it to the server’s response. I also add the required HTTP headers that I’ll Comet server, whose URL I put in the web.confi g fi le so it can be send with the message back to the client: easily modifi ed later: try string newUrl = ConfigurationManager.AppSettings["CometServer"]; { HttpWebRequest cometRequest = (HttpWebRequest)HttpWebRequest.Create(newUrl); Response.ClearHeaders(); HttpWebResponse res = (HttpWebResponse)cometRequest.GetResponse(); for (int i = 0; i < res.Headers.Count; i++) 160 { string headerName = res.Headers.GetKey(i); // Response.Headers.Set(headerName, res.Headers[headerName]); 140 Response.AddHeader(headerName, res.Headers[headerName]); } 120 System.Diagnostics.Trace.WriteLineIf(_proxySwitch.TraceVerbose, 100 "Added headers."); I then wait for the server’s response: 80 Stream s = res.GetResponseStream(); 60 When I receive the Comet server’s message, I write it to the original HTTP request’s response stream so the client can receive 40 it, as shown in Figure 13. 20 Test the Application 0 To test your application, create a Web site to hold the sample 0 10,000 20,000 30,000 40,000 50,000 60,000 application pages. Make sure the URL to your Windows service is Figure 15 Response Times for Varying Numbers of Users for a correct and the message queue is properly confi gured and useable. 512-Byte Message Start the service and open the Comet client page in one browser 50 msdn magazine ASP.NET WINDOWS FORMS | WPF | ASP.NET FLOW TYPE LAYOUT REPORTING Reuse MS Word documents or templates as your reporting Integrate dynamic 2D and 3D templates. charting to your reports. Easy database connection with Create print-ready, digitally master-detail nested blocks. signed Adobe PDF and PDF/A documents. Powerful, programmable template designer with full &UHDWHÁRZW\SHOD\RXWVZLWK sources for Visual Studio®. tables, columns, images, headers and footers and more. US +1 877 - 462 - 4772 EU +49 421 - 4270671 - 0 WWW.TEXTCONTROL.COM Untitled-2 1 12/7/12 10:48 AM Figure 16 Testing with a Message Size of 1,024 Bytes I then created a utility that dumps a message of x number of bytes to the message queue, with the number of bytes set by a text fi eld Response Time (in Users Repetitions Milliseconds) on the main screen, and a text fi eld to set the number of millisec- onds to wait between messages sent from the server. I’ll use this 1,000 10 144.227 to send the test message back to the client. I then started the test 5,000 10 169.648 client, specifi ed the number of users plus the number of times the 10,000 10 233.031 client will reopen the Comet connection, and the threads opened 15,000 10 272.919 the connections against my server. I waited a few seconds for all the 20,000 10 279.701 connections to be opened, then went to the message-sending utility 25,000 10 220.209 and submitted a certain number of bytes. I repeated this for various 30,000 10 271.799 combinations of total users, total repetitions and message sizes. 35,000 10 230.114 Th e fi rst data sampling I took was for a single user with increasing 40,000 10 381.29 repetitions but with the response message a consistent (small) size 45,000 10 344.129 throughout the testing. As you can see in Figure 14, the number of repetitions doesn’t seem to have an impact on system performance 50,000 10 342.452 or reliability. Th e times are gradually increasing in a linear/constant manner, and the page to send messages in another. Type in a message which means the code on the Comet server is generally robust. and press send; aft er roughly 10 ms you should see the message Figure 15 graphs the number of users against the response time appear in the other browser window. Try this with various brows- for a 512-byte message. Figure 16 shows some statistics for a mes- ers—especially some of the older ones. As long as they support sage size of 1,024 bytes. Finally, Figure 17 shows the chart from the xmlHttpRequest object, it should work. Th is provides almost Figure 16 in graphical format.All of these tests were done on a real-time Web behavior (en.wikipedia.org/wiki/Real-time_web), where single laptop with 8GB of RAM and a 2.4 GHz Intel Core i3 CPU. content is pushed to the browser almost instantaneously without Th e numbers don’t show any particular trend, except that response requiring action from the user. times are reasonable, remaining at below one second for message Before any new application is deployed, you have to do perfor- sizes up to 1KB. I didn’t bother tracking bandwidth use because that’s mance and load testing. To do this, you should fi rst identify the aff ected by the message format. Also, because all testing was done metrics you want to gather. I suggest measuring usage load against on a single computer, network latency was eliminated as a factor. I both response times and data-transfer size. Additionally, you should could’ve tried it against my home network, but I didn’t think it would test usage scenarios that are relevant to Comet, in particular broad- be worthwhile because the public Internet is far more complex than my casting a single message to multiple clients without postback. wireless router and cable modem setup. However, because the key point To do the testing, I constructed a utility that opens multiple of Comet communication techniques is to reduce server round-trips threads, each with a connection to the Comet server, and waits by pushing content from the server as updated, theoretically half the until the server fires a response. This test utility allows me to network bandwidth usage should be reduced through Comet techniques. set a few parameters, such as the total number of users that will connect to my Comet server and the number of times they Wrapping Up reopen the connection (currently the connection is closed aft er I hope you can now successfully implement your own Comet-style the server’s response is sent). applications and use them eff ectively to reduce network bandwidth and increase Web site application performance. Of course, you’ll 450 want to check out the new technologies included with HTML5, bit.ly/UVMcBg 400 which can replace Comet, such as WebSockets ( ) and Server-Sent Events (SSE) (bit.ly/UVMhoD). Th ese technologies hold 350 the promise of providing a simpler way of pushing content to the 300 browser, but they do require the user to have a browser that sup- ports HTML5. If you still have to support users on older browsers, 250 Comet-style communication remains the best choice. Q 200 150 DERRICK LAU is an experienced soft ware development team leader with approxi- mately 15 years of relevant experience. He has worked in the IT shops of fi nancial 100 fi rms and the government, as well as in the soft ware development sections of technology-focused companies. He won the grand prize in an EMC development 50 contest in 2010 and came in as a fi nalist in 2011. He is also certifi ed as an MCSD and as an EMC content management developer. 0 0 10,000 20,000 30,000 40,000 50,000 60,000 THANKS to the following technical expert for reviewing this article: Figure 17 User Load vs Response Time for a 1KB Message Francis Cheung 52 msdn magazine ASP.NET SpreadsheetGear Performance Spreadsheet Components SpreadsheetGear 2012 Excel Reporting for ASP.NET, Now Available WinForms, WPF and Silverlight WPF and Silverlight controls, ĂƐŝůLJĐƌĞĂƚĞƌŝĐŚůLJĨŽƌŵĂƩĞĚ ŵƵůƟƚŚƌĞĂĚĞĚƌĞĐĂůĐ͕ϲϰŶĞǁ džĐĞůƌĞƉŽƌƚƐǁŝƚŚŽƵƚ džĐĞů NEW! džĐĞůĐŽŵƉĂƟďůĞĨƵŶĐƟŽŶƐ͕ƐĂǀĞ ĨƌŽŵĂŶLJ^W͘E d͕tŝŶĚŽǁƐ to XPS, improved ĞĸĐŝĞŶĐLJĂŶĚ Forms, WPF or Silverlight ƉĞƌĨŽƌŵĂŶĐĞ͕tŝŶĚŽǁƐϴƐƵƉƉŽƌƚ͕ ĂƉƉůŝĐĂƟŽŶƵƐŝŶŐƐƉƌĞĂĚƐŚĞĞƚ tŝŶĚŽǁƐ^ĞƌǀĞƌϮϬϭϮƐƵƉƉŽƌƚ͕sŝƐƵĂů^ƚƵĚŝŽ ƚĞĐŚŶŽůŽŐLJďƵŝůƚĨƌŽŵƚŚĞŐƌŽƵŶĚƵƉĨŽƌ 2012 support and more. ƉĞƌĨŽƌŵĂŶĐĞ͕ƐĐĂůĂďŝůŝƚLJĂŶĚƌĞůŝĂďŝůŝƚLJ͘ Excel Compatible Windows Forms, Excel Dashboards, Calculations, WPF and Silverlight Controls Charting and More ĚĚƉŽǁĞƌĨƵů džĐĞů zŽƵĂŶĚLJŽƵƌƵƐĞƌƐĐĂŶ ĐŽŵƉĂƟďůĞǀŝĞǁŝŶŐ͕ĞĚŝƟŶŐ͕ ĚĞƐŝŐŶĚĂƐŚďŽĂƌĚƐ͕ƌĞƉŽƌƚƐ͕ ĨŽƌŵĂƫŶŐ͕ĐĂůĐƵůĂƟŶŐ͕ charts, and models in Excel ĮůƚĞƌŝŶŐ͕ĐŚĂƌƟŶŐ͕ƉƌŝŶƟŶŐ or the SpreadsheetGear ĂŶĚŵŽƌĞƚŽLJŽƵƌtŝŶĚŽǁƐ tŽƌŬŬĞƐŝŐŶĞƌƌĂƚŚĞƌ Forms, WPF and ^ŝůǀĞƌůŝŐŚƚĂƉƉůŝĐĂƟŽŶƐǁŝƚŚ ƚŚĂŶŚĂƌĚƚŽůĞĂƌŶĚĞǀĞůŽƉĞƌƚŽŽůƐĂŶĚLJŽƵ ƚŚĞĞĂƐLJƚŽƵƐĞtŽƌŬŬsŝĞǁĐŽŶƚƌŽůƐ͘ ĐĂŶĞĂƐŝůLJĚĞƉůŽLJƚŚĞŵǁŝƚŚŽŶĞůŝŶĞŽĨĐŽĚĞ͘ ŽǁŶůŽĂĚ ŽƵƌ ĨƵůůLJ ĨƵŶĐƟŽŶĂů ϯϬͲĂLJ ĞǀĂůƵĂƟŽŶ ĂŶĚ ďƌŝŶŐ džĐĞů ZĞƉŽƌƟŶŐ͕ džĐĞů Free ĐŽŵƉĂƟďůĞĐŚĂƌƟŶŐ͕ džĐĞůĐŽŵƉĂƟďůĞĐĂůĐƵůĂƟŽŶƐĂŶĚŵƵĐŚŵŽƌĞƚŽLJŽƵƌ^W͘E d͕ 30 Day tŝŶĚŽǁƐ&ŽƌŵƐ͕tW&͕^ŝůǀĞƌůŝŐŚƚĂŶĚŽƚŚĞƌDŝĐƌŽƐŽŌ͘E d&ƌĂŵĞǁŽƌŬƐŽůƵƟŽŶƐ͘ Trial www.SpreadsheetGear.com SpreadsheetGear Toll Free USA (888) 774-3273 | Phone (913) 390-4797 | [email protected] Untitled-3 1 11/30/12 11:25 AM DATA CLUSTERING Detecting Abnormal Data Using k-Means Clustering James McCaffrey Consider the problem of identifying abnormal data items Aft er reading this article you’ll be able to experiment with k-means in a very large data set, for example, identifying potentially fraudu- clustering and have the base knowledge to add clustering function- lent credit-card transactions, risky loan applications and so on. One ality to a .NET application. approach to detecting abnormal data is to group the data items into Th e best way to get a feel for what k-means clustering is and to similar clusters and then seek data items within each cluster that see where I’m headed in this article is to take a look at Figure 1. Th e are diff erent in some sense from other data items within the cluster. demo program begins by creating a dummy set of 20 data items. In There are many different clustering algorithms. One of the clustering terminology, data items are sometimes called tuples. Each oldest and most widely used is the k-means algorithm. In this tuple here represents a person and has two numeric attribute values, article I’ll explain how the k-means algorithm works and present a height in inches and a weight in pounds. One of the limitations of a complete C# demo program. Th ere are many existing standalone the k-means algorithm is that it applies only in cases where the data data-clustering tools, so why would you want to create k-means tuples are completely numeric. clustering code from scratch? Existing clustering tools can be Th e dummy data is loaded into an array in memory. Next, the diffi cult or impossible to integrate into a soft ware system, they number of clusters is set to three. Although there are advanced might not be customizable to deal with unusual scenarios, and the clustering techniques that can suggest the optimal number of clus- tools might have copyright or other intellectual property issues. ters to use, in general data clustering is an exploratory process and the best number of clusters to use is typically found through trial and error. As you’ll see shortly, k-means clustering is an iterative This article discusses: process. The demo program has a variable maxCount, which is • The k-means clustering algorithm used to limit the number of times the main clustering loop will • Computing cluster centroids execute. Here that value is arbitrarily set to 30. • Euclidian distance Next, behind the scenes, the demo program uses the k-means • Looking for abnormal data algorithm to place each data tuple into one of three clusters. Th ere are many ways to encode a clustering. In this case, a clustering is Technologies discussed: defi ned by an array of int where the array index represents a tuple, C#, Visual Studio 2010 and the associated array value represents the 0-based cluster ID. Code download available at: So, in Figure 1, tuple 0 (65.0, 220.0) is assigned to cluster 0, tuple 1 (73.0, 160.0) is assigned to cluster 1, tuple 2 (59.0, 110.0) is assigned archive.msdn.microsoft.com/mag201302kmeans to cluster 2, tuple 3 (61.0, 120.0) is assigned to cluster 2 and so on. 54 msdn magazine Notice there are eight tuples assigned to cluster 0, five tuples Because the smallest of the three distances is the distance between assigned to cluster 1, and seven tuples assigned to cluster 2. the math average and tuple [c], the centroid of the three tuples is Next, the demo program displays the data, grouped by cluster. tuple [c]. You might wish to experiment with the demo program If you examine the clustered data you’ll see that cluster 0 might be by using diff erent defi nitions of the distance between two tuples called the heavy people cluster, cluster 1 might be called the tall to see how those aff ect the fi nal clustering produced. people cluster, and cluster 2 might be called the short people clus- With the notion of a cluster centroid established, the k-means ter. Th e demo program concludes by analyzing the tuples assigned algorithm is relatively simple. In pseudo-code: to cluster 0 and determines that by some criterion, tuple 5 (67.0, assign each tuple to a randomly selected cluster compute the centroid for each cluster 240.0) is the most abnormal tuple. loop until no improvement or until maxCount assign each tuple to best cluster In the sections that follow, I’ll walk you through the code that (the cluster with closest centroid to tuple) produced the screenshot in Figure 1 so that you’ll be able to update each cluster centroid (based on new cluster assignments) modify this code to meet your own needs. Th is article assumes end loop you have at least intermediate-level programming skill with a return clustering C-family language, but does not assume you know anything If you search the Web, you can fi nd several good online anima- about data clustering. I coded the demo program using C#, but tions of the k-means algorithm in action. Th e image in Figure 2 I used a non-OOP style so you shouldn’t have too much difficulty refactoring the demo to another language if you wish. I present all the source code for the demo pro- gram in this article. The source code is also available at archive.msdn.microsoft.com/mag201302kmeans. The k-Means Algorithm In principle, at least, the k-means algorithm is quite simple. But as you’ll see, some of the implementation details are a bit tricky. Th e central concept in the k-means algorithm is the centroid. In data clustering, the centroid of a set of data tuples is the one tuple that’s most represen- tative of the group. Th e idea is best explained by example. Suppose you have three height-weight tuples similar to those shown in Figure 1: [a] (61.0, 100.0) [b] (64.0, 150.0) [c] (70.0, 140.0) Which tuple is most representative? One approach is to compute a mathematical average (mean) tuple, and then select as the centroid the tuple that is closest to that aver- age tuple. So, in this case, the average tuple is: [m] = ((61.0 + 64.0 + 70.0) / 3, (100.0 + 150.0 + 140.0) / 3) = (195.0 / 3, 390.0 / 3) = (65.0, 130.0) And now, which of the three tuples is closest to (65.0, 130.0)? Th ere are several ways to defi ne closest. eTh most common approach, and the one used in the demo program, is to use the Euclidean distance. In words, the Euclidean distance between two tuples is the square root of the sum of the squared differences between each component of the tuples. Again, an example is the best way to explain. Th e Euclidean distance between tuple (61.0, 100.0) and the average tuple (65.0, 130.0) is: dist(m,a) = sqrt((65.0 - 61.0)^2 + (130.0 - 100.0)^2) = sqrt(4.0^2 + 30.0^2) = sqrt(16.0 + 900.0) = sqrt(916.0) = 30.27 Similarly: dist(m,b) = sqrt((65.0 - 64.0)^2 + (130.0 - 150.0)^2) = 20.02 dist(m,c) = sqrt((65.0 - 70.0)^2 + (130.0 - 140.0)^2) = 11.18 Figure 1 Clustering Using k-Means msdnmagazine.com February 2013 55 260 For simplicity I used a static method approach and removed all error-checking. Th e fi rst part of the demo code sets up the height 240 and weight data to be clustered. Because there are only 20 tuples, 220 I hardcoded the data and stored the data in memory in an array Cluster 0 named rawData. Typically, your data will be stored in a text fi le or 200 SQL table. In those cases you’ll have to write a helper function to 180 load the data into memory. If your data source is too large to fi t into machine memory, you’ll have to modify the demo code to iterate 160 Cluster 2 through an external data source rather than a data array. Weight (pounds) Weight 140 Cluster 1 120 In principle, at least, 100 80 the k-means algorithm is quite 55 60 65 70 75 80 Height (inches) simple. But as you’ll see, Figure 2 Clustered Data and Centroids some of the implementation shows the clustering produced by the demo program. Th e circled details are a bit tricky. data item in each cluster is the cluster centroid. Aft er setting up the raw data, the demo program calls helper Overall Program Structure function ShowMatrix to display the data. Next, variables num- Th e overall program structure for the demo shown in Figure 1, with Attributes, numClusters, and maxCount are assigned values of 2 a few minor edits, is listed in Figure 3. I used Visual Studio 2010 to (height and weight), 3 and 30, respectively. Recall maxCount limits create a new C# console application named ClusteringKMeans; any the number of iterations in the main algorithm processing loop. recent version of Visual Studio should work, too. In the Solution Th e k-means algorithm tends to converge quickly, but you might Explorer window I renamed fi le Program.cs to ClusteringKMeans- have to experiment a bit with the value of maxCount. Program.cs, which automatically renamed the template-generated All the clustering work is performed by method Cluster. class. I removed unneeded using statements at the top of the fi le. The method returns an int array that defines how each tuple is Figure 3 Overall Program Structure using System; rawData[19] = new double[] { 61.0, 130.0 }; namespace ClusteringKMeans { Console.WriteLine("\nRaw data:\n"); class ClusteringKMeansProgram ShowMatrix(rawData, rawData.Length, true); { static void Main(string[] args) int numAttributes = attributes.Length; { int numClusters = 3; try int maxCount = 30; { Console.WriteLine("\nBegin outlier data detection demo\n"); Console.WriteLine("\nk = " + numClusters + " and maxCount = " + maxCount); int[] clustering = Cluster(rawData, numClusters, numAttributes, maxCount); Console.WriteLine("Loading all (height-weight) data into memory"); Console.WriteLine("\nClustering complete"); string[] attributes = new string[] { "Height", "Weight" }; double[][] rawData = new double[20][]; Console.WriteLine("\nClustering in internal format: \n"); ShowVector(clustering, true); rawData[0] = new double[] { 65.0, 220.0 }; rawData[1] = new double[] { 73.0, 160.0 }; Console.WriteLine("\nClustered data:"); rawData[2] = new double[] { 59.0, 110.0 }; ShowClustering(rawData, numClusters, clustering, true); rawData[3] = new double[] { 61.0, 120.0 }; rawData[4] = new double[] { 75.0, 150.0 }; double[] outlier = Outlier(rawData, clustering, numClusters, 0); rawData[5] = new double[] { 67.0, 240.0 }; Console.WriteLine("Outlier for cluster 0 is:"); rawData[6] = new double[] { 68.0, 230.0 }; ShowVector(outlier, true); rawData[7] = new double[] { 70.0, 220.0 }; rawData[8] = new double[] { 62.0, 130.0 }; Console.WriteLine("\nEnd demo\n"); rawData[9] = new double[] { 66.0, 210.0 }; } rawData[10] = new double[] { 77.0, 190.0 }; catch (Exception ex) rawData[11] = new double[] { 75.0, 180.0 }; { rawData[12] = new double[] { 74.0, 170.0 }; Console.WriteLine(ex.Message); rawData[13] = new double[] { 70.0, 210.0 }; } rawData[14] = new double[] { 61.0, 110.0 }; } // Main rawData[15] = new double[] { 58.0, 100.0 }; rawData[16] = new double[] { 66.0, 230.0 }; // 14 short static method definitions here rawData[17] = new double[] { 59.0, 120.0 }; } rawData[18] = new double[] { 68.0, 210.0 }; } 56 msdn magazine Data Clustering Are your .NET apps slowing down? Are your .NET apps slowing down as you increase user activity or transaction load on them? If so then consider using NCache. NCache is an extremely fast and scalable in-memory distributed cache for .NET. Performance & Scalability thru Data Caching Cache app data, reduce expensive database trips, and scale your .NET apps. Performance: extremely fast in-memory cache Linear Scalability: just add servers and keep growing 100% uptime: self-healing dynamic cache cluster Mirrored, Replicated, Partitioned, and Client Cache topologies Use for Following in Web Farms ASP.NET Session Storage: Replicate sessions for reliability ASP.NET View State: Cache it to reduce payload sent to the browser ASP.NET Output Cache: Cache page output & improve response time NHibernate Level-2 Cache: Plug-in without any code change Entity Framework Cache: Plug-in without any code change Fast Runtime Data Sharing between Apps Powerful event notifications for pub/sub data sharing Continuous Query and group based events Download a 60-day FREE trial today! NCacheTM Distributed Cache for .NET & Java www.alachisoft.com 1-800-253-8195 Figure 4 Method UpdateMeans returning it. Because array means is assumed to exist, you might static void UpdateMeans(double[][] rawData, int[] clustering, want to make it a ref parameter. Array means is created using helper double[][] means) method Allocate: { static double[][] Allocate(int numClusters, int numAttributes) int numClusters = means.Length; { for (int k = 0; k < means.Length; ++k) double[][] result = new double[numClusters][]; for (int j = 0; j < means[k].Length; ++j) for (int k = 0; k < numClusters; ++k) means[k][j] = 0.0; result[k] = new double[numAttributes]; return result; int[] clusterCounts = new int[numClusters]; } for (int i = 0; i < rawData.Length; ++i) { Th e fi rst index in the means array represents a cluster ID and int cluster = clustering[i]; the second index indicates the attribute. For example, if means[0] ++clusterCounts[cluster]; [1] = 150.33 then the average of the weight (1) values of the tuples for (int j = 0; j < rawData[i].Length; ++j) in cluster 0 is 150.33. means[cluster][j] += rawData[i][j]; } Method UpdateMeans fi rst zeros out the existing values in array means, then iterates through each data tuple and tallies the count for (int k = 0; k < means.Length; ++k) for (int j = 0; j < means[k].Length; ++j) of tuples in each cluster and accumulates the sums for each attri- means[k][j] /= clusterCounts[k]; // danger bute, and then divides each accumulated sum by the appropriate return; cluster count. Notice that the method will throw an exception if } any cluster count is 0, so you might want to add an error-check. Method ComputeCentroid (listed in Figure 5) determines the assigned to one cluster. Aft er finishing, the demo program displays centroid values—the values of the one tuple that’s closest to the the encoded clustering and also displays the raw data, grouped average tuple values for a given cluster. according to cluster. Method ComputeCentroid iterates through each tuple in the data Th e demo program concludes by analyzing the clustered data set, skipping tuples that aren’t in the specifi ed cluster. For each tuple for outlier, possibly abnormal, tuples using method Outliers. Th at in the specifi ed cluster, the Euclidean distance between the tuple method accepts a cluster ID and returns the values of the data tuple and the cluster mean is calculated using helper method Distance. that’s the farthest (as measured by Euclidean distance) from Th e tuple values that are closest (having the smallest distance) to the cluster centroid (most representative tuple). In this case, for the mean values are stored and returned. cluster 0, the heavy person cluster, the outlier tuple is (67.0, 240.0), Method UpdateCentroids calls ComputeCentroid for each clus- the heaviest person. ter to give the centroids for all clusters: static void UpdateCentroids(double[][] rawData, int[] clustering, double[][] means, double[][] centroids) Computing Cluster Centroids { Recall that a cluster centroid is a tuple that is most representative for (int k = 0; k < centroids.Length; ++k) { of the tuples assigned to a cluster, and that one way to determine a double[] centroid = ComputeCentroid(rawData, clustering, k, means); cluster centroid is to compute a math average tuple and then fi nd centroids[k] = centroid; } the one tuple that’s closest to the average tuple. Helper method } UpdateMeans computes the math average tuple for each cluster Method UpdateCentroids assumes that an array of arrays named and is listed in Figure 4. centroids exists. Array centroids is very similar to array means: Th e Method UpdateMeans assumes that an array of arrays named fi rst index represents a cluster ID and the second index indicates means already exists, as opposed to creating the array and then the data attribute. Figure 5 Method ComputeCentroid Figure 6 Method Assign static double[] ComputeCentroid(double[][] rawData, int[] clustering, int cluster, double[][] means) static bool Assign(double[][] rawData, int[] clustering, double[][] centroids) { { int numAttributes = means[0].Length; int numClusters = centroids.Length; double[] centroid = new double[numAttributes]; bool changed = false; double minDist = double.MaxValue; for (int i = 0; i < rawData.Length; ++i) // walk thru each data tuple double[] distances = new double[numClusters]; { for (int i = 0; i < rawData.Length; ++i) int c = clustering[i]; { if (c != cluster) continue; for (int k = 0; k < numClusters; ++k) distances[k] = Distance(rawData[i], centroids[k]); double currDist = Distance(rawData[i], means[cluster]); if (currDist < minDist) int newCluster = MinIndex(distances); { if (newCluster != clustering[i]) minDist = currDist; { for (int j = 0; j < centroid.Length; ++j) changed = true; centroid[j] = rawData[i][j]; clustering[i] = newCluster; } } } } return centroid; return changed; } } 58 msdn magazine Data Clustering All these data sources at your fingertips – and that is just a start. RSSBus Data Providers [ADO.NET] Build cutting-edge .NET applications that connect to any data source with ease. t&BTJMZiEBUBCJOEwUPBQQMJDBUJPOT EBUBCBTFT BOETFSWJDFT VTJOHTUBOEBSE7JTVBM4UVEJPXJ[BSET t$PNQSFIFOTJWFTVQQPSUGPS$36% $SFBUF 3FBE 6QEBUF BOE%FMFUFPQFSBUJPOT t*OEVTUSZTUBOEBSE"%0/&5%BUB1SPWJEFS GVMMZ JOUFHSBUFEXJUI7JTVBM4UVEJP %BUBCJOEUPUIF8FC The RSSBus Data Providers give your .NET applications the power to databind (just like SQL) to Amazon, PayPal, eBay, QuickBooks, FedEx, Salesforce, MS-CRM, Twitter, Share- Point, Windows Azure, and much more! Leverage your existing knowledge to deliver cutting-edge WinForms, ASP.NET, and Windows Mobile solutions with full readwrite functionality quickly and easily. %BUBCJOEUP-PDBM"QQT The RSSBus Data Providers make everything look like a SQL table, even local application data. Using the RSSBus Data Providers your .NET applications interact with local applications, databases, and services in the same way you work with SQL Tables and Stored Procedures. No code required. It simply doesn’t get any easier! i%BUBCJOEUPBOZUIJOH KVTUMJLFZPVEPXJUI42-w Also available for: JDBC | ODBC | SQL SSIS | Excel | OData | SharePoint ... Download a free 30-day trial online at: www.rssbus.com © Copyright 2013 RSSBus Inc. All rights reserved. All trademarks and registered trademarks are the property of their respective owners. Untitled-1 1 1/11/13 1:55 PM Figure 7 The Cluster Method of the differences between each component. Because Euclidean static int[] Cluster(double[][] rawData, int numClusters, distance squares diff erences, larger diff erences are weighted much int numAttributes, int maxCount) more heavily than smaller diff erences. { bool changed = true; Another important factor related to the choice of distance func- int ct = 0; tion in the k-means clustering algorithm is data normalization. int numTuples = rawData.Length; Th e demo program uses raw, un-normalized data. Because tuple int[] clustering = InitClustering(numTuples, numClusters, 0); weights are typically values such as 160.0 and tuple heights are double[][] means = Allocate(numClusters, numAttributes); double[][] centroids = Allocate(numClusters, numAttributes); typically values like 67.0, diff erences in weights have much more UpdateMeans(rawData, clustering, means); infl uence than diff erences in heights. In many situations, in addi- UpdateCentroids(rawData, clustering, means, centroids); tion to exploring clustering on raw data, it’s useful to normalize the while (changed == true && ct < maxCount) raw data before clustering. Th ere are many ways to normalize data. { ++ct; A common technique is to compute the mean (m) and standard changed = Assign(rawData, clustering, centroids); deviation (sd) for each attribute, then for each attribute value (v) UpdateMeans(rawData, clustering, means); UpdateCentroids(rawData, clustering, means, centroids); compute a normalized value nv = (v-m)/sd. } return clustering; Assigning Each Tuple to a Cluster } With a method to compute the centroid of each cluster in hand, it’s possible to write a method to assign each tuple to a cluster. Method To summarize, each cluster has a centroid, which is the most Assign is listed in Figure 6. representative tuple in the cluster. Centroid values are computed Method Assign accepts an array of centroid values and iterates by fi nding the one tuple in each cluster that’s closest to the average through each data tuple. For each data tuple, the distance to each tuple (the mean) in each cluster. Each data tuple is assigned to the of the cluster centroids is computed and stored in a local array cluster whose cluster centroid is closest to the tuple. named distances, where the index of the array represents a cluster ID. Th en helper method MinIndex determines the index in array The Distance Function and Data Normalization distances that has the smallest distance value, which is the cluster Method ComputeCentroid calls a Distance method to determine ID of the cluster that has centroid closest to the tuple. which data tuple is closest to a cluster mean. As described earlier, Here’s helper method MinIndex: the most common way to measure distance from tuples to means static int MinIndex(double[] distances) { is to use Euclidean distance: int indexOfMin = 0; static double Distance(double[] tuple, double[] vector) double smallDist = distances[0]; { for (int k = 0; k < distances.Length; ++k) double sumSquaredDiffs = 0.0; { for (int j = 0; j < tuple.Length; ++j) if (distances[k] < smallDist) sumSquaredDiffs += Math.Pow((tuple[j] - vector[j]), 2); { return Math.Sqrt(sumSquaredDiffs); smallDist = distances[k]; indexOfMin = k; } } } You might want to consider alternative ways to defi ne distance. return indexOfMin; A very common option is to use the sum of the absolute values } In Assign, if the computed cluster ID is diff erent from the existing Figure 8 The Outlier Method cluster ID stored in array clustering, array clustering is updated and static double[] Outlier(double[][] rawData, int[] clustering, a Boolean fl ag to indicate that there has been at least one change in int numClusters, int cluster) the clustering is toggled. Th is fl ag will be used to determine when { int numAttributes = rawData[0].Length; to stop the main algorithm loop—when the maximum number of double[] outlier = new double[numAttributes]; iterations is exceeded or when there’s no change in the clustering. double maxDist = 0.0; This implementation of the k-means algorithm assumes that double[][] means = Allocate(numClusters, numAttributes); there’s always at least one data tuple assigned to each cluster. As given double[][] centroids = Allocate(numClusters, numAttributes); in Figure 6, method Assign does not prevent a situation where a UpdateMeans(rawData, clustering, means); UpdateCentroids(rawData, clustering, means, centroids); cluster has no tuples assigned. In practice, this usually isn’t a prob- lem. Preventing the error condition is a bit tricky. Th e approach I for (int i = 0; i < rawData.Length; ++i) { generally use is to create an array named centroidIndexes that works int c = clustering[i]; in conjunction with array centroids. Recall that array centroids if (c != cluster) continue; double dist = Distance(rawData[i], centroids[cluster]); holds centroid values, for example (61.0, 120.0) is the centroid for if (dist > maxDist) cluster 2 in Figure 2. Array centroidIndexes holds the associated { maxDist = dist; tuple index, for example [3]. Then in the Assign method, the Array.Copy(rawData[i], outlier, rawData[i].Length); fi rst step is to assign to each cluster the data tuple that holds the } } centroid values, and only then does the method iterate through return outlier; each remaining tuple and assign each to a cluster. Th is approach } guarantees that every cluster has at least one tuple. 60 msdn magazine Data Clustering HTML5+jQUERY Any App - Any Browser - Any Platform - Any Device Dowload Your Free Trial! www.infragistics.com/igniteui-trial Infragistics Sales US 800 231 8588 • Europe +44 (0) 800 298 9055 • India +91 80 4151 8042 • APAC +61 3 9982 4545 Copyright 1996-2013 Infragistics, Inc. All rights reserved. Infragistics and NetAdvantage are registered trademarks of Infragistics, Inc. The Infragistics logo is a trademark of Infragistics, Inc. All other trademarks or registered trademarks are the respective property of their owners. Untitled-2 1 12/5/12 10:49 AM The Cluster Method static void ShowMatrix(double[][] matrix) { Method Cluster, listed in Figure 7, is the high-level routine that for (int i = 0; i < numRows; ++i) calls all the helper and sub-helper methods to actually perform { Console.Write("[" + i.ToString().PadLeft(2) + "] "); the data clustering. for (int j = 0; j < matrix[i].Length; ++j) Th e main while loop repeatedly assigns each data tuple to a cluster, Console.Write(matrix[i][j].ToString("F1") + " "); Console.WriteLine(""); computes the new tuple means for each cluster, then uses the new } means to compute the new centroid values for each cluster. Th e loop } exits when there’s no change in cluster assignment or some maximum To display the clustering array you can use: count is reached. Because the means array is used only to compute static void ShowVector(int[] vector) { centroids, you might want to refactor Cluster by placing the call to for (int i = 0; i < vector.Length; ++i) UpdateMeans inside method UpdateCentroids. Console.Write(vector[i] + " "); Console.WriteLine(""); Before kicking the processing loop off , the clustering array is } initialized by method InitClustering: To display an outlier’s values you can use: static int[] InitClustering(int numTuples, int numClusters, int static void ShowVector(double[] vector) randomSeed) { { for (int i = 0; i < vector.Length; ++i) Random random = new Random(randomSeed); Console.Write(vector[i].ToString("F1") + " "); int[] clustering = new int[numTuples]; Console.WriteLine(""); for (int i = 0; i < numClusters; ++i) } clustering[i] = i; for (int i = numClusters; i < clustering.Length; ++i) And to display raw data grouped by cluster you can use: clustering[i] = random.Next(0, numClusters); static void ShowClustering(double[][] rawData, int numClusters, int[] clustering) return clustering; { } for (int k = 0; k < numClusters; ++k) // Each cluster The InitClustering method first assigns tuples 0 through { for (int i = 0; i < rawData.Length; ++i) // Each tuple num Clusters-1 to clusters 0 through numClusters-1, respectively, if (clustering[i] == k) so that every cluster will start with at least one tuple assigned. Th e { for (int j = 0; j < rawData[i].Length; ++j) remaining tuples are assigned to a randomly selected cluster. Console.Write(rawData[i][j].ToString("F1") + " "); A somewhat surprising amount of research has been done on Console.WriteLine(""); } k-means clustering initialization and you may want to experiment Console.WriteLine(""); with alternatives to the approach given here. In many cases, the } } fi nal clustering produced by the k-means algorithm depends on how the clustering is initialized. Wrapping Up Data clustering is closely related to and sometimes confused Looking for Abnormal Data with data classifi cation. Clustering is an unsupervised technique One way to use data clustering is to simply explore diff erent clus- that groups data items together without any foreknowledge terings and look for unexpected or surprising results. Another of what those groups might be. Clustering is typically an explor- possibility is to look for unusual data tuples within a cluster. Th e atory process. Classifi cation, in contrast, is a supervised technique demo program checks cluster 0 to fi nd the tuple in that cluster that’s that requires the specifi cation of known groups in training data, farthest from the cluster centroid using a method named Outlier, aft er which each data tuple is placed into one of these groups. which is listed in Figure 8. Classifi cation is typically used for prediction purposes. Aft er initializing means and centroids arrays, method Outlier The code and explanation presented in this article should iterates through each tuple in the specifi ed cluster and computes give you enough information to experiment with k-means data the Euclidean distance from the tuple to the cluster centroid, then clustering, or to create a fully customizable standalone clustering returns the values of the tuple that has the greatest distance to tool, or to add clustering features to a .NET application without the centroid values. A minor alternative for you to consider is to relying on any external dependencies. Th ere are many other clus- return the index of the farthest data tuple. tering algorithms in addition to k-means and I’ll present some of Th ere are many other ways you can examine clustered data for these in future MSDN Magazine articles, including data entropy abnormalities. For example, you might want to determine the minimization, category utility and Naive Bayes inference. Q average distance between each tuple and its assigned cluster cen- troid, or you might want to examine the distances of the cluster centroids from each other. DR. JAMES MCCAFFREY works for Volt Information Sciences Inc., where he manages technical training for soft ware engineers working at the Microsoft Redmond, Wash., campus. He has worked on several Microsoft products including Internet Display Routines Explorer and MSN Search. He’s the author of “.NET Test Automation Recipes” For the sake of completeness, here are some simplifi ed display rou- (Apress, 2006), and can be reached at jammc@microsoft .com. tines. Th e code download has slightly fancier versions. If you use these simplifi ed routines, you’ll have to modify their calls in the THANKS to the following technical expert for reviewing this article: Main method. To display raw data, means and centroids you can use: Darren Gehring 62 msdn magazine Data Clustering YOURR BAACCKSTAAGEE PASS TOO THE MICCROSSOFTT PLAATFORM Intense Take-Home Training for Developers, Software Architects and Designers SweetSS 127.0.0.1 Chicago! Visual Studio Live! is thrilled to be back in Chicago! Register for your backstage pass to the Microsoft Platform and join your fellow developers, software architects, designers and more for 4 days of unbiased training at Visual Studio Live! Chicago. Topics will include: ³ ASP.NET REGISTER TODAY ³ Azure / Cloud Computing AND SAVE $300 ³ Cross-Platform Mobile USE PROMO CODE CHFEB1 ³ Data Management ³ HTML5 / JavaScript ³ Windows 8 / WinRT visual studio live! chicago ³ WPF / Silverlight HILTON CHICAGO|MAy 13-16, 2013 ³ Visual Studio 2012 / .NET 4.5 vslive.com/chicago SUPPORTED BY PRODUCED BY magazine Untitled-8 1 1/2/13 1:33 PM STREAMINSIGHT Taming the Event Stream: Fast Approximate Counting Michael Meijer So you have a voluminous and potentially infi nite stream of second on commodity hardware. Th is article also shows how to events such as a clickstream, sensor data, credit-card transaction embed the data structure in a user-defined stream operator in data or Internet traffi c. It’s infeasible to store all events or analyze C# for the Microsoft streaming data processor, StreamInsight 2.1. them in multiple passes. Why not resort to a window of recent Intermediate programming skills are required to follow along, and events to simplify analysis? some experience with StreamInsight can come in handy. Suppose you want to count the number of interesting events in a large window covering the latest N events of the stream. A naïve A Tale of Counting approach to counting requires all N events to be in memory and a Before diving into StreamInsight, I’ll investigate the seemingly full iteration over them. As the window slides upon the arrival of trivial problem of counting. For simplicity, assume the stream has a new event, its oldest event expires and the new event is inserted. events with payloads of 0 or 1—uninteresting and interesting events, Counting over the new window from scratch wastes the processing respectively (regardless of what constitutes “interesting” in your time spent on N-2 events shared. Yuck! Th is article explains a data specifi c scenario). Th e number of 1s is counted over a (fi xed-size) structure to reduce memory space usage and processing time to a count-based window containing the most recent N events. Naïve small fraction of what would be required with that method, while counting takes O(N) time and space. supporting an event rate exceeding many thousands of events per As an astute reader, you probably came up with the idea of main- taining the count between consecutive windows and incrementing This article discusses: it for new 1s and decrementing it for expired 1s, sharing the N-2 • Approximate event counting basics events already processed. Good thinking! Maintaining the count • Using buckets for counting now takes O(1) time. However, should you decrement for an expired • Using StreamInsight for counting event or not? Unless you know the actual event, the count can’t be maintained. Unfortunately, to know the events until they have Technologies discussed: expired requires the entire window in memory—that is, it takes StreamInsight 2.1 O(N) space. Another strategy might be to fi lter out the uninter- Code download available at: esting events and count only the remaining interesting events. But that doesn’t reduce computational complexity and leaves you with archive.msdn.microsoft.com/mag201302StreamInsight a variable-size window. 64 msdn magazine Figure 1 The Exponential Histogram Class Outline To Bucket or Not to Bucket [DataContract] Here’s the bucket class: public class ExponentialHistogram [DataContract] { public class Bucket [DataMember] { private long n; [DataMember] private long timestamp; [DataMember] private double epsilon; [DataMember] private long count; [DataMember] private long total; public long Timestamp { get { return timestamp; } [DataMember] set { timestamp = value; } } private LinkedList |exact count – approximate count| ε ε requires all N events to be exact count ≤ , where 0 < < 1 Conceptually, the histogram stores events in buckets. Every bucket in memory and a full iteration initially covers one event, so it has a count of 1 and a timestamp of the event it covers. When an event arrives, expired buckets (cov- over them. ering expired events) are removed. A bucket is created only for an interesting event. As buckets are created over time, they’re merged Th e maintenance of the histogram is performed by this update method: public void Update(long timestamp, bool eventPayload) to save memory. Buckets are merged so they have exponentially { growing counts from the most recent to the last bucket, that is, 1, RemoveExpiredBuckets(timestamp); 1, ..., 2, 2, ..., 4, 4, ..., 8, 8 and so on. Th is way, the number of buckets // No new bucket required; done processing is logarithmic in the window size N. More precisely, it requires if (!eventPayload) return; O( ε 1 log N) time and space for maintenance. All but the last bucket cover only non-expired events. Th e last bucket covers at least one PrependNewBucket(timestamp); non-expired event. Its count must be estimated, which causes the MergeBuckets(); error in approximating the overall count. Hence, the last bucket } must be kept small enough to respect the relative error upper bound. It accepts a discrete timestamp, as opposed to wall-clock time, In the next section, the implementation of the exponential to determine what the latest N events are. Th is is used to fi nd and histogram in C# is discussed with a bare minimum of math. Read remove expired buckets. If the new event has a payload of 0 (false), the aforementioned paper for the intricate details. I’ll explain the processing stops. When the new event has a payload of 1 (true), a code and follow up with a pen-and-paper example. Th e histogram new bucket is created and prepended to the list of buckets. Th e real is a building block for the StreamInsight user-defined stream fi reworks are in merging the buckets. eTh methods called by the operator developed later in this article. update method are discussed in sequence. msdnmagazine.com February 2013 65 Here’s the code for the removal of buckets: A new bucket for the event with a payload of 1 (true) is created protected void RemoveExpiredBuckets(long timestamp) with a count of 1 and a timestamp equal to the current timestamp. { LinkedListNode // A bucket expires if its timestamp all bucket counts (total) is incremented. // is before or at the current timestamp - n Th e memory space-saving and error-bounding magic is in the while (node != null && node.Value.Timestamp <= timestamp - n) { merging of buckets. Th e code is listed inFigure 2. Buckets are total -= node.Value.Count; merged so that consecutive buckets have exponentially growing buckets.RemoveLast(); counts, that is, 1, 1, ..., 2, 2, ..., 4, 4, ..., 8, 8 and so on. Th e number node = buckets.Last; of buckets with the same count is determined by the choice of the } } relative error upper bound ε. Th e total number of buckets grows Th e traversal starts from the oldest (last) bucket and ends at the logarithmically with the size of the window n, which explains the fi rst non-expired bucket. Each bucket whose most recent event’s memory space savings. As many buckets as possible are merged, but timestamp is expired—that is, whose timestamp is no greater than the last bucket’s count is kept small enough (compared to the sum the current timestamp minus the window size—is removed from of the other bucket counts) to ensure the relative error is bounded. the list. Th is is where the discrete timestamp comes in. Th e sum More formally, buckets have non-decreasing counts from the fi rst (most recent) to the last (oldest) bucket in the list. eTh bucket of all bucket counts (total) is decremented by the count of each k counts are constrained to powers of two. Let k = ε1 and be an expired bucket. k 2 integer, or else replace the latter by . Except for the last bucket’s After expired events and buckets are accounted for, the new k 2 k count, let there be at least and at most + 1 buckets of the same event is processed: 2 k 2 protected void PrependNewBucket(long timestamp) count. Whenever there are 2 + 2 buckets of the same count, the { Bucket newBucket = new Bucket() oldest two are merged into one bucket with twice the count { of the oldest bucket and the most recent of their timestamps. Timestamp = timestamp, Count = 1 Whenever two buckets are merged, traversal continues from the }; merged bucket. Th e merge can cause a cascade of merges. Other- buckets.AddFirst(newBucket); wise traversal continues from the next bucket. total++; To get a feeling for the count approximation, look at the histo- } gram’s query method: public long Query() Figure 2 Merging Buckets in the Histogram { long last = buckets.Last != null ? buckets.Last.Value.Count : 0; protected void MergeBuckets() return (long)Math.Ceiling(total - last / 2.0); { } LinkedListNode buckets.Remove(previous); Suppose you have a newly initialized histogram with window size n = 7 and relative error upper bound ε = 0.5, so k = 2. Th e his- // A merged bucket can cause a cascade of merges due to // its new count, continue iteration from merged bucket togram develops as shown in Figure 3, and a schematic overview // otherwise the cascade might go unnoticed of this histogram is depicted in Figure 4. In Figure 3, merges are previous = current.Previous; } at timestamps 5, 7 and 9. A cascaded merge is at timestamp 9. An else expired bucket is at timestamp 13. I’ll go into more detail about this. { Th e fi rst event has no eff ect. At the fi fth event, a merge of the // No merge, continue iteration with next bucket k previous = current; oldest buckets occurs because there are + 2 buckets with the current = current.Next; 2 } same count of 1. Again, a merge happens at the seventh event. At } the ninth event, a merge cascades into another merge. Note that } aft er the seventh event, the fi rst event expires. No bucket carries 66 msdn magazine StreamInsight Figure 3 Example of the Exponential Histogram A Beauty Called Buckets (Timestamp, Count) Relative StreamInsight Timestamp Event Newest ´ … ´ Oldest Total Query Exact Error Perhaps you’re wondering what 1 0 0000 Microsoft StreamInsight is and where 2 1 (2,1) 1110 it fits in. This section provides some 31(3,1) ´ (2,1) 2220 basics. StreamInsight is a robust, high-performance, low-overhead, 4 0 (3,1) ´ (2,1) 2220 near-zero-latency and extremely fl ex- 5 (5,1) ´ (3,1) ´ (2,1) 1 3 2 3 0.333... ible engine for processing on streams. (merge) (5,1) ´ (3,2) It’s currently at version 2.1. The full 6 1 (6,1) ´ (5,1) ´ (3,2) 4 3 4 0.25 version requires a SQL Server license, 7 (7,1) ´ (6,1) ´ (5,1) ´ (3,2) though a trial version is available. It’s 1 5450.2 (merge) (7,1) ´ (6,2) ´ (3,2) run either as a stand-alone service or 81(8,1) ´ (7,1) ´ (6,2) ´ (3,2) 6560.166... embedded in-process. (9,1) ´ (8,1) ´ (7,1) ´ (6,2) ´ (3,2) At the heart of streaming data pro- 9 cessing is a model with temporal (merge) 1 (9,1) ´ (8,2) ´ (6,2) ´ (3,2) 7560.166... streams of events. Conceptually, it’s a (cascaded merge) (9,1) ´ (8,2) ´ (6,4) potentially infinite and voluminous 10 0 (9,1) ´ (8,2) ´ (6,4) 7550 collection of data arriving over time. 11 0 (9,1) ´ (8,2) ´ (6,4) 7550 Th ink of stock exchange prices, weather 12 0 (9,1) ´ (8,2) ´ (6,4) 7540.25 telemetry, power monitoring, Web 13 0 (9,1) ´ (8,2) 3 2 3 0.333... clicks, Internet traffi c, toll booths and so on. Each event in the stream has a header with metadata and a payload an expired timestamp until the 13th event. At the 13th event, the of data. In the header of the event, a timestamp is kept, at a mini- bucket with timestamp 6 no longer covers at least one non-expired mum. Events can arrive steadily, intermittently or perhaps in bursts event and thus expires. Note that the observed relative error is of up to many thousands per second. Events come in three fl avors: clearly less than the relative error upper bound. An event can be confi ned to a point in time; be valid for a certain interval; or be valid for an open-ended interval (edge). Besides events from the stream, a special punctuation event is issued by the The memory savings of the engine called the Common Time Increment (CTI). Events can’t be inserted into the stream with a timestamp less than the CTI’s time- histogram are huge compared stamp. Eff ectively, CTI events determine the extent to which events can arrive out of order. Th ankfully, StreamInsight takes care of this. to windows. Heterogeneous sources of input and sinks of output streams must somehow be adapted to fi t into this model. Th e events in the (query- In Figure 4, a dotted box is the window size at that point; it con- able) temporal streams are captured in an IQStreamable [Dat aContract] count—that’s fed into a sink. public class ApproximateCountUDSO : CepPointStreamOperator [DataMember] defy the memory-saving benefi ts of the exponential histogram. private long currentTimestamp; // Current (discrete) timestamp Why? Th e construct forces entire windows of events to be kept in public ApproximateCountUDSO(long n, double epsilon) memory. It’s not required by the exponential histogram, because it { has an equivalent implicit notion of windowing through the main- histogram = new ExponentialHistogram(n, epsilon); } tenance of buckets. Moreover, having an operator over windows is non-incremental, that is, with no processing of events as they public override IEnumerable public static IEnumerable yield return payload; variable-size windows, such as } } } time-based windows. Th is generates random payloads of 0s and 1s. Th e yield-return syntax turns it into an enumerable source. Put it in a utility class, if you will. To briefly recap, this article explains how to approximate the count over a window of events in logarithmic time and space with Figure 7 Embedding and Executing in StreamInsight upper-bounded error using an exponential histogram data structure. cla ss Program Th e histogram is embedded in a StreamInsight user-defi ned operator. { Th e histogram and operator can be extended to support variable- public const long N = 10000; public const double Epsilon = 0.05; size windows, such as time-based windows. This requires the histogram to know the window interval/timespan rather than the static void Main(string[] args) { window size. Buckets are expired when their timestamp is before using (Server server = Server.Create("StreamInsight21")) the new event’s timestamp minus the window timespan. An exten- { var app = server.CreateApplication("ApproximateCountDemo"); sion to compute variance is proposed in the paper, “Maintaining // Define an enumerable source Variance and k–Medians over Data Stream Windows,” by Brian var source = app.DefineEnumerable(() => Babcock, Mayur Datar, Rajeev Motwani and Liadan O’Callaghan Utility.EnumeratePayloads()); (stanford.io/UEUG0i). Apart from histograms, other so-called synop- // Wrap the source in a (temporal) point-in-time event stream sis structures are described in literature. You can think of random // The time settings determine when CTI events // are generated by StreamInsight samples, heavy hitters, quantiles and so on. var sourceStream = source.ToPointStreamable(e => Th e source code accompanying this article is written in C# 4.0 PointEvent.CreateInsert(DateTime.Now, e), AdvanceTimeSettings.IncreasingStartTime); with Visual Studio 2010 and requires StreamInsight 2.1. Th e code is free for use under the Microsoft Public License (Ms-PL). Note // Produces a stream of approximate counts // over the latest N events with relative error bound Epsilon that it was developed for educational purposes and was neither var query = optimized nor tested for production environments. Q from e in sourceStream.ApproximateCount(N, Epsilon) select e; // Unwrap the query's (temporal) point-in-time // stream to an enumerable sink MICHAEL MEIJER is as a soft ware engineer at CIMSOLUTIONS BV, where he var sink = query.ToEnumerable Naive Bayes Classifi cation with C# Naive Bayes classifi cation is a machine-learning technique that can of each dependent value (the attribute to be predicted, male or be used to predict to which category a particular data case belongs. female in this example) are computed. You can see that there are In this article I explain how Naive Bayes classifi cation works and 24 males and 16 females in the training data. present an example coded with the C# language. Th e demo program then has all the information needed to clas- Th ere are plenty of standalone tools available that can perform sify the sex of a new data case where the occupation is education, Naive Bayes classifi cation. However, these tools can be diffi cult or impossible to integrate directly into your application, and diffi cult to customize to meet specifi c needs. And they might have hid- den copyright issues. Th is article will give you a solid foundation for adding Naive Bayes classi- fi cation features to a .NET application, without relying on any external dependencies. Th e best way to understand what Naive Bayes classifi cation is and to see where I’m headed in the article is to examine the screenshot of a demo program in Figure 1. Th e demo program begins by generating 40 lines of data that will be used to train the classifi er. In most cases you’d be using an existing data source, but I generated dummy data to keep the demo simple. Th e fi rst line of data is “administrative,right,72.0,female.” Th e fi rst fi eld is an occupation, the second is hand dominance, the third is height in inches and the fourth is sex. Th e goal of the classifi er is to predict sex from a given set of values for occupation, dominance and height. Because the dependent variable sex has two possible values, this is an example of binary classifi cation. After generating raw data, the demo pro- gram converts each numeric height field to a category—short, medium or tall—by binning height. As I’ll explain, binning numeric data into categorical data is an approach that has pros and cons. Aft er the training data has been binned, the demo program scans the 40 lines of categorical data and computes joint counts. For example, the number of data cases where the person’s occupation is administrative and the person’s sex is male is 2. Additionally, the total numbers Code download available at archive.msdn.microsoft.com/mag201302TestRun. Figure 1 Naive Bayes Classifi cation Demo 70 msdn magazine Figure 2 Naive Bayes Program Structure using System; Console.WriteLine("Total female = " + dependentCounts[1]); namespace NaiveBayes { ShowJointCounts(jointCounts, attributeValues); class Program { string occupation = "education"; static Random ran = new Random(25); // Arbitrary string dominance = "right"; string height = "tall"; static void Main(string[] args) { bool withLaplacian = true; try { Console.WriteLine(" occupation = " + occupation); string[] attributes = new string[] { "occupation", "dominance", Console.WriteLine(" dominance = " + dominance); "height", "sex"}; Console.WriteLine(" height = " + height); string[][] attributeValues = new string[attributes.Length][]; int c = Classify(occupation, dominance, height, jointCounts, attributeValues[0] = new string[] { "administrative", dependentCounts, withLaplacian, 3); "construction", "education", "technology" }; if (c == 0) attributeValues[1] = new string[] { "left", "right" }; Console.WriteLine("\nData case is most likely male"); attributeValues[2] = new string[] { "short", "medium", "tall" }; else if (c == 1) attributeValues[3] = new string[] { "male", "female" }; Console.WriteLine("\nData case is most likely female"); double[][] numericAttributeBorders = new double[1][]; Console.WriteLine("\nEnd demo\n"); numericAttributeBorders[0] = new double[] { 64.0, 71.0 }; } catch (Exception ex) string[] data = MakeData(40); { for (int i = 0; i < 4; ++i) Console.WriteLine(ex.Message); Console.WriteLine(data[i]); } } // End Main string[] binnedData = BinData(data, attributeValues, numericAttributeBorders); // Methods to create data for (int i = 0; i < 4; ++i) // Method to bin data Console.WriteLine(binnedData[i]); // Method to compute joint counts // Helper method to compute partial probabilities int[][][] jointCounts = MakeJointCounts(binnedData, attributes, // Method to classify a data case attributeValues); int[] dependentCounts = MakeDependentCounts(jointCounts, 2); } // End class Program Console.WriteLine("Total male = " + dependentCounts[0]); } the dominance is right and the height is tall. In this example, it that explain the rather interesting mathematics behind Naive Bayes turns out the demo determined the probability that the data case is classifi cation, but the result is relatively simple. Symbolically: a male is 0.3855 and the probability that the case is female is 0.6145, P(male | X) = [ P(education | male) * P(right | male) * P(tall | male) * P(male) ] / and so the system concludes the data case is most likely a female. [ PP(male | X) + PP(female | X) ] In the sections that follow I’ll fi rst explain exactly how Naive Bayes classifi cation works, walk you through the code in the demo program, and describe how to modify the demo to meet your The term “naive” in Naive Bayes own needs. Th is article assumes you have at least beginning pro- gramming skills with a C-family language, but doesn’t assume you means that all X attributes are know anything about Naive Bayes classifi cation. Th e code for the demo program is a bit too long to present in its entirety here, but assumed to be mathematically the complete source is available from the MSDN download site at archive.msdn.microsoft.com/mag201302TestRun. independent, which greatly How Naive Bayes Classifi cation Works simplifi es classifi cation. Using the example shown in Figure 1, the goal is to predict the sex (male or female) of a person whose occupation is education, who Notice the equation is a fraction. The numerator, sometimes is right-handed and whose height is tall (greater than or equal to loosely called a partial probability, consists of four terms multi- 71.0 inches). To do this, we can compute the probability that the plied together. In this article I use the nonstandard notation of PP person is male given that information, and the probability the for a partial probability term. Th e denominator is the sum of two person is female given the information, and then predict the sex terms, one of which is the numerator. Th e fi rst piece to compute with the larger probability. Expressed symbolically, we want to is P(education | male), or the probability that a person’s occupa- know P(male | X), usually read as, “the probability of male given tion is education, given that he is male. Th is, as it turns out, can independent variable values X”) and P(female | X), where X is be estimated by the count of training cases where occupation is (education, right, tall). Th e term “naive” in Naive Bayes means that all education and sex is male, divided by the number of cases that are X attributes are assumed to be mathematically independent, which male (with any occupation), so: greatly simplifi es classifi cation. You can fi nd many online references P(education | male ) = count(education & male) / count(male) = 2/24 = 0.0833 msdnmagazine.com February 2013 71 Figure 3 Method BinData for Categorizing Height males and 16 females and estimate a probability of 24/40 = 0.6000 static string[] BinData(string[] data, string[][] attributeValues, for P(male). I prefer this approach, where priors are estimated double[][] numericAttributeBorders) using training data. { string[] result = new string[data.Length]; Now, if you refer to the earlier equation for P(male | X), you’ll string[] tokens; note that it contains the PP(female | X). Th e bottom sum, PP(male double heightAsDouble; string heightAsBinnedString; | X) + PP(female | X), is sometimes called the evidence. Th e pieces for PP(female | X) are computed like so: for (int i = 0; i < data.Length; ++i) P(education | female) = count(education & female) / count(female) = 4/16 = 0.2500 { P(right | female) = count(right & female) / count(female) = 14/16 = 0.8750 tokens = data[i].Split(','); P(tall | female) = count(tall & female) / count(female) = 2/16 = 0.1250 heightAsDouble = double.Parse(tokens[2]); P(female) = 16/40 = 0.4000 if (heightAsDouble <= numericAttributeBorders[0][0]) // Short heightAsBinnedString = attributeValues[2][0]; So the partial probability numerator for P(male | X) is: else if (heightAsDouble >= numericAttributeBorders[0][1]) // Tall PP(male | X) = 0.0833 * 0.7083 * 0.1667 * 0.6000 = 0.005903 heightAsBinnedString = attributeValues[2][2]; else Using the same logic, the partial probability for female given X heightAsBinnedString = attributeValues[2][1]; // Medium = (education, right, tall) is: string s = tokens[0] + "," + tokens[1] + "," + heightAsBinnedString + PP(female | X) = 0.2500 * 0.8750 * 0.1250 * 0.4000 = 0.010938 "," + tokens[3]; And, fi nally, the overall probabilities of male and female are: result[i] = s; P(male | X) = 0.005903 / (0.005903 + 0.010938) = 0.3505 } P(female | X) = 0.010938 / (0.005903 + 0.010938) = 0.6495 return result; } Th ese overall probabilities are sometimes called the posteriors. Because P(female | X) is greater than P(male | X), the system con- cludes the sex of the unknown person is female. But wait. Th ese two Using the same logic: P(right | male) = count(right & male) / count(male) = 17/24 = 0.7083 probabilities, 0.3505 and 0.6495, are close to but defi nitely not the P(tall | male) = count(tall & male) / count(male) = 4/24 = 0.1667 same as the two probabilities, 0.3855 and 0.6145, shown in Figure Th e next piece of the puzzle is P(male). In Naive Bayes termi- 1. Th e reason for this discrepancy is that the demo program uses nology, this is called a prior. Th ere’s some debate about how best to an important optional modifi cation of basic Naive Bayes called compute priors. On the one hand, we can hypothesize that there’s Laplacian smoothing. no reason to believe that the presence of males is more or less likely than the presence of females and so assign 0.5 to P(male). Laplacian Smoothing On the other hand, we can use the fact that the training data has 24 If you refer to Figure 1, you’ll see that the count of training cases in which the person has occupation = construction and sex = female is Figure 4 Method MakeJointCounts 0. In the demo, the X values are (education, right, tall), which doesn’t include construction. But suppose X had been (construction, right, static int[][][] MakeJointCounts(string[] binnedData, string[] attributes, string[][] attributeValues) tall). In the computation of PP(female | X) it would be necessary to { compute P(construction | female) = count(construction & female) int[][][] jointCounts = new int[attributes.Length - 1][][]; // -1 (no sex) / count(female), which would be 0, and which in turn would jointCounts[0] = new int[4][]; // 4 occupations jointCounts[1] = new int[2][]; // 2 dominances zero-out the entire partial probability. In short, it’s bad when a joint jointCounts[2] = new int[3][]; // 3 heights count is 0. Th e most common technique to avoid this situation is jointCounts[0][0] = new int[2]; // 2 sexes for administrative to simply add 1 to all joint counts. Th is has the feel of a hack but, jointCounts[0][1] = new int[2]; // construction in fact, has a solid mathematical basis. Th e technique is called add- jointCounts[0][2] = new int[2]; // education jointCounts[0][3] = new int[2]; // technology one smoothing, which is a specifi c kind of Laplacian smoothing. With Laplacian smoothing, if X = (education, right, tall) as in jointCounts[1][0] = new int[2]; // left jointCounts[1][1] = new int[2]; // right the previous section, P(male | X) and P(female | X) are calculated as follows: jointCounts[2][0] = new int[2]; // short jointCounts[2][1] = new int[2]; // medium P(education | male ) = jointCounts[2][2] = new int[2]; // tall count(education & male) + 1 / count(male) + 3 = 3/27 = 0.1111 P(right | male) = for (int i = 0; i < binnedData.Length; ++i) count(right & male) + 1 / count(male) + 3 = 18/27 = 0.6667 { P(tall | male) = string[] tokens = binnedData[i].Split(','); count(tall & male) + 1 / count(male) + 3 = 5/27 = 0.1852 P(male) = 24/40 = 0.6000 int occupationIndex = AttributeValueToIndex(0, tokens[0]); int dominanceIndex = AttributeValueToIndex(1, tokens[1]); P(education | female) = int heightIndex = AttributeValueToIndex(2, tokens[2]); count(education & female) + 1 / count(female) + 3 = 5/19 = 0.2632 int sexIndex = AttributeValueToIndex(3, tokens[3]); P(right | female) = count(right & female) + 1 / count(female) + 3 = 15/19 = 0.7895 ++jointCounts[0][occupationIndex][sexIndex]; P(tall | female) = ++jointCounts[1][dominanceIndex][sexIndex]; count(tall & female) + 1 / count(female) + 3 = 3/19 = 0.1579 ++jointCounts[2][heightIndex][sexIndex]; P(female) = 16/40 = 0.4000 } Th e partial probabilities are: return jointCounts; PP(male | X) = 0.1111 * 0.6667 * 0.1852 * 0.6000 = 0.008230 } PP(female | X) = 0.2632 * 0.7895 * 0.1579 * 0.4000 = 0.013121 72 msdn magazine Test Run Figure 5 Method Classify The demo program calls a helper method MakeData to gen- static int Classify(string occupation, string dominance, string height, erate somewhat random training data. MakeData calls helpers int[][][] jointCounts, int[] dependentCounts, bool withSmoothing, MakeSex, MakeOccupation, MakeDominance and MakeHeight. int xClasses) { For example, these helpers generate data so that male occupations double partProbMale = PartialProbability("male", occupation, dominance, are more likely to be construction and technology, male dominance height, jointCounts, dependentCounts, withSmoothing, xClasses); double partProbFemale = PartialProbability("female", occupation, dominance, is more likely to be right, and male height is most likely to be height, jointCounts, dependentCounts, withSmoothing, xClasses); between 66.0 and 72.0 inches. double evidence = partProbMale + partProbFemale; double probMale = partProbMale / evidence; Th e key methods called in Main are BinData to categorize height double probFemale = partProbFemale / evidence; data; MakeJointCounts to scan binned data and compute the joint if (probMale > probFemale) return 0; counts; MakeDependentCounts to compute total number of males else return 1; and females; and Classify, which uses joint counts and dependent } counts to perform a Naive Bayes classifi cation. And so the two fi nal probabilities are: P(male | X) = 0.008230 / (0.008230 + 0.013121) = 0.3855 Binning Data P(female | X) = 0.013121 / (0.008230 + 0.013121) = 0.6145 Method BinData is listed in Figure 3. Th e method accepts an array Th ese are the values shown in the screenshot in Figure 1. Notice of comma-delimited strings where each string looks like “educa- that 1 is added to each joint count but that 3 is added to denom- tion,left ,67.5,male.” In many situations, you’ll be reading training inators count(male) and count(female). Th e 3 is to some extent data from a text fi le where each line is a string. Th e method uses arbitrary in the sense that Laplacian smoothing doesn’t specify any String.Split to parse each string into tokens. Token[2] is the height. particular value to be used. In this case, it’s the number of X attri- It’s converted from a string into type double using the double.Parse butes (occupation, dominance, height). Th is is the most common method. Th e numeric height is compared against the boundary value to add to denominators of partial probabilities in Laplacian values until the height’s interval is found, and then the correspond- smoothing, but you may wish to experiment with other values. ing height category as a string is determined. A result string is Th e value to add to the denominator is oft en given the symbol stitched together using the old tokens, comma delimiters and the k in math literature on Naive Bayes. Also, notice that the priors, new computed-height category string. P(male) and P(female), are typically not modifi ed in Naive Bayes Laplacian smoothing. It’s not a requirement to bin Overall Program Structure Th e demo program shown running in Figure 1 is a single C# con- numeric data when performing sole application. Th e Main method, with some WriteLine statements removed, is listed in Figure 2. Naive Bayes classifi cation. Th e program begins by setting up the hardcoded X attributes occupation, dominance, and height, and the dependent attribute sex. In some situations you may prefer to scan your existing data It’s not a requirement to bin numeric data when performing source to determine the attributes, especially when the source is Naive Bayes classifi cation. Naive Bayes can deal with numeric data a data fi le with headers or a SQL table with column names. eTh directly, but those techniques are outside the scope of this article. demo program also specifi es the nine categorical X attribute values: Binning data has the advantages of simplicity and avoiding the need (administrative, construction, education, technology) for occupa- to make any particular explicit assumptions about the mathemati- tion; (left , right) for dominance; and (short, medium, tall) for height. cal distribution (such as Gaussian or Poisson) of the data. However, In this example there are two dependent variable attribute values: binning essentially loses information and does require you to (male, female) for sex. Again, you may want to programmatically determine and specify into how many categories to divide the data. determine attribute values by scanning your data. Th e demo sets up hardcoded boundary values of 64.0 and 71.0 Determining Joint Counts to bin the numeric height values so that values less than or equal Th e key to Naive Bayes classifi cation is computing joint counts. In to 64.0 are categorized as short; heights between 64.0 and 71.0 are the demo example, there are nine total independent X attribute medium; and heights greater than or equal to 71.0 are tall. When values (administrative, construction, … tall) and two dependent binning numeric data for Naive Bayes, the number of boundary val- attribute values (male, female), so a total of 9 * 2 = 18 joint counts ues will be one less than the number of categories. In this example, must be computed and stored. My preferred approach is to store the 64.0 and 71.0 were determined by scanning the training data for joint counts in a three-dimensional array int[][][] jointCounts. Th e minimum and maximum height values (57.0 and 78.0), computing fi rst index indicates the independent X attribute; the second index the diff erence, 21.0, and then computing interval size by dividing by indicates the independent X attribute value; and the third index number of height categories, 3, giving 7.0. In most situations, you’ll indicates the dependent attribute value. For example, jointCounts[0] want to the determine boundary values for numeric X attributes [3][1] means attribute 0 (occupation), attribute value 3 (technology) programmatically rather than manually. and sex 1 (female), or in other words the value at jointCounts[0][3] msdnmagazine.com February 2013 73 [1] is the count of training cases where occupation is technology Figure 6 Method PartialProbability and sex is female. Method MakeJointCounts is listed in Figure 4. static double PartialProbability(string sex, string occupation, string dominance, The implementation has many hardcoded values to make it string height, int[][][] jointCounts, int[] dependentCounts, bool withSmoothing, int xClasses) easier to understand. For example, these three statements could { be replaced by a single for loop that allocates space using Length int sexIndex = AttributeValueToIndex(3, sex); int occupationIndex = AttributeValueToIndex(0, occupation); properties in array attributeValues: int dominanceIndex = AttributeValueToIndex(1, dominance); jointCounts[0] = new int[4][]; // 4 occupations int heightIndex = AttributeValueToIndex(2, height); jointCounts[1] = new int[2][]; // 2 dominances jointCounts[2] = new int[3][]; // 3 heights int totalMale = dependentCounts[0]; Helper function AttributeValueToIndex accepts an attribute int totalFemale = dependentCounts[1]; int totalCases = totalMale + totalFemale; index and an attribute value string and returns the appropriate index. For example, AttributeValueToIndex(2, “medium”) returns int totalToUse = 0; if (sex == "male") totalToUse = totalMale; the index of “medium” in the height attribute, which is 1. else if (sex == "female") totalToUse = totalFemale; Th e demo program uses a method MakeDependentCounts to double p0 = (totalToUse * 1.0) / (totalCases); // Prob male or female determine the number of male and number of female data cases. double p1 = 0.0; Th ere are several ways to do this. If you refer to Figure 1, you’ll double p2 = 0.0; double p3 = 0.0; observe that one approach is to add the number of joint counts of any of the three attributes. For example, the number of males is if (withSmoothing == false) { the sum of count(administrative & male), count(construction & p1 = (jointCounts[0][occupationIndex][sexIndex] * 1.0) / totalToUse male), count(education & male) and count(technology & male): p2 = (jointCounts[1][dominanceIndex][sexIndex] * 1.0) / totalToUse; p3 = (jointCounts[2][heightIndex][sexIndex] * 1.0) / totalToUse; static int[] MakeDependentCounts(int[][][] jointCounts, } int numDependents) else if (withSmoothing == true) { { int[] result = new int[numDependents]; p1 = (jointCounts[0][occupationIndex][sexIndex] + 1) / for (int k = 0; k < numDependents; ++k) ((totalToUse + xClasses) * 1.0); // Male then female p2 = (jointCounts[1][dominanceIndex][sexIndex] + 1) / for (int j = 0; j < jointCounts[0].Length; ++j) ((totalToUse + xClasses) * 1.0 ; // Scanning attribute 0 p3 = (jointCounts[2][heightIndex][sexIndex] + 1) / result[k] += jointCounts[0][j][k]; ((totalToUse + xClasses) * 1.0); } return result; } //return p0 * p1 * p2 * p3; // Risky if any very small values return Math.Exp(Math.Log(p0) + Math.Log(p1) + Math.Log(p2) + Math.Log(p3)); Classifying a Data Case } Method Classify, shown in Figure 5, is short because it relies on helper methods. Notice that instead of returning the product of the four probabil- Method Classify accepts the jointCounts and dependentCounts ity pieces, the method returns the equivalent Exp of the sum of the arrays; a Boolean fi eld to indicate whether or not to use Laplacian Log of each piece. Using log probabilities is a standard technique in smoothing; and parameter xClasses, which in this example will machine-learning algorithms that’s used to prevent numeric errors be 3 because there are three independent variables (occupation, that can occur with very small real numeric values. dominance, height). Th is parameter could also be inferred from the jointCounts parameter. Wrapping Up Method Classify returns an int that represents the index of the Th e example presented here should give you a good foundation for predicted dependent variable. Instead, you might want to return adding Naive Bayes classifi cation features to your .NET applications. an array of probabilities for each dependent variable. Notice that Naive Bayes classifi cation is a relatively crude technique, but it does the classifi cation is based on probMale and probFemale, both of have several advantages over more-sophisticated alternatives such which are computed by dividing partial probabilities by the evi- as neural network classifi cation, logistic regression classifi cation dence value. You might want to simply omit the evidence term and and support vector machine classifi cation. Naive Bayes is simple, just compare the values of the partial probabilities by themselves. relatively easy to implement and scales well to very large data Method Classify returns the index of the dependent variable that sets. And Naive Bayes easily extends to multinomial classifi cation has the largest probability. An alternative is to supply a threshold problems—those with three or more dependent variables. Q value. For example, suppose probMale is 0.5001 and probFemale is 0.4999. You may wish to consider these values too close to call and return a classifi cation value representing “undetermined.” DR. JAMES MCCAFFREY works for Volt Information Sciences Inc., where he manages Method PartialProbability does most of the work for Classify technical training for soft ware engineers working at the Microsoft Redmond, Wash., campus. He has worked on several Microsoft products including Internet and is listed in Figure 6. Explorer and MSN Search. He’s the author of “.NET Test Automation Recipes” Method PartialProbability is mostly hardcoded for clarity. For (Apress, 2006), and can be reached at jammc@microsoft .com. example, there are four probability pieces, p0, p1, p2 and p3. You can make PartialProbability more general by using an array of probabilities THANKS to the following Microsoft technical expert for reviewing this article: where the size of the array is determined from the jointCounts array. Rich Caruana 74 msdn magazine Test Run YEARS OF IT EDUCATION ANNIVERSARY Celebrating 15 years of IT education, TechMentor returns in 2013 with immediately usable training that will keep you relevant in the workforce. Get inside the IT classroom and learn how you can build a more productive IT environment. Choose Your Campus REGISTRATION OPEN TECHMENTOREVENTS.COM SUPPORTED BY PRODUCED BY Untitled-1 1 1/11/13 10:02 AM THE WORKING PROGRAMMER TED NEWARD .NET Collections, Part 2: Working with C5 Welcome back. the time, the collection should be referenced by interface type. (For In the fi rst part of this series, I looked briefl y at the Copenhagen a description of the interface types, see the previous column in the Comprehensive Collection Classes for C# (C5) library, a set of classes series at msdn.microsoft.com/magazine/jj883961, or the C5 documentation designed to supplement (if not replace) the System.Collections at bit.ly/UcOcZH.) Here are the implementations: classes that ship with the Microsoft .NET Framework runtime • CircularQueue Implementations // Create a list view comprising post-WW2 presidents First of all, let’s take a quick look at the diff erent collection implemen- IList [TestMethod] switch it around to a diff erent implementation later, assuming you public void ViewExploration() follow the C5 design guideline of always referencing the collections { IList IList names.Insert(3, "Jackson"); the notion of “views”: subcollections of elements from the source Assert.AreEqual("Jackson", names[3]); collection that are, in fact, not copies but backed by the original Assert.AreEqual("Jackson", preWWII[3]); } collection. This was actually what the code from the previous column did, in the exploration test. See Figure 1 for how to create views on a collection. • TreeSet Assert.IsFalse(names.Contains("Neward")); Assert.IsTrue(names.Contains("Neward")); } } 78 msdn magazine The Working Programmer programming, but also because many developers fi nd immutable poorer than simply using a collection that supports events natively. objects easier to understand and reason about. Corollary to that All collections in C5 off er the ability to hang delegates off the col- concept, then, follows the concept of immutable collections—the lection, to be invoked when certain operations take place against idea that regardless of whether the objects inside the collection are the collection (see Figure 4). immutable, the collection itself is fi xed and unable to change (add Of course, the event handler can be written as a lambda; it’s just a or remove) the elements in the collection. (Note: You can see a pre- little more descriptive to show you the actual argument types. Th e view of immutable collections released on NuGet in the MSDN fi rst argument is—as is canon—the collection itself. Base Class Library (BCL) blog at bit.ly/12AXD78.) Just a NuGet Away No part of C5 couldn’t be built around the .NET FCL (aside All collections in C5 offer the from the emphasis on interfaces, which the FCL supports, but doesn’t really endorse that strongly, it seems), but the nice ability to hang delegates off the thing about C5 is that it’s done, it’s tested and it’s just a NuGet “Install-Package” away. collection. Happy coding! Q Within C5, immutable collections are handled by instantiating TED NEWARD is a principal with Neward & Associates LLC. He has written more “wrapper” collections around the collection containing the data of than 100 articles and authored and co-authored a dozen books, including “Profes- interest; these collections are “Guarded” collections and are used sional F# 2.0” (Wrox, 2010). He is an F# MVP and noted Java expert, and speaks in classic Decorator pattern style: at both Java and .NET conferences around the world. He consults and mentors regularly—reach him at [email protected] or [email protected] public void ViewImmutableExploration() { if you’re interested in having him come work with your team. He blogs at IList names.InsertFirst("Washington"); Assert.AreEqual("Washington", names[0]); } If anyone tries to write code that adds or removes elements from the list, C5 quickly disabuses said developer of the idea: An excep- tion is thrown as soon as any of the “modifying” methods (Add, Insert, InsertFirst and so on) are called. Th is off ers a pretty powerful opportunity, by the way. In the pre- vious column, I mentioned that one of the key design points that went into C5 is the idea that collections should only be used through interfaces. Assuming developers using C5 carry that design idea forward, it now becomes really simple to ensure that a collection is never modifi ed by a method to which it is passed (see Figure 3). Again, when the IWannaBePresidentToo method tries to modify the collection passed in (which, arguably, is bad design on the part of the programmer who wrote it, but unfortunately there’s a lot of that kind of code out there), an exception is thrown. By the way, should you prefer that the collection not throw an exception and instead silently fail the modifi cation (which I think is too subtle, but some developers may need that functionality), it’s relatively easy to put together your own version of Guarded- Array Create Windows Store Apps with HTML5 and JavaScript Not only has Bill Gates’ dream of a computer on every desk and is because the Windows Runtime (WinRT) is a platform that in every home come to fruition, but the advent of devices such as contains APIs that sit on top of the Windows core services, as the Surface tablet has taken his dream even further. In addition illustrated in Figure 1. to the Surface, there has been an explosion of new consumer- Th e WinRT APIs give you access to everything Windows 8 has oriented devices in every form factor possible. In other words, to off er, including APIs for hardware such as built-in webcams, computers are everywhere. geolocation, light sensors and accelerometers. Of course, platform Consider that on those computers are more than 1 billion existing fundamentals such as memory management, authentication, glo- Windows installations worldwide, with 300 million Windows 7 balization and asynchronous processing—as well as Windows Store licenses sold each year in the previous two years. Combine the app features such as search, share and communications—are also current, upgradable Windows installation base with the rapidly readily available. Th ere are even APIs for manipulating audio and growing market of Windows 8 devices such as the Surface and video; however, if you’re writing JavaScript apps, HTML5 Windows Core OS Services Code download available at archive.msdn.microsoft.com/ mag201302ModernApps. Figure 1 Architecture of Windows Store Apps 80 msdn magazine Users expect consistency between apps and between apps and the Aft er you create a new JavaScript Grid project template, examining OS. Windows Store apps are all about UX, and employing design its structure reveals a project full of standard Web fi le types—such principles—such as using a consistent font, the Windows UI as .html, .css and .js fi les—organized in folders under the project’s silhouette and a scalable grid system—enhance the user’s comfort root. You can then debug and run a Windows Store app by pressing level when using your app. Th is consistency is carried throughout F5 or selecting Start Debugging from the Debug menu. both apps and Windows 8 itself. In the Windows Store app templates, default.html is the starting Ensuring both touch and mouse input work reliably and page for a Windows Store app and has a companion script file, consistently is important because users now have more ways to /js/default.js, which contains basic process lifecycle management interact with their computing devices in the form of mice, pens, code. As with any other HTML fi le, default.html has code you might touch, cameras and sensors. expect, including script references and new HTML5 semantic markup that defi nes the page structure. Th e following code frag- ment lives inside the default.html In the data.js fi le you can fi nd the only // TODO comment near {"key":"6","group":{"key":"group2","title":"Holidays"},"title":"New Year's Day", the beginning of the fi le. Replace the code under the comment with "eventDate":"1/1/2013","image":"/images/celebrate.png","color":"#f8baba"}] your own, so the code looks something like the following code Now that you’ve loaded the data, you need to ensure the List snippet, which makes an XMLHttpRequest call to retrieve JSON object is bound to the correct HTML elements. Modifi cations to data, then also creates the data set, including dynamic properties the /pages/groupedItems/groupedItems.html page in Figure 2 such as the daysToGo and message fi elds: shows data binding in action. var list = new WinJS.Binding.List(); Each HTML element in Figure 2 that contains a data-win-bind ... WinJS.xhr({ url: "data.json" }).then(function (xhr) { attribute has a binding expression that matches a property name of var items = JSON.parse(xhr.responseText); the item variable from the preceding code snippet, so all you need items.forEach(function (item) { item.daysToGo = Math.floor( to do is make sure that the binding expressions match the names (Date.parse(item.eventDate) - new Date()) / 86400000); of the fi elds. Don’t forget to ensure that you also modify the bind- item.message = item.daysToGo + " days until " + item.title; if (item.daysToGo >= 0) { ing expressions in the groupedDetail.html and itemDetails.html list.push(item); pages so correct data will show when a user navigates to them. }; }) Running the project in the Windows Simulator yields results }) similar to that in Figure 3. (You can learn more about using the In the beginning of data.js is a line of code that instantiates a simulator at bit.ly/M1nWOY.) WinJS.Binding.List object aptly named list, and the preceding code As you can see, you can simply replace the code from the Visual pushes individual items onto this List. Studio template for quick data access. However, projects are oft en Th e List object enables binding between JSON data or JavaScript quite large or complex, making maintenance diffi cult. If this is the arrays and HTML elements. Once the list variable is populated with case, then use the Model-View-ViewModel (MVVM) pattern to make maintenance easier. This pattern is extremely well-documented on the Web. While your app now works, it’s time take advantage of the many great Windows 8 features that can make your app stand out in the crowd. Branding Your Windows Store App Considering that the focal point of Windows 8 is the Start page, it makes sense to start branding there. Th e Start page is fi lled with live tiles, and they aren’t just a bunch of square icons, either. Instead, they’re the best way to show off and attract users to your app. Live tiles are called “live” for a reason, and that’s because you can dynamically display information and images in them, making your app even more attractive. Windows Store apps require three separate tile Figure 3 Replace the Sample Data to Make a Basic App images with the following pixel dimensions: 82 msdn magazine Modern Apps Because snapped-view support is built into the Grid template, you need to verify that the data displays nicely while snapped, showing pertinent and readable information. Th e AppBar also works while the app is snapped, so that also might need style adjustments. Semantic Zoom Th is new touch-friendly feature of Windows 8 is a way to aggregate large amounts of data in a single, easy-to-digest view. Users invoke Semantic Zoom in the following ways: Figure 4 Countdown App Standard and Wide Tiles • Touch: Pinch gesture • Mouse: Ctrl+Scroll Wheel • Logo: 150 x 150 (standard tile) • Keyboard: Ctrl - and Ctrl + • Wide Logo: 150 x 310 (wide tile) Semantic Zoom is more about visualizing data in a meaningful • Small Logo: 30 x 30 (this shows only in app lists in the store) way that assists with navigation than simply exposing a zoomed view The images can be any popular image format, and those with of it. If there’s a lot of data, it’s better for the user to have a bird’s-eye transparent backgrounds work best. Opening the package.appx- view rather than having to scroll through an overload of information. manifest fi le from the project’s root reveals the confi guration palette, Consider how to best present the data so it’s the most meaningful. where you can select the tile images and set the background colors. Search and Share Searching and sharing data between apps are Figure 4 illustrates both a standard and wide tile. core aspects of modern apps. Users can now search across multiple When you’re setting up the tiles is a good time to confi gure the apps at one time and then share the data they fi nd. Or your app can splash screen by selecting just an image and background color— register itself as a share target and accept data that users share from no code is used. Although tiles and splash screens are important other Windows Store apps. Never before has app-to-app commu- factors in branding your app, you can do many more things to nication been so straightforward and consistent. brand and polish your app, which you can read about at bit.ly/M4HYmL. Picker Controls Th ese are traditional Windows controls that have been updated for a modern UI—such as the File Open Picker Windows 8 ‘Must-Have’ Features for Your App or File Save Picker—or print settings dialogs that have been staples While your app might work at this point, there are many new of Windows apps for many versions. features and APIs in the Windows 8 app ecosystem that you can Media Because Windows Store apps built with JavaScript fully tap into to really make your app stand out. I’ll briefl y discuss each. support HTML5, the
"); getResponse(); AJAX Request #1 Long-Lived HTTP return; Request Held } in games. How long until...
• Navigation: A template with a Blank project structure