DOCSLIB.ORG

Investigating the Feasibility of an MPI-Like Library Implemented in .Net Using Only Fully Managed Code

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Investigating the Feasibility of an MPI-Like Library Implemented in .Net Using Only Fully Managed Code Investigating the Feasibility of an MPI-like Library Implemented in .Net Using Only Fully Managed Code Daniel Holmes MSc in High Performance Computing The University of Edinburgh Year of Presentation: 2007 Abstract The .Net development platform and the C# language, in particular, offer many benefits to programmers including increased productivity, security, reliability and robustness, as well as standards-based application portability and cross-language inter-operation. The Message Passing Interface (MPI) is a standardised high performance computing paradigm with efficient, frequently-used implementations in many popular languages. A partial implementation of McMPI, the first MPI-like library to be targeted at .Net and written in pure C#, is presented. It is sufficiently complete to demonstrate typical application code and to evaluate relative performance. Although the effective bandwidth for large messages (over 100 Kbytes) using 100Mbit/s Ethernet is good, the overheads introduced by .Net remoting and object serialisation are shown to result in high latency and to limit bandwidth to 166Mbit/s when using a 1Gbit/s Ethernet interconnection. A possible resolution that still uses pure C#, i.e. using .Net sockets, is proposed but not implemented. Contents Chapter 1. Introduction ..................................................................................... 1 Chapter 2. Background ..................................................................................... 2 2.1 Object-Oriented HPC .................................................................................. 2 2.2 Comparing .Net and Java ............................................................................ 3 2.2.1 Portability ............................................................................................... 5 2.2.2 Connectivity ........................................................................................... 5 2.2.3 Software Engineering ............................................................................. 6 2.2.4 Security .................................................................................................. 6 2.2.5 Other Benefits ........................................................................................ 8 2.2.6 Numerical Issues .................................................................................... 8 2.2.7 Performance Issues ................................................................................. 8 2.3 Programming in .Net ................................................................................... 9 2.4 HPC and .Net Programming...................................................................... 11 2.5 Implementation Paradigms for MPI in C# ................................................. 12 2.5.1 Platform Invoke – MPI.NET ................................................................ 13 2.5.2 Sockets vs. Remoting ........................................................................... 14 Chapter 3. Requirements Analysis for a Solution in C# ............................... 16 3.1 Headers and Message Data........................................................................ 16 3.2 Queues and Progress ................................................................................. 17 3.3 Threads and Locks .................................................................................... 19 3.4 External Interface ...................................................................................... 20 3.4.1 Data types ............................................................................................. 20 3.4.2 Memory References ............................................................................. 21 3.4.3 Layout .................................................................................................. 23 3.4.4 Process Identifiers ................................................................................ 23 Chapter 4. Designing and Building a Solution in C# ..................................... 24 4.1 Object Interaction ...................................................................................... 24 4.1.1 Non-Blocking Send .............................................................................. 26 i 4.1.2 Non-Blocking Receive ......................................................................... 27 4.2 Class Structure........................................................................................... 28 4.2.1 The Internal Storage Structures ............................................................ 29 4.2.2 The RemoteController Class ................................................................ 30 4.2.3 The Process Class ................................................................................ 32 4.2.4 The Communicator Class ..................................................................... 33 4.2.5 The Request and Status Classes ............................................................ 34 4.3 Configuration, Start up and Shutdown....................................................... 35 4.3.1 The Node Manager Utility.................................................................... 35 4.3.2 The Initialise Method ........................................................................... 36 4.3.3 The Finalise Method ............................................................................. 36 4.4 Summary ................................................................................................... 36 Chapter 5. Testing............................................................................................ 37 5.1 Proving Correctness using Ping-Pong Duplex ........................................... 37 5.2 Investigating Performance using Ping-Pong Simplex ................................ 38 5.3 Discussion of Ping Pong Simplex Results ................................................. 42 5.4 Demonstrating Real-World Code using Image Processor.......................... 43 5.5 Discussion of Image Processor Results ..................................................... 49 Chapter 6. Further Work ................................................................................ 50 6.1 Reducing Latency...................................................................................... 50 6.2 Testing Support for InfiniBand and Myrinet ............................................. 51 6.3 Increasing Portability ................................................................................. 52 6.4 Scaling to Large Parallel Systems.............................................................. 53 6.5 Completing McMPI .................................................................................. 53 6.6 Building Remoting from MPI ................................................................... 53 Chapter 7. Conclusion ..................................................................................... 54 Appendix A. Project Plan .................................................................................... A Appendix B. Data tables ...................................................................................... C Appendix C. References & Bibliography ........................................................... N Appendix D. Glossary .......................................................................................... R ii List of Figures Figure 1: Building and Executing Java Programs ...................................................... 3 Figure 2: Building and Executing .Net Programs ...................................................... 4 Figure 3: Definition of Type-safe and Verifiably Type-safe in .Net (25) ................... 6 Figure 4: Simplified View of Code Access Security in .Net (26) .............................. 7 Figure 5: Remoting infrastructure overview (33) .................................................... 15 Figure 6: Source code for using the IsSerializable property .................................... 21 Figure 7: UML Sequence diagram for non-blocking send and receive .................... 25 Figure 8: UML Static Structure for the public interface of the McMPI classes ....... 28 Figure 9: Source code for the internal storage structures ......................................... 29 Figure 10: Source code for the PullMessageData method ...................................... 30 Figure 11: Source code for the PushSendHeader method ....................................... 31 Figure 12: Source code for the CombineMatched and PullData functions .............. 32 Figure 13: Ping Pong round-trip time for small messages – 100Mbit/s ................... 39 Figure 14: Ping Pong round-trip time for small messages – 1Gbit/s ........................ 39 Figure 15: Ping Pong round-trip time for all message sizes – 100Mbit/s ................. 40 Figure 16: Ping Pong round-trip times for all message sizes – 1Gbit/s .................... 40 Figure 17: Ping Pong effective bandwidth – 100Mbit/s .......................................... 41 Figure 18: Ping Pong effective bandwidth – 1Gbit/s ............................................... 41 Figure 19: Image Processor times – small image, single precision .......................... 45 Figure 20: Image Processor times – small image, double precision ......................... 45 Figure 21: Image Processor times – medium image, single precision...................... 46 Figure 22: Image Processor times – medium image, double precision .................... 46 iii Figure 23: Image Processor times – large image, single
Load more

Recommended publications
  • NET Framework
    Advanced Windows Programming .NET Framework based on: A. Troelsen, Pro C# 2005 and .NET 2.0 Platform, 3rd Ed., 2005, Apress J. Richter, Applied .NET Frameworks Programming, 2002, MS Press D. Watkins et al., Programming in the .NET Environment, 2002, Addison Wesley T. Thai, H. Lam, .NET Framework Essentials, 2001, O’Reilly D. Beyer, C# COM+ Programming, M&T Books, 2001, chapter 1 Krzysztof Mossakowski Faculty of Mathematics and Information Science http://www.mini.pw.edu.pl/~mossakow Advanced Windows Programming .NET Framework - 2 Contents The most important features of .NET Assemblies Metadata Common Type System Common Intermediate Language Common Language Runtime Deploying .NET Runtime Garbage Collection Serialization Krzysztof Mossakowski Faculty of Mathematics and Information Science http://www.mini.pw.edu.pl/~mossakow Advanced Windows Programming .NET Framework - 3 .NET Benefits In comparison with previous Microsoft’s technologies: Consistent programming model – common OO programming model Simplified programming model – no error codes, GUIDs, IUnknown, etc. Run once, run always – no "DLL hell" Simplified deployment – easy to use installation projects Wide platform reach Programming language integration Simplified code reuse Automatic memory management (garbage collection) Type-safe verification Rich debugging support – CLR debugging, language independent Consistent method failure paradigm – exceptions Security – code access security Interoperability – using existing COM components, calling Win32 functions Krzysztof
    [Show full text]
  • Adding Self-Healing Capabilities to the Common Language Runtime
    Adding Self-healing capabilities to the Common Language Runtime Rean Griffith Gail Kaiser Columbia University Columbia University [email protected] [email protected] Abstract systems can leverage to maintain high system availability is to perform repairs in a degraded mode of operation[23, 10]. Self-healing systems require that repair mechanisms are Conceptually, a self-managing system is composed of available to resolve problems that arise while the system ex- four (4) key capabilities [12]; Monitoring to collect data ecutes. Managed execution environments such as the Com- about its execution and operating environment, performing mon Language Runtime (CLR) and Java Virtual Machine Analysis over the data collected from monitoring, Planning (JVM) provide a number of application services (applica- an appropriate course of action and Executing the plan. tion isolation, security sandboxing, garbage collection and Each of the four functions participating in the Monitor- structured exception handling) which are geared primar- Analyze-Plan-Execute (MAPE) loop consumes and pro- ily at making managed applications more robust. How- duces knowledgewhich is integral to the correct functioning ever, none of these services directly enables applications of the system. Over its execution lifetime the system builds to perform repairs or consistency checks of their compo- and refines a knowledge-base of its behavior and environ- nents. From a design and implementation standpoint, the ment. Information in the knowledge-base could include preferred way to enable repair in a self-healing system is patterns of resource utilization and a “scorecard” tracking to use an externalized repair/adaptation architecture rather the success of applying specific repair actions to detected or than hardwiring adaptation logic inside the system where it predicted problems.
    [Show full text]
  • Python Guide Documentation 0.0.1
    Python Guide Documentation 0.0.1 Kenneth Reitz 2015 11 07 Contents 1 3 1.1......................................................3 1.2 Python..................................................5 1.3 Mac OS XPython.............................................5 1.4 WindowsPython.............................................6 1.5 LinuxPython...............................................8 2 9 2.1......................................................9 2.2...................................................... 15 2.3...................................................... 24 2.4...................................................... 25 2.5...................................................... 27 2.6 Logging.................................................. 31 2.7...................................................... 34 2.8...................................................... 37 3 / 39 3.1...................................................... 39 3.2 Web................................................... 40 3.3 HTML.................................................. 47 3.4...................................................... 48 3.5 GUI.................................................... 49 3.6...................................................... 51 3.7...................................................... 52 3.8...................................................... 53 3.9...................................................... 58 3.10...................................................... 59 3.11...................................................... 62
    [Show full text]
  • ARCHIVED: Developing and Deploying .NET Applications On
    Developing and Deploying .NET Applications on AWS July 2020 This version has been archived. For the most recent version, see https://docs.aws.amazon.com/whitepapers/latest/develop-deploy-dotnet- apps-on-aws/develop-deploy-dotnet-apps-on-aws.html Archived Notices Customers are responsible for making their own independent assessment of the information in this document. This document: (a) is for informational purposes only, (b) represents current AWS product offerings and practices, which are subject to change without notice, and (c) does not create any commitments or assurances from AWS and its affiliates, suppliers or licensors. AWS products or services are provided “as is” without warranties, representations, or conditions of any kind, whether express or implied. The responsibilities and liabilities of AWS to its customers are controlled by AWS agreements, and this document is not part of, nor does it modify, any agreement between AWS and its customers. © 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved. Archived 2 Contents Abstract ................................................................................................................................ 5 Introduction .......................................................................................................................... 6 Working with Different Variants of .NET .......................................................................... 6 Running .NET Applications in the AWS Cloud ................................................................
    [Show full text]
  • Copyrighted Material
    Preface . xi Acknowledgments . xiv Introduction . xv Part I: Silverlight Basics 1 Chapter 1: Getting to Know Silverlight . .3 What Is Silverlight? ...............................................................................................................3 Why Use Silverlight? .............................................................................................................4 Comparing Silverlight 1.0 and 2 ...........................................................................................5 Silverlight Framework Architecture .......................................................................................7 The Silverlight plug-in.................................................................................................7 The Silverlight core presentation framework ...............................................................8 The .NET Silverlight framework ..................................................................................9 Silverlight application architecture ............................................................................10 Summary ............................................................................................................................11 Chapter 2: Getting Started with Silverlight . 13 Implementing the Silverlight Development Environment ....................................................13 Installing Visual Studio 2008 ....................................................................................14 Installing the Silverlight plug-in ................................................................................14
    [Show full text]
  • Comparing Java and .NET Security
    Comparing Java and .NET Security: Lessons Learned and Missed Nathanael Paul David Evans University of Virginia Department of Computer Science [nate, evans ]@cs.virginia.edu Abstract Many systems execute untrusted programs in virtual machines (VMs) to mediate access to system resources. Sun introduced the Java VM in 1995, primarily intended as a lightweight platform for executing untrusted code inside web pages. More recently, Microsoft developed the .NET platform with similar goals. Both platforms share many design and implementation properties, but there are key differences between Java and .NET that have an impact on their security. This paper examines how .NET’s design avoids vulnerabilities and limitations discovered in Java and discusses lessons learned (and missed) from experience with Java security. Keywords: virtual machine security, Java security, .NET security, security design principles. 1. Introduction Java and .NET are both platforms for executing untrusted programs with security restrictions. Although they share similar goals and their designs are similar in most respects, there appear to be significant differences in the likelihood of security vulnerabilities in the two platforms. Figure 1 shows the number of major security vulnerabilities reported for each platform. As of December 2005, the Common Vulnerabilities and Exposures (CVE) database contains 150 entries concerning Java vulnerabilities [27], 38 of which we classify as major Java platform security vulnerabilities (we do not include application-specific bugs unrelated to the VM itself). The remaining vulnerabilities included in Figure 1 but not in the CVE are from Sun [39] (9 vulnerabilities) and McGraw and Felten [22] (5 vulnerabilities). The contrast with the .NET platform, in which no major security vulnerabilities have yet been found, appears striking.
    [Show full text]
  • Pro Visual C++/CLI and the .NET 3.5 Platform Copyright © 2009 by Stephen R
    Fraser_10535FRONT.fm Page ii Monday, November 17, 2008 7:03 AM Pro Visual C++/CLI and the .NET 3.5 Platform Copyright © 2009 by Stephen R. G. Fraser All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and the publisher. ISBN-13 (pbk): 978-1-4302-1053-5 ISBN-13 (electronic): 978-1-4302-1054-2 Printed and bound in the United States of America 9 8 7 6 5 4 3 2 1 Trademarked names may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, we use the names only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. Lead Editor: Matthew Moodie Technical Reviewer: Don Reamy Editorial Board: Clay Andres, Steve Anglin, Mark Beckner, Ewan Buckingham, Tony Campbell, Gary Cornell, Jonathan Gennick, Jonathan Hassell, Michelle Lowman, Matthew Moodie, Duncan Parkes, Jeffrey Pepper, Frank Pohlmann, Ben Renow-Clarke, Dominic Shakeshaft, Matt Wade, Tom Welsh Project Manager: Sofia Marchant Copy Editor: Liz Welch Associate Production Director: Kari Brooks-Copony Production Editor: Laura Cheu Compositor: Susan Glinert Stevens Proofreader: Linda Seifert, Lisa Hamilton Indexer: John Collin Artist: April Milne Cover Designer: Kurt Krames Manufacturing Director: Tom Debolski Distributed to the book trade worldwide by Springer-Verlag New York, Inc., 233 Spring Street, 6th Floor, New York, NY 10013.
    [Show full text]
  • C++/CLI Language Specification
    ECMA-372 1st Edition / December 2005 C++/CLI Language Specification Standard ECMA-372 st 1 Edition / December 2005 C++/CLI Language Specification Ecma International Rue du Rhône 114 CH-1204 Geneva T/F: +41 22 849 6000/01 www.ecma-international.org . Table of Contents Table of Contents Introduction...................................................................................................................................................xii 1. Scope............................................................................................................................................................. 1 2. Conformance ............................................................................................................................................... 2 3. Normative references .................................................................................................................................. 3 4. Definitions .................................................................................................................................................... 4 5. Notational conventions................................................................................................................................ 7 6. Acronyms and abbreviations ..................................................................................................................... 8 7. General description....................................................................................................................................
    [Show full text]
  • Understanding and Using Assemblies and Namespaces in .NET
    Understanding and Using Assemblies and Namespaces in .NET Summary: Creating assemblies and namespaces in Microsoft Visual Basic .NET Objectives Understand assemblies in Microsoft® .NET. Understand namespaces in .NET. Use Microsoft Visual Basic® .NET to create and customize an assembly. Use Visual Basic .NET to create a namespace. Assumptions The following should be true for you to get the most out of this document: You are familiar with Visual Basic programming. You understand the basic concept of object-oriented programming. You have access to Visual Basic .NET. Contents Introduction Assemblies Namespaces Practice Creating an Assembly Practice Creating a Namespace What's New Since Visual Basic 6.0? Summary Introduction Microsoft .NET provides several ways to think of your code as more than just a bunch of disconnected lines. As a Visual Basic programmer, you're already familiar with the concept of a class, a section of code that defines an object and its behavior. But two of the higher-level groupings may be unfamiliar to you: An assembly provides a fundamental unit of physical code grouping. A namespace provides a fundamental unit of logical code grouping. As you'll see in this document, you can use Visual Basic .NET to create both assemblies and namespaces. You'll need to understand both of these concepts to be a productive Visual Basic .NET developer. Assemblies An assembly is a collection of types and resources that forms a logical unit of functionality. All types in the .NET Framework must exist in assemblies; the common language runtime does not support types outside of assemblies. Each time you create a Microsoft Windows® Application, Windows Service, Class Library, or other application with Visual Basic .NET, you're building a single assembly.
    [Show full text]
  • NET Interoperability Guidelines
    CAPE-OPEN Delivering the power of component software and open standard interfaces in Computer-Aided Process Engineering .NET Interoperability Guidelines www.colan.org ARCHIVAL INFORMATION Filename Interoperability.doc Authors CO-LaN consortium Status Internal Date June 2006 Version version 0.70 Number of pages 43 Versioning Version 0.4 edited by Bill Barrett (US EPA) Version 0.5 edited by Michel Pons (CO-LaN) Version 0.6 edited by Lars von Wedel (AixCAPE) Version 0.61 edited by Lars von Wedel (AixCAPE) Version 0.62 proofread by Michel Pons (CO-LaN) Version 0.70 edited by Lars von Wedel and Bill Barrett Additional material Web location Implementation specifications version Comments 2 IMPORTANT NOTICES Disclaimer of Warranty CO-LaN documents and publications include software in the form of sample code. Any such software described or provided by CO-LaN --- in whatever form --- is provided "as-is" without warranty of any kind. CO-LaN and its partners and suppliers disclaim any warranties including without limitation an implied warrant or fitness for a particular purpose. The entire risk arising out of the use or performance of any sample code --- or any other software described by the CAPE-OPEN Laboratories Network --- remains with you. Copyright © 2006 CO-LaN and/or suppliers. All rights are reserved unless specifically stated otherwise. CO-LaN is a non for profit organization established under French law of 1901. Trademark Usage Many of the designations used by manufacturers and seller to distinguish their products are claimed as trademarks. Where those designations appear in CO-LaN publications, and the authors are aware of a trademark claim, the designations have been printed in caps or initial caps.
    [Show full text]
  • Microsoft.Smartphone.Programming
    01_762935 ffirs.qxp 11/20/06 7:48 AM Page iii Professional Microsoft® Smartphone Programming Baijian Yang Pei Zheng Lionel M. Ni 01_762935 ffirs.qxp 11/20/06 7:48 AM Page ii 01_762935 ffirs.qxp 11/20/06 7:48 AM Page i Professional Microsoft® Smartphone Programming 01_762935 ffirs.qxp 11/20/06 7:48 AM Page ii 01_762935 ffirs.qxp 11/20/06 7:48 AM Page iii Professional Microsoft® Smartphone Programming Baijian Yang Pei Zheng Lionel M. Ni 01_762935 ffirs.qxp 11/20/06 7:48 AM Page iv Professional Microsoft® Smartphone Programming Published by Wiley Publishing, Inc. 10475 Crosspoint Boulevard Indianapolis, IN 46256 www.wiley.com Copyright © 2007 by Wiley Publishing, Inc., Indianapolis, Indiana Published simultaneously in Canada ISBN: 978-0-471-76293-5 Manufactured in the United States of America 10 9 8 7 6 5 4 3 2 1 1B/QX/RS/QW/IN Library of Congress Cataloging-in-Publication Data: Yang, Baijian, 1972– Microsoft smartphone programming / Baijian Yang, Pei Zheng, Lionel Ni. p. cm. Includes index. ISBN-13: 978-0-471-76293-5 (paper/website) ISBN-10: 0-471-76293-8 (paper/website) 1. Cellular telephones—Computer programs. 2. Pocket computers—Computer programs. 3. Microsoft .NET Frame- work. I. Zheng, Pei, 1972– II. Ni, Lionel M. III. Title. TK6570.M6Y37 2006 621.3845—dc22 2006033469 No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600.
    [Show full text]
  • Introducing .NET and ASP.NET5 the Next Layer up from the CLR Is the .NET Framework Base Class Libraries (BCL)
    526286 Ch01.qxd 9/8/03 11:55 PM Page 3 Chapter Introducing .NET and 1 ASP.NET by Bill Evjen If you are new to .NET—Welcome! If you are a .NET Framework 1.0 veteran—Welcome to .NET version 1.1! .NET 1.0 was introduced with tremendous excitement. It was original, answered many developers’ problems, and truly leap-frogged any of the other technologies out there— especially in the realm of browser-based Internet application development—or ASP.NET. .NET 1.1 and ASP.NET 1.1 are minor releases and should not be considered substantially different versions from 1.0. The next major revision of .NET and ASP.NET will come with the release of version 2.0. This chapter introduces the .NET Framework 1.1 and also shows you what’s new in ASP.NET 1.1. Welcome to .NET Every so often, a technology company needs to step back from itself, look at what it is try- ing to achieve, and then determine whether it needs to try a different approach. Microsoft did this as it stood back from the COM world it had created and asked itself, “Is there a better way?” The .NET Framework is this better way. .NET is confusing to many people because the Microsoft marketing folks took hold of the name and started applying it to every product that Microsoft produced. .NET had almost nothing to do with many products that acquired the .NET moniker. This problem is slowly being corrected. More products are now coming from Microsoft without .NET in their titles.
    [Show full text]