DOCSLIB.ORG

SOFTWARE ARCHITECTURES and PATTERNS for PERSISTENCE in HETEROGENEOUS DATA-INTENSIVE SYSTEMS by AARON SCHRAM B.S., University Of

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
SOFTWARE ARCHITECTURES and PATTERNS for PERSISTENCE in HETEROGENEOUS DATA-INTENSIVE SYSTEMS by AARON SCHRAM B.S., University Of SOFTWARE ARCHITECTURES AND PATTERNS FOR PERSISTENCE IN HETEROGENEOUS DATA-INTENSIVE SYSTEMS by AARON SCHRAM B.S., University of Colorado at Boulder, 2006 M.S., University of Colorado at Boulder, 2011 A thesis submitted to the Faculty of the Graduate School of the University of Colorado in partial fulfillment of the requirement for the degree of Doctor of Philosophy Department of Computer Science 2015 This thesis entitled: Software Architectures and Patterns for Persistence in Heterogeneous Data-Intensive Systems written by Aaron Schram has been approved for the Department of Computer Science ____________________________________ Professor Kenneth M. Anderson, Chair ____________________________________ Professor Richard Han ____________________________________ Professor James Martin ____________________________________ Professor Nenad Medvidovic ____________________________________ Professor Judith Stafford Date___________ The final copy of this thesis has been examined by the signatories, and we find that both the content and the form meet acceptable presentation standards of scholarly work in the above mentioned discipline. iii ABSTRACT Schram, Aaron (Ph.D., Computer Science) Software Architectures and Patterns for Persistence in Heterogeneous Data- Intensive Systems Thesis directed by Associate Professor Kenneth M. Anderson Software engineers are faced with a variety of difficult choices when selecting appropriate technologies on which to base a software system. As the typical software user has become accustomed to systems being “on-demand” and “always- available,” the software engineer is more concerned than ever before about the issues of system scalability, availability, and durability. In the absence of expertise in distributed systems, architectural decisions become complex, slowing feature development and introducing error. Software engineering is in need of robust patterns and tools that increase the accessibility of specialized technologies developed for the completion of specialized tasks. This dissertation describes my existing work related to the challenges of domain modeling and data-access in large- scale, heterogeneous data-intensive systems and extends this work to include novel architectures for utilizing multiple large-scale data stores effectively. This research focuses on increasing the accessibility and flexibility of these data stores, which typically afford scalability, availability, and durability at the cost of added complexity for the application developer. The resulting architecture and associated implementation alleviates common challenges faced by small and medium software enterprises during the development of heterogeneous data-intensive software applications. DEDICATION To my parents and grandparents who I will always admire for their struggles, sacrifices, and hard work v ACKNOWLEDGEMENTS I would like to thank my colleagues in the various software engineering organizations that I have been a part of throughout my career for the valuable experience and lessons taken from each. Specifically I would like to thank Nate Sammons, William Butler, and Burke Webster for their continued support and friendship through this and many other shared experiences. This dissertation would not have been possible without the positive and long lasting relationship with my advisor, Dr. Kenneth M. Anderson, which began so many years ago when I was an optimistic and overly curious undergraduate. Thank you for guiding me on a journey only you saw I needed to take. Finally, I would like to thank my incredibly supportive network of friends and colleagues for, among many other things, ensuring I enjoy the moment. vi CONTENTS CHAPTER 1: INTRODUCTION .................................................................................... 1 CHAPTER 2: LARGE-SCALE DATA STORES .......................................................... 10 2.1 DATA STORE TECHNOLOGY OVERVIEW ......................................................................... 11 2.1.1 Relational ............................................................................................................... 12 2.1.2 Columnar ............................................................................................................... 14 2.1.3 Key-Value ............................................................................................................... 15 2.1.4 Document ............................................................................................................... 17 2.1.5 Network .................................................................................................................. 19 CHAPTER 3: TOWARDS N DATA STORE ARCHITECTURES .............................. 21 3.1 CHALLENGES FOR SOFTWARE ENGINEERING ............................................................... 21 3.2 OPPORTUNITIES FOR SOFTWARE ENGINEERING ........................................................... 26 CHAPTER 4: RELATED WORK ................................................................................. 28 CHAPTER 5: EVOLUTION OF THE PERSISTENCE TIER .................................... 35 5.1 TRADITIONAL ENTERPRISE ARCHITECTURE ................................................................. 37 5.2 TRADITIONAL MULTI-DATA STORE APPROACH ............................................................. 39 5.3 TRADITIONAL PERSISTENCE ARCHITECTURE DESIGN CONSTRAINTS ........................... 41 CHAPTER 6: THE DIAMOND ARCHITECTURE ..................................................... 45 6.1 OVERVIEW ..................................................................................................................... 45 6.2 APPROACH ..................................................................................................................... 46 6.3 DIAMOND ARCHITECTURAL COMPONENTS ................................................................... 49 vii 6.3.1 Command ............................................................................................................... 49 6.3.2 Runner ................................................................................................................... 51 6.3.3 Data Model Converter ........................................................................................... 52 6.3.4 Reconciler ............................................................................................................... 56 6.3.5 Pipeline .................................................................................................................. 57 6.3.6 RepositoryFacade (Optional) ................................................................................ 60 6.3.7 Final Architecture ................................................................................................. 61 CHAPTER 7: IMPLEMENTATION: THE MACHINIST FRAMEWORK ................. 64 7.1 THE MACHINIST FRAMEWORK ...................................................................................... 64 7.1.1 Command ............................................................................................................... 65 7.1.2 Runner ................................................................................................................... 73 7.1.3 Reconciler ............................................................................................................... 75 7.1.4 Pipeline .................................................................................................................. 75 7.1.5 Decorators .............................................................................................................. 83 CHAPTER 8: MBOX: A MULTI-DATA STORE INBOX ............................................ 87 8.1 OVERVIEW ..................................................................................................................... 87 8.2 DESIGN AND ARCHITECTURE ......................................................................................... 89 8.2.1 Data Model ............................................................................................................. 91 8.2.2 Persist .................................................................................................................... 92 8.2.3 Save ........................................................................................................................ 93 8.2.4 Find By User .......................................................................................................... 99 8.2.5 Search ................................................................................................................... 102 8.2.6 Decorators ............................................................................................................ 104 CHAPTER 9: EVALUATION ..................................................................................... 106 viii 9.1 OVERVIEW ................................................................................................................... 106 9.2 APPROACH ................................................................................................................... 107 9.3 ASSUMPTIONS .............................................................................................................. 107 9.4 EVALUATION SCENARIOS ............................................................................................ 109 9.4.1 Additional Data Store ......................................................................................... 109 9.4.2 Persist Data to Additional
Load more

Recommended publications
  • Chatbot on Serverless/Lamba Architecture Nandan.A Prof.Shilpa Choudary Student, Reva University Professor, Reva University
    SECOND NATIONAL CONFERENCE ON ADVANCES IN COMPUTING AND INFORMATION TECHNOLOGY ISSN:2347-7385 Chatbot on Serverless/Lamba Architecture Nandan.A Prof.Shilpa Choudary Student, Reva University Professor, Reva University Abstract—The OpenLambda, a new, opensource The logic tier contains the code required to translate platform for building next-generation web services user actions at the presentation tier to the and applications on serverless computation. The functionality key aspects of serverless computation and that drives the application’s behavior. The data tier present numerous research challenges that consists of storage media (databases, object stores, must be, addressed in the design and caches, file systems, etc.) that hold the data relevant implementation of such systems. The study of to the application. Figure 1 shows an example of a current web applications, so as to better motivate simple three-tier application. some aspects of serverless application construction. Chatbots platform are used by consumers worldwide for integrating it with backend services . It is still difficult to build and deploy chatbots developers need to handle the coordination of the backend services to build the chatbot interface, integrate the chatbot with external services, and worry about extensibility, scalability, and maintenance. The serverless architecture could be ideal platform to build the chatbot. Figure 1: Architectural pattern for a simple three- tier application Keywords: Lambda Architecture, Chat-bot, Multitier Architecture, Microservices. In Serverless Multi-Tier Architectures a backend remains private and secure. The benefits of this powerful pattern across each tier of a multi-tiered I. INTRODUCTION architecture. Example of a multitiered architecture is The multi-tier application has been a well- a three-tier web application.
    [Show full text]
  • Download Vol 11, No 1&2, Year 2018
    The International Journal on Advances in Internet Technology is published by IARIA. ISSN: 1942-2652 journals site: http://www.iariajournals.org contact: [email protected] Responsibility for the contents rests upon the authors and not upon IARIA, nor on IARIA volunteers, staff, or contractors. IARIA is the owner of the publication and of editorial aspects. IARIA reserves the right to update the content for quality improvements. Abstracting is permitted with credit to the source. Libraries are permitted to photocopy or print, providing the reference is mentioned and that the resulting material is made available at no cost. Reference should mention: International Journal on Advances in Internet Technology, issn 1942-2652 vol. 11, no. 1 & 2, year 2018, http://www.iariajournals.org/internet_technology/ The copyright for each included paper belongs to the authors. Republishing of same material, by authors or persons or organizations, is not allowed. Reprint rights can be granted by IARIA or by the authors, and must include proper reference. Reference to an article in the journal is as follows: <Author list>, “<Article title>” International Journal on Advances in Internet Technology, issn 1942-2652 vol. 11, no. 1 & 2, year 2018, <start page>:<end page> , http://www.iariajournals.org/internet_technology/ IARIA journals are made available for free, proving the appropriate references are made when their content is used. Sponsored by IARIA www.iaria.org Copyright © 2018 IARIA International Journal on Advances in Internet Technology Volume 11, Number 1 & 2, 2018 Editors-in-Chief Mariusz Głąbowski, Poznan University of Technology, Poland Editorial Advisory Board Eugen Borcoci, University "Politehnica"of Bucharest, Romania Lasse Berntzen, University College of Southeast, Norway Michael D.
    [Show full text]
  • Web-Based Content Management System
    Maciej Dobecki, Wojciech Zabierowski / Computing, 2010, Vol. 9, Issue 2, 127-130 [email protected] ISSN 1727-6209 www.computingonline.net International Journal of Computing WEB-BASED CONTENT MANAGEMENT SYSTEM Maciej Dobecki, Wojciech Zabierowski Technical University of Lodz, al. Politechniki 11, 90-924 Łódź, Poland, e-mail: [email protected], [email protected] http://www.dmcs.p.lodz.pl Abstract: This paper describes how to design content management system using the newest web-based techniques. It contains helpful information that can be used during selecting programming language. It introduces multi layer architecture with description and functionality of each layer. It provides description of Model View Controller pattern and how to use it in multi-layer application design. It shows the most powerful Java frameworks that can be applied for each layer and how to connect them in simple way, using Inversion of Control container. It shows power of Spring Framework as business layer, Hibernate as integration layer and ZK Ajax as presentation layer. It proves, that Java combined with applicable libraries can be very powerful tool in good hands. Keywords: CMS, JEE, Spring, Hibernate, AJAX. 1. INTRODUCTION CMS is prepared through a simple-to-use user interface. Usually it is a set of web pages containing The Internet – today is the most powerful and complex forms and modules. popular information media. What was impossible The primary task of the CMS platform is even few years ago is now available by “clicking separation of data content from presentation (the the mouse”. Both small firms and global giants do way of its look).
    [Show full text]
  • MV* Design Patterns
    MV* Design Patterns Alexander Nelson August 30, 2021 University of Arkansas - Department of Computer Science and Computer Engineering Reminders Course Mechanics Course Webpage: https://ahnelson.uark.edu/courses/ csce-4623-mobile-programming-fall-2021/ Syllabus is on the website. Course Communication: https://csce4623-uark.slack.com/ This slack channel is to be the primary mode of communication Projects Choose a project idea and team for the final project ASAP First project report is due September 10th Multitier Architectures What is a multitier architecture? Physical separation of data concerns Examples: • Presentation (UI) • Application Processing • Data Management Why split into layers? OSI Model Why split into layers? Separation of concerns! A change to one layer can have no bearing on the rest of the model e.g. Fiberoptic instead of Coax at the PHY layer OSI Model How does this apply to mobile? Application designers often want separation of UI and logic! Three tier architecture These software engineering abstractions relate to the MV* architectures that are common in mobile computing systems Model View Controller (MVC) Model View Controller 1 1Krasner 1988 Definitions Model: Models are those components of the system application that actually do the work View: Display aspects of the models Controller: Used to send messages to the model, provide interface between model, views, and UI devices. Models Models enable encapsulation Model encapsulates all data as well as methods to change them • Can change the underlying data structures without
    [Show full text]
  • Electronic Commerce Basics
    Electronic Commerce Principles and Practice This Page Intentionally Left Blank Electronic Commerce Principles and Practice Hossein Bidgoli School of Business and Public Administration California State University Bakersfield, California San Diego San Francisco New York Boston London Sydney Tokyo Toronto This book is printed on acid-free paper. ∞ Copyright © 2002 by ACADEMIC PRESS All Rights Reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher. Requests for permission to make copies of any part of the work should be mailed to: Permissions Department, Harcourt Inc., 6277 Sea Harbor Drive, Orlando, Florida 32887-6777 Academic Press A Harcourt Science and Technology Company 525 B Street, Suite 1900, San Diego, California 92101-4495, USA http://www.academicpress.com Academic Press Harcourt Place, 32 Jamestown Road, London NW1 7BY, UK http://www.academicpress.com Library of Congress Catalog Card Number: 2001089146 International Standard Book Number: 0-12-095977-1 PRINTED IN THE UNITED STATES OF AMERICA 010203040506EB987654321 To so many fine memories of my brother, Mohsen, for his uncompromising belief in the power of education This Page Intentionally Left Blank Contents in Brief Part I Electronic Commerce Basics CHAPTER 1 Getting Started with Electronic Commerce 1 CHAPTER 2 Electronic Commerce Fundamentals 39 CHAPTER 3 Electronic Commerce in Action
    [Show full text]
  • Client Server Communications Middleware Components
    1 Assistant lecturer Ahmed S. Kareem CLIENT SERVER COMMUNICATIONS MIDDLEWARE COMPONENTS The communication middleware software provides the means through which clients and servers communicate to perform specific actions. It also provides specialized services to the client process that insulates the front-end applications programmer from the internal working of the database server and network protocols. In the past, applications programmers had to write code that would directly interface with specific database language (generally a version of SQL) and the specific network protocol used by the database server. Multitier architecture In software engineering, multi-tier architecture (often referred to as n-tier architecture) is a client–server architecture in which the presentation, the application processing, and the data management are logically separate processes. For example, an application that uses middleware to service data requests between a user and a database employs multi-tier architecture. The most widespread use of multi-tier architecture is the three-tier architecture. N-tier application architecture provides a model for developers to create a flexible and reusable application. By breaking up an application into tiers, developers only have to modify or add a specific layer, rather than have to rewrite the entire application over. There should be a presentation tier, a business or data access tier, and a data tier. The concepts of layer and tier are often used interchangeably. However, one fairly common point of view is that there is indeed a difference, and that a layer is a logical structuring mechanism for the elements that make up the software solution, while a tier is a physical structuring mechanism for the system infrastructure.
    [Show full text]
  • Design Patterns Past and Future
    Proceedings of Informing Science & IT Education Conference (InSITE) 2011 Design Patterns Past and Future Aleksandar Bulajic Metropolitan University, Belgrade, Serbia [email protected]; [email protected] Abstract A very important part of the software development process is service or component internal de- sign and implementation. Design Patterns (Gamma et al., 1995) provide list of the common pat- terns used in the object-oriented software design process. The primary goal of the Design Patterns is to reuse good practice in the design of new developed components or applications. Another important reason of using Design Patterns is improving common application design understand- ing and reducing communication overhead by reusing the same generic names for implemented solution. Patterns are designed to capture best practice in a specific domain. A pattern is supposed to present a problem and a solution that is supported by an example. It is always worth to listen to an expert advice, but keep in mind that common sense should decide about particular implemen- tation, even in case when are used already proven Design Patterns. Critical view and frequent and well designed testing would give an answer about design validity and a quality. Design Patterns are templates and cannot be blindly copied. Each design pattern records design idea and shall be adapted to particular implementation. Using time to research and analyze existing solutions is recommendation supported by large number of experts and authorities and fits very well in the pattern basic philosophy; reuse solution that you know has been successfully implemented in the past. Sections 2 and 3 are dedicated to the Design Patterns history and theory as well as literature sur- vey.
    [Show full text]
  • Firewall Deployment for Multitier Applications by Lenny Zeltser
    Firewall Deployment for Multitier Applications By Lenny Zeltser This article examines considerations for deploying firewalls as part of a network perimeter around Internet-facing servers. The discussion focuses on situations that may warrant strict separation of network resources into dedicated subnets, and explains how to enforce access restrictions using firewalls in a way that matches business and technical requirements of multitier applications. The article introduces several network architectures that use a single firewall as well as firewalls deployed one behind another in series, and addresses the strengths and weaknesses of each approach. Partitioned network architectures can be used to protect multitier applications accessible over the Web. Following the trend of designing applications in an expandable and scalable manner, these applications are often created by using modules that run on different servers and that typically form three distinct groups: presentation, middleware, and data tiers. Let’s begin by examining how the architecture of such applications may influence the design the network’s security perimeter. Multitier Applications By segmenting a Web-based application into several logical tiers, software architects isolate core functional areas into groupings that can be designed, developed, and maintained somewhat independently of each other. The following tiers are present in some way in most Web-facing applications of moderate complexity: • Presentation components are usually adjacent to the Internet and are the only modules directly accessed by end users. Such publicly accessible services are often implemented using Web, DNS, and mail servers. Software running on these servers, operating as part of a unified system, presents the application to users and handles interactions between users and back-end components.
    [Show full text]
  • State of Washington Department of Labor and Industries
    State of Washington Department of Labor and Industries Centers of Occupation Health and Education Occupational Health Management System (OHMS) Feasibility Study March 5, 2012 FINAL (Version 4.0) Washington Department of Labor & Industries Occupation Health Management System (OHMS) Feasibility Study Status: FINAL Date: March 5, 2012 Document Control This document represents the feasibility study analysis of the information system required to support the Department of Labor and Industries (L&I) Centers of Occupational Health and Education (COHE) as part of the state Workers’ Compensation insurance reform. The study complies with the state Office of the Chief Information Officer (OCIO) requirement for a feasibility study for level 3 projects. Version Date Description/Changes 1.0 1/20/2012 Initial working draft of chapters I–V and initial appendices for review and comment. Preliminary, NOT EDITED version. 2.0 2/7/2012 Incorporated comments on version 1.0 draft. Updated Work Breakdown Structure. Added new chapters VI–X, XIII. Added new appendices for Vendor Survey, Washington HIE, and Risk Factor Analysis. Preliminary, NOT EDITED version. 3.0 2/20/2012 Incorporated comments on version 1.0 draft. Updated Alternatives Comparison chapter. Updated Major Alternatives Considered chapter. Added Estimated Timeframe and Work Plan chapter. Added Cost-Benefit Analysis chapter. Full Draft. EDITED, except Chapter XII – Cost Benefit Analysis 4.0 3/5/2012 Incorporated comments from version 3.0 draft. Incorporated Executive Summary. Incorporated Alternatives vs. Scope Analysis Appendix. The anticipated audience for this report includes L&I, the OCIO, the Technology Advisory Board, and other interested stakeholders. 4016.001\OHMS Feasibility Study\4.0 i FINAL sooscreekconsulting.com 3/5/2012 Washington Department of Labor & Industries Occupation Health Management System (OHMS) Feasibility Study TABLE OF CONTENTS Page I.
    [Show full text]
  • Chapter 4 INTERNET ARCHITECTURES FOR
    Chapter 4 INTERNET ARCHITECTURES FOR APPLICATION SERVICE PROVIDERS Borko Furht, Chris Phoenix, John Yin, and Zijad Aganovic Abstract In recent years, business on the Internet has exponentially increased. Consequently, the deployment and management of business applications on the Internet is becoming more and more complex, which requires the development of new Internet architectures suitable to efficiently run these business applications. In this chapter we present and evaluate several computing models and related Internet architectures for application service providers. We also introduce the server-based model and the corresponding Internet architecture and two case studies, which use the proposed architecture for application deployment. 1. INTRODUCTION 1.1 COMPUTING MODELS FOR INTERNET-BASED ARCHITECTURES The increasingly competitive, global marketplace puts pressure on companies to create and deliver their products faster, with high quality and greater performance. To get the new products and technologies to consumers is through a new industry called Application Service Providers (ASPs). Similarly to Internet Service Providers, that linked businesses and consumers up to the Internet, ASPs lease software applications to businesses and consumers via the Internet. These applications range from word processing programs to payroll management software, document management systems, and many others. The major challenge is to develop an efficient Internet-based architecture, which will efficiently provide access to these software applications over the Internet. Application architectures have traditionally followed software development architectures. The software development architectures can be classified into: · Traditional desktop computing model · Client-server computing model 68 Chapter 4 · Network computing model · Server-based computing model Traditional desktop computing model assumes that the whole application is on the client and the application is executed locally.
    [Show full text]
  • Integration of Struts & Spring & Hibernate for Enterprise Applications
    International OPEN ACCESS Journal Of Modern Engineering Research (IJMER) Integration of Struts & Spring & Hibernate for Enterprise Applications Deepak Kasgar1, Harish Chandra Maurya2 ¹(Department of Computer Science and Engineering,Bhagwant University, India ²(Department of Computer Science and Engineering, Bhagwant University, India ABSTRACT: The proposal of this paper is to present Spring Framework which is widely used in developing enterprise applications. Considering the current state where applications are developed using the EJB model, Spring Framework assert that ordinary java beans(POJO) can be utilize with minimal modifications. This modular framework can be used to develop the application faster and can reduce complexity. This paper will highlight the design overview of Spring Framework along with its features that have made the framework useful. The integration of multiple frameworks for an E-commerce system has also been addressed in this paper. This paper also proposes structure for a website based on integration of Spring, Hibernate and Struts Framework. Keywords- Spring Framework, IOC, Struts, Hibernate, E-commerce I. Introduction Frameworks are abstraction layer on existing technology. If developer’s directly works on technologies of java then project development time and cost will increase time to time.Non Invasive Frameworks does not force a developer to extend this class from a framework specific class or implement a framework specific interface.Spring framework is a complete and a modular framework. It means complete
    [Show full text]
  • Middleware Architecture with Patterns and Frameworks
    Middleware Architecture with Patterns and Frameworks Sacha Krakowiak Distributed under a Creative Commons license http://creativecommons.org/licenses/by-nc-nd/3.0/ February 27, 2009 Contents Preface ix References........................................ x 1 An Introduction to Middleware 1-1 1.1 Motivation for Middleware . 1-1 1.2 CategoriesofMiddleware . 1-6 1.3 A Simple Instance of Middleware: Remote Procedure Call . ......... 1-7 1.3.1 Motivations and Requirements . 1-8 1.3.2 Implementation Principles . 1-9 1.3.3 Developing Applications with RPC . 1-11 1.3.4 SummaryandConclusions. .1-13 1.4 Issues and Challenges in Middleware Design . 1-14 1.4.1 DesignIssues ...............................1-14 1.4.2 Architectural Guidelines . 1-15 1.4.3 Challenges ................................1-17 1.5 HistoricalNote .................................. 1-18 References........................................1-19 2 Middleware Principles and Basic Patterns 2-1 2.1 ServicesandInterfaces . 2-1 2.1.1 Basic Interaction Mechanisms . 2-2 2.1.2 Interfaces ................................. 2-3 2.1.3 Contracts and Interface Conformance . 2-5 2.2 ArchitecturalPatterns . 2-9 2.2.1 Multilevel Architectures . 2-9 2.2.2 DistributedObjects . .. .. .. .. .. .. .. .2-12 2.3 Patterns for Distributed Object Middleware . 2-15 2.3.1 Proxy ...................................2-15 2.3.2 Factory ..................................2-16 2.3.3 Adapter..................................2-18 2.3.4 Interceptor ................................2-19 2.3.5 Comparing and Combining Patterns . 2-20 2.4 Achieving Adaptability and Separation of Concerns . ..........2-21 2.4.1 Meta-ObjectProtocols. 2-21 2.4.2 Aspect-Oriented Programming . 2-23 ii CONTENTS 2.4.3 PragmaticApproaches. .2-25 2.4.4 ComparingApproaches .
    [Show full text]