DOCSLIB.ORG

Semi-Automated Parallel Programming in Heterogeneous Intelligent Reconfigurable Environments (SAPPHIRE) Sean Stanek Iowa State University

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Semi-Automated Parallel Programming in Heterogeneous Intelligent Reconfigurable Environments (SAPPHIRE) Sean Stanek Iowa State University Iowa State University Capstones, Theses and Graduate Theses and Dissertations Dissertations 2012 Semi-automated parallel programming in heterogeneous intelligent reconfigurable environments (SAPPHIRE) Sean Stanek Iowa State University Follow this and additional works at: https://lib.dr.iastate.edu/etd Part of the Computer Sciences Commons Recommended Citation Stanek, Sean, "Semi-automated parallel programming in heterogeneous intelligent reconfigurable environments (SAPPHIRE)" (2012). Graduate Theses and Dissertations. 12560. https://lib.dr.iastate.edu/etd/12560 This Dissertation is brought to you for free and open access by the Iowa State University Capstones, Theses and Dissertations at Iowa State University Digital Repository. It has been accepted for inclusion in Graduate Theses and Dissertations by an authorized administrator of Iowa State University Digital Repository. For more information, please contact [email protected]. Semi-automated parallel programming in heterogeneous intelligent reconfigurable environments (SAPPHIRE) by Sean Stanek A dissertation submitted to the graduate faculty in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Major: Computer Science Program of Study Committee: Carl Chang, Major Professor Johnny Wong Wallapak Tavanapong Les Miller Morris Chang Iowa State University Ames, Iowa 2012 Copyright © Sean Stanek, 2012. All rights reserved. ii TABLE OF CONTENTS LIST OF TABLES ...................................................................................................................... viii LIST OF FIGURES ....................................................................................................................... ix PROJECT SUMMARY .................................................................................................................. xi CHAPTER 1. INTRODUCTION .................................................................................................. 1 CHAPTER 2. RELATED WORK ................................................................................................ 5 2.1 Program Construction ............................................................................................... 5 2.1.1 DirectShow ....................................................................................................... 5 2.1.2 Circuit Languages ............................................................................................. 6 2.1.3 Aspect Oriented Programming ......................................................................... 6 2.2 Parallel Computing ................................................................................................... 7 2.2.1 Data Parallelism ................................................................................................ 7 2.2.2 Task Parallelism and Stream Programming ...................................................... 8 2.2.3 Hybrid Task and Data Parallelism .................................................................... 9 2.3 Multiprocessor Task Scheduling............................................................................. 10 2.3.1 Homogeneous Task Scheduling ...................................................................... 10 2.3.2 Heterogeneous Task Scheduling ..................................................................... 11 2.3.3 Stream Task Scheduling ................................................................................. 13 2.3.4 Heterogeneous Stream Task Scheduling ........................................................ 15 2.3.5 Windows Task Scheduler ............................................................................... 16 2.4 Endoscopy Video Analysis ..................................................................................... 17 iii 2.4.1 Picture Archiving and Communication Systems (PACS) .............................. 17 2.4.2 Scene Segmentation and Object Detection for Colonoscopy ......................... 18 CHAPTER 3. OVERVIEW OF SAPPHIRE............................................................................... 20 3.1 Design and Overview of SAPPHIRE ..................................................................... 20 3.2 Features and Strategies ........................................................................................... 25 3.2.1 Semi-Automated Program Construction ......................................................... 26 3.2.2 Module Implementation .................................................................................. 28 3.2.3 Data Packets .................................................................................................... 30 3.2.4 Communication ............................................................................................... 32 3.2.5 Synchronization .............................................................................................. 33 3.2.6 Feedback Loop ................................................................................................ 35 3.2.7 Data Filtering .................................................................................................. 35 3.2.8 Profiling .......................................................................................................... 36 3.2.9 Memory Leak Detection ................................................................................. 38 3.2.10 Crash Reporting .............................................................................................. 40 3.3 Common Modules and Data Types......................................................................... 40 3.3.1 Video Data and Modules ................................................................................ 41 3.3.2 Simple Analysis Modules ............................................................................... 44 3.3.3 Helper and Extension Modules ....................................................................... 45 3.4 Example EndoCapture.ini ....................................................................................... 46 3.4.1 Example Task Graph....................................................................................... 47 CHAPTER 4. SAPPHIRE INTERNALS ................................................................................... 48 4.1 Pseudocode for a Main Program ............................................................................. 48 4.2 Program and Virtual Graph Construction ............................................................... 48 iv 4.2.1 Packet Filters and Priority Overrides .............................................................. 50 4.2.2 Generating the Virtual Parallel Task Graph .................................................... 51 4.2.3 Data and Module Pruning ............................................................................... 53 4.2.4 Updating the Internal State ............................................................................. 53 4.2.5 Starting the Program ....................................................................................... 53 4.2.6 Data Processing ............................................................................................... 53 4.3 Work Loop using Windows Scheduler ................................................................... 54 4.4 Data Structures ........................................................................................................ 57 4.4.1 Packets ............................................................................................................ 57 4.4.2 Modules........................................................................................................... 59 4.4.3 Internal Arrays and Lists ................................................................................. 59 4.4.4 Common Data Packet Formats ....................................................................... 59 4.5 Synchronization ...................................................................................................... 61 4.5.1 Middleware Synchronization .......................................................................... 61 4.5.2 Application Synchronization .......................................................................... 62 4.6 Runtime Profiling.................................................................................................... 64 4.7 Memory Leak Detection ......................................................................................... 65 4.8 Crash Reporting ...................................................................................................... 66 4.9 Video Processing Considerations ........................................................................... 68 4.10 Evaluation of SAPPHIRE ....................................................................................... 69 4.10.1 Case Ctudy Implementation ............................................................................ 69 4.10.2 Consistency Checking ..................................................................................... 70 4.10.3 Stress Testing .................................................................................................. 70 4.10.4 Error Reporting ............................................................................................... 71 v 4.11 Summary and Future Work ..................................................................................... 71 CHAPTER 5. TASK SCHEDULING OF STREAM PROGRAMS ON HETEROGENEOUS SYSTEMS ... 73 5.1 Our Contributions
Load more

Recommended publications
  • [MS-ERREF]: Windows Error Codes
    [MS-ERREF]: Windows Error Codes Intellectual Property Rights Notice for Open Specifications Documentation . Technical Documentation. Microsoft publishes Open Specifications documentation for protocols, file formats, languages, standards as well as overviews of the interaction among each of these technologies. Copyrights. This documentation is covered by Microsoft copyrights. Regardless of any other terms that are contained in the terms of use for the Microsoft website that hosts this documentation, you may make copies of it in order to develop implementations of the technologies described in the Open Specifications and may distribute portions of it in your implementations using these technologies or your documentation as necessary to properly document the implementation. You may also distribute in your implementation, with or without modification, any schema, IDL's, or code samples that are included in the documentation. This permission also applies to any documents that are referenced in the Open Specifications. No Trade Secrets. Microsoft does not claim any trade secret rights in this documentation. Patents. Microsoft has patents that may cover your implementations of the technologies described in the Open Specifications. Neither this notice nor Microsoft's delivery of the documentation grants any licenses under those or any other Microsoft patents. However, a given Open Specification may be covered by Microsoft Open Specification Promise or the Community Promise. If you would prefer a written license, or if the technologies described in the Open Specifications are not covered by the Open Specifications Promise or Community Promise, as applicable, patent licenses are available by contacting [email protected]. Trademarks. The names of companies and products contained in this documentation may be covered by trademarks or similar intellectual property rights.
    [Show full text]
  • Microsoft Directshow: a New Media Architecture
    TECHNICAL PAPER Microsoft Directshow: A New Media Architecture By Amit Chatterjee and Andrew Maltz The desktop revolution in production and post-production has dramatical- streaming. Other motivating factors are ly changed the way film and television programs are made, simultaneously the new hardware buses such as the reducing equipment costs and increasing operator eficiency. The enabling IEEE 1394 serial bus and Universal digital innovations by individual companies using standard computing serial bus (USB), which are designed with multimedia devices in mind and platforms has come at a price-these custom implementations and closed promise to enable broad new classes of solutions make sharing of media and hardware between applications difi- audio and video application programs. cult if not impossible. Microsoft s DirectShowTMStreaming Media To address these and other require- Architecture and Windows Driver Model provide the infrastructure for ments, Microsoft introduced Direct- today’s post-production applications and hardware to truly become inter- ShowTM, a next-generation media- operable. This paper describes the architecture, supporting technologies, streaming architecture for the and their application in post-production scenarios. Windows and Macintosh platforms. In development for two and a half years, Directshow was released in August he year 1989 marked a turning Additionally, every implementation 1996, primarily as an MPEG-1 play- Tpoint in post-production equip- had to fight with operating system back vehicle for Internet applications, ment design with the introduction of constraints and surprises, particularly although the infrastructure was desktop digital nonlinear editing sys- in the areas of internal stream synchro- designed with a wide range of applica- tems.
    [Show full text]
  • Docker Windows Task Scheduler
    Docker Windows Task Scheduler Genealogical Scarface glissading, his karyotype outgone inflicts overflowingly. Rudolph is accessorial and suckers languorously as sociologistic Engelbart bridled sonorously and systematises sigmoidally. Which Cecil merchandises so unbelievably that Cole comedowns her suavity? Simple task runner that runs pending tasks in Redis when Docker container. With Docker Content Trust, see will soon. Windows Tip Run applications in extra background using Task. Cronicle is a multi-server task scheduler and runner with a web based front-end UI It handles both scheduled repeating and on-demand jobs targeting any. Django project that you would only fetch of windows task directory and how we may seem. Docker schedulers and docker compose utility program by learning service on a scheduled time, operators and manage your already interact with. You get a byte array elements followed by the target system privileges, manage such data that? Machine learning service Creatio Academy. JSON list containing all my the jobs. As you note have noticed, development, thank deity for this magazine article. Docker-crontab A docker job scheduler aka crontab for. Careful with your terminology. Sometimes you and docker schedulers for task failed job gets silently redirected to get our task. Here you do want to docker swarm, task scheduler or scheduled background tasks in that. Url into this script in one easy to this was already existing cluster created, it retry a little effort. Works pretty stark deviation from your code is followed by searching for a process so how to be executed automatically set. Now docker for windows service container in most amateur players play to pass as.
    [Show full text]
  • AVT Active Firepackage V1.1 – Release and Revision Notes 19 March 2008
    AVT Active FirePackage v1.1 – Release and revision notes 19 March 2008 Overview The AVT Active FirePackage (AFP) is a software development kit (SDK) that focuses on ActiveX Control based programming, but also provides interfaces for DirectShow and TWAIN for interfacing to third-party imaging software. The SDK is compatible with Microsoft Windows (Vista, XP, 2000) and includes an IEEE1394 digital camera system driver that is based on the Windows Driver Model (WDM). The driver can be installed manually, but also via an automatic driver install tool. The AFP has been created for programmers who are familiar with ActiveX Controls, COM, DirectShow or TWAIN, and who want to achieve their goals quickly in a comfortable way by using application development tools such as Visual C++, Visual Basic, VB.NET, C#, Java, Delphi, or others. This document provides an overview of the components and their versions provided with the AVT Active FirePackage v1.1. Furthermore, the package architecture is shown (see figure 1) and additional information about the system requirements and certain constraints of this release are listed. Package content This version of the AVT Active FirePackage contains the following components: • Camera system driver – based on the Microsoft IEEE1394 driver set, suitable for all AVT IEEE1394 cameras. • ActiveX Control – powerful, multi-function COM interface that provides various PropertyPages to configure the camera and the settings for image acquisition and supports many events. • DirectShow filter – In addition to a (WDM) Video Capture Source filter (see figure 1 for supported DirectShow interfaces) AVT camera specific transform filters for YUV411 and Y800 output formats are provided to support DirectX based video streaming applications.
    [Show full text]
  • FROM SHAMOON to STONEDRILL Wipers Attacking Saudi Organizations and Beyond
    FROM SHAMOON TO STONEDRILL Wipers attacking Saudi organizations and beyond Version 1.05 2017-03-07 Beginning in November 2016, Kaspersky Lab observed a new wave of wiper attacks directed at multiple targets in the Middle East. The malware used in the new attacks was a variant of the infamous Shamoon worm that targeted Saudi Aramco and Rasgas back in 2012. Dormant for four years, one of the most mysterious wipers in history has returned. So far, we have observed three waves of attacks of the Shamoon 2.0 malware, activated on 17 November 2016, 29 November 2016 and 23 January 2017. Also known as Disttrack, Shamoon is a highly destructive malware family that effectively wipes the victim machine. A group known as the Cutting Sword of Justice took credit for the Saudi Aramco attack by posting a Pastebin message on the day of the attack (back in 2012), and justified the attack as a measure against the Saudi monarchy. The Shamoon 2.0 attacks observed since November 2016 have targeted organizations in various critical and economic sectors in Saudi Arabia. Just like the previous variant, the Shamoon 2.0 wiper aims for the mass destruction of systems inside targeted organizations. The new attacks share many similarities with the 2012 wave, though featuring new tools and techniques. During the first stage, the attackers obtain administrator credentials for the victim’s network. Next, they build a custom wiper (Shamoon 2.0) which leverages these credentials to spread widely inside the organization. Finally, on a predefined date, the wiper activates, rendering the victim’s machines completely inoperable.
    [Show full text]
  • Enterprise Job Scheduling Checklist
    ••• ••• ••• 2 Forest Park Drive Farmington, CT 06032 Tel: 800 261 JAMS www.JAMSScheduler.com Enterprise Job Scheduling Checklist Following is a detailed list of evaluation criteria that you can use to benchmark the features and functions of various job schedulers your organization is considering. This checklist provides a way to thoroughly assess how well a given product meets your needs now and in the future. Product X General Cross Platform Scheduling Capabilities (Windows, UNIX, Linux, OpenVMS, System i, All Virtual Platforms, MacOS, zLinux, etc.) Support for native x64 and x86 Windows platforms Single GUI to connect to multiple Schedulers and Agents if necessary Kerberos Support Active Directory Support / ADAM Support Windows Management Instrumentation High Availability Architecture supporting Clustering and Standalone Automated Failover Scalable Architecture to support 500k+ jobs/day and more than 2,500 Server Connections Event-Driven Architecture Free, Unlimited Deployment of Admin Clients Scheduling Date/time based scheduling Event based scheduling Ad hoc scheduling Multiple jobs can be tied together in a Setup or Workflow Nested Jobs and Job Plans supported through Setups and Workflows Unlimited number of Job dependencies on one or multiple jobs in a Setup or Workflow Job dependencies between different schedulers File presence, absence & available dependencies and events Variable comparison dependencies Graphical Gantt view of the schedule with projected time runs Hooks for user defined dependencies Graphical view of job stream
    [Show full text]
  • Windows Intruder Detection Checklist
    Windows Intruder Detection Checklist http://www.cert.org/tech_tips/test.html CERT® Coordination Center and AusCERT Windows Intruder Detection Checklist This document is being published jointly by the CERT Coordination Center and AusCERT (Australian Computer Emergency Response Team). printable version A. Introduction B. General Advice Pertaining to Intrusion Detection C. Look for Signs that Your System may have been Compromised 1. A Word on Rootkits 2. Examine Log Files 3. Check for Odd User Accounts and Groups 4. Check All Groups for Unexpected User Membership 5. Look for Unauthorized User Rights 6. Check for Unauthorized Applications Starting Automatically 7. Check Your System Binaries for Alterations 8. Check Your Network Configurations for Unauthorized Entries 9. Check for Unauthorized Shares 10. Check for Any Jobs Scheduled to Run 11. Check for Unauthorized Processes 12. Look Throughout the System for Unusual or Hidden Files 13. Check for Altered Permissions on Files or Registry Keys 14. Check for Changes in User or Computer Policies 15. Ensure the System has not been Joined to a Different Domain 16. Audit for Intrusion Detection 17. Additional Information D. Consider Running Intrusion Detection Systems If Possible 1. Freeware/shareware Intrusion Detection Systems 2. Commercial Intrusion Detection Systems E. Review Other AusCERT and CERT Documents 1. Steps for Recovering from a Windows NT Compromise 2. Windows NT Configuration Guidelines 3. NIST Checklists F. Document Revision History A. Introduction This document outlines suggested steps for determining whether your Windows system has been compromised. System administrators can use this information to look for several types of break-ins. We also encourage you to review all sections of this document and modify your systems to address potential weaknesses.
    [Show full text]
  • WS-Biometric Devices Walkthrough How to Build a WS-BD Web Camera Service
    WS-Biometric Devices Walkthrough How to Build a WS-BD Web Camera Service For questions or comments, contact [email protected]. 1 Introduction Web Services for Biometric Devices, or WS-BD, is an open source command & control protocol specifically for biometric acquisition devices. Web services use protocols that underlie the web for machine to machine communication. WS-BD allows a target biometric sensor to be exposed to and controlled by a client(s) via a web service. It replaces the need for proprietary software/hardware (e.g. drivers, firewire/USB connectors), eliminates platform restrictions, and allows wired or wireless communication. With a focus on data acquisition, this RESTFUL service architecture affords biometric sensors of any modality communication with any device that is Internet-enabled. This document is written as a “quickstart” aid for development using the WS-Biometric Devices CSDv1.0 document. The specification can be accessed at https://www.oasis- open.org/committees/document.php?document_id=54815&wg_abbrev=biometrics. A .NET reference implementation exists to demonstrate one way to implement a WS-BD service. Libraries from the reference implementation will be used in this walkthrough to build a service. The complete .NET reference implementation can also be downloaded for free at http://www.nist.gov/itl/iad/ig/upload/WS-BD-RefImpl- Jan2015.zip. 1.1 Overview This document provides step by step instructions and source code on how to construct a WS-BD conformant web camera service. It uses a commercial off-the-shelf (COTS) web camera as the biometric sensor. The intent is to provide a quick start to WS-BD development as well as to shorten future WS-BD development time.
    [Show full text]
  • Vbscripting For
    Paper AD09 Integrating Microsoft® VBScript and SAS® Christopher Johnson, BrickStreet Insurance ABSTRACT VBScript and SAS are each powerful tools in their own right. These two technologies can be combined so that SAS code can call a VBScript program or vice versa. This gives a programmer the ability to automate SAS tasks, traverse the file system, send emails programmatically, manipulate Microsoft® Word, Excel, and PowerPoint files, get web data, and more. This paper will present example code to demonstrate each of these capabilities. Contents Abstract .......................................................................................................................................................................... 1 Introduction .................................................................................................................................................................... 2 Getting Started ............................................................................................................................................................... 2 VBScript Running SAS ................................................................................................................................................... 2 Creating and Running Code ....................................................................................................................................... 2 Running Existing Code ..............................................................................................................................................
    [Show full text]
  • Microsoft Palladium
    Microsoft Palladium: A Business Overview Combining Microsoft Windows Features, Personal Computing Hardware, and Software Applications for Greater Security, Personal Privacy, and System Integrity by Amy Carroll, Mario Juarez, Julia Polk, Tony Leininger Microsoft Content Security Business Unit June 2002 Legal Notice This is a preliminary document and may be changed substantially prior to final commercial release of the software described herein. The information contained in this document represents the current view of Microsoft Corporation on the issues discussed as of the date of publication. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information presented after the date of publication. This White Paper is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, AS TO THE INFORMATION IN THIS DOCUMENT. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation. Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property. Unless otherwise noted, the example companies, organizations, products, domain names, e-mail addresses, logos, people, places and events depicted herein are fictitious, and no association with any real company, organization, product, domain name, e-mail address, logo, person, place or event is intended or should be inferred.
    [Show full text]
  • Datastead Multipurpose Directshow Encoder SDK Version 1.1.2
    DATASTEAD SOFTWARE Multipurpose DirectShow Encoder version 1.7.8 March 3, 2021 Copyright © Datastead 2021 www.datastead.com Overview......................................................................................................................................................3 Features...................................................................................................................................................3 System requirements...............................................................................................................................4 Download.................................................................................................................................................4 License.....................................................................................................................................................4 Contact.....................................................................................................................................................4 Limitations of the evaluation version........................................................................................................4 FAQ..............................................................................................................................................................5 Licensing..................................................................................................................................................5 Evaluation version....................................................................................................................................5
    [Show full text]
  • Choosing the Right Media Paths to Facilitate Hardware Acceleration
    White Paper Fan, Fred Choosing the Right Graphics Platform Application Engineer Media Paths to Intel Corporation Tan, Hui Li Facilitate Graphics Platform Application Engineer Hardware Intel Corporation Acceleration Media Solution for Retail Independent Software Vendors (ISVs) September, 2013 329650-001US Choosing the Right Media Paths to Facilitate Hardware Acceleration Executive Summary This paper provides detailed guidance for choosing the right media- development Application Programming Interfaces (APIs) for different retail usage models and devices on Windows* 7 and Windows* 8 operating systems (OSs). When Independent Software Vendors (ISVs) make decisions on which approach to take for their own media applications, there are five key factors that may influence their decisions: hardware investment, software resource investment, performance, flexibility, and software implementation speed. This paper provides detailed guidance for choosing the right media development APIs for different retail segments, from Entry to Mainstream and High-end. This paper does not include implementation details, but does provide related media references, such as Intel® Media SDK and other media development APIs, to help ISVs develop their media content as the next step. The Intel® Embedded Design Center provides qualified developers with web-based access to technical resources. Access Intel Confidential design materials, step-by-step guidance, application reference solutions, training, Intel’s tool loaner program, and connect with an e-help desk and the
    [Show full text]