DOCSLIB.ORG

Distributed Tracing in Practice Instrumenting, Analyzing, and Debugging Microservices

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Distributed Tracing in Practice Instrumenting, Analyzing, and Debugging Microservices Distributed Tracing in Practice Instrumenting, Analyzing, and Debugging Microservices Austin Parker, Daniel Spoonhower, Jonathan Mace, and Rebecca Isaacs with Ben Sigelman Beijing Boston Farnham Sebastopol Tokyo Distributed Tracing in Practice by Austin Parker, Daniel Spoonhower, Jonathan Mace, and Rebecca Isaacs, with Ben Sigelman Copyright © 2020 Ben Sigelman, Austin Parker, Daniel Spoonhower, Jonathan Mace, and Rebecca Isaacs. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or [email protected]. Acquisitions Editor: John Devins Indexer: Sue Klefstad Development Editor: Sarah Grey Interior Designer: David Futato Production Editor: Katherine Tozer Cover Designer: Karen Montgomery Copyeditor: Chris Morris Illustrator: Rebecca Demarest Proofreader: JM Olejarz April 2020: First Edition Revision History for the First Edition 2020-04-13 First Release See http://oreilly.com/catalog/errata.csp?isbn=9781492056638 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Distributed Tracing in Practice, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the authors, and do not represent the publisher’s views. While the publisher and the authors have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights. This work is part of a collaboration between O’Reilly and Lightstep. See our statement of editorial inde‐ pendence. 978-1-492-09503-3 [LSI] Table of Contents Foreword. xi Introduction: What Is Distributed Tracing?. xv 1. The Problem with Distributed Tracing. 1 The Pieces of a Distributed Tracing Deployment 3 Distributed Tracing, Microservices, Serverless, Oh My! 4 The Benefits of Tracing 6 Setting the Table 7 2. An Ontology of Instrumentation. 9 White Box Versus Black Box 10 Application Versus System 13 Agents Versus Libraries 15 Propagating Context 16 Interprocess Propagation 18 Intraprocess Propagation 20 The Shape of Distributed Tracing 23 Tracing-Friendly Microservices and Serverless 23 Tracing in a Monolith 25 Tracing in Web and Mobile Clients 27 3. Open Source Instrumentation: Interfaces, Libraries, and Frameworks. 31 The Importance of Abstract Instrumentation 32 OpenTelemetry 34 OpenTracing and OpenCensus 43 OpenTracing 43 OpenCensus 48 v Other Notable Formats and Projects 53 X-Ray 53 Zipkin 54 Interoperability and Migration Strategies 54 Why Use Open Source Instrumentation? 57 Interoperability 58 Portability 58 Ecosystem and Implicit Visibility 59 4. Best Practices for Instrumentation. 61 Tracing by Example 61 Installing the Sample Application 62 Adding Basic Distributed Tracing 62 Custom Instrumentation 70 Where to Start—Nodes and Edges 71 Framework Instrumentation 72 Service Mesh Instrumentation 75 Creating Your Service Graph 76 What’s in a Span? 79 Effective Naming 79 Effective Tagging 80 Effective Logging 81 Understanding Performance Considerations 82 Trace-Driven Development 85 Developing with Traces 86 Testing with Traces 89 Creating an Instrumentation Plan 91 Making the Case for Instrumentation 91 Instrumentation Quality Checklist 93 Knowing When to Stop Instrumenting 95 Smart and Sustainable Instrumentation Growth 96 5. Deploying Tracing. 99 Organizational Adoption 100 Start Close to Your Users 100 Start Centrally: Load Balancers and Gateways 101 Leverage Infrastructure: RPC Frameworks and Service Meshes 102 Make Adoption Repeatable 103 Tracer Architecture 104 In-Process Libraries 105 Sidecars and Agents 106 Collectors 107 vi | Table of Contents Centralized Storage and Analysis 108 Incremental Deployment 109 Data Provenance, Security, and Federation 110 Frontend Service Telemetry 110 Server-Side Telemetry for Managed Services 114 6. Overhead, Costs, and Sampling. 117 Application Overhead 118 Latency 118 Throughput 120 Infrastructure Costs 122 Network 122 Storage 123 Sampling 124 Minimum Requirements 124 Strategies 126 Selecting Traces 130 Off-the-Shelf ETL Solutions 131 7. A New Observability Scorecard. 135 The Three Pillars Defined 136 Metrics 136 Logging 138 Distributed Tracing 139 Fatal Flaws of the Three Pillars 140 Design Goals 141 Assessing the Three Pillars 142 Three Pipes (Not Pillars) 144 Observability Goals and Activities 145 Two Goals in Observability 145 Two Fundamental Activities in Observability 146 A New Scorecard 148 The Path Ahead 152 8. Improving Baseline Performance. 153 Measuring Performance 154 Percentiles 156 Histograms 158 Defining the Critical Path 160 Approaches to Improving Performance 163 Individual Traces 163 Biased Sampling and Trace Comparison 165 Table of Contents | vii Trace Search 167 Multimodal Analysis 169 Aggregate Analysis 171 Correlation Analysis 173 9. Restoring Baseline Performance. 179 Defining the Problem 180 Human Factors 182 (Avoiding) Finger-Pointing 182 “Suppressing” the Messenger 183 Incident Hand-off 184 Good Postmortems 184 Approaches to Restoring Performance 185 Integration with Alerting Workflows 185 Individual Traces 186 Biased Sampling 187 Real-Time Response 189 Knowing What’s Normal 191 Aggregate and Correlation Root Cause Analysis 195 10. Are We There Yet? The Past and Present. 201 Distributed Tracing: A History of Pragmatism 202 Request-Based Systems 202 Response Time Matters 202 Request-Oriented Information 203 Notable Work 203 Pinpoint 204 Magpie 204 X-Trace 206 Dapper 207 Where to Next? 208 11. Beyond Individual Requests. 209 The Value of Traces in Aggregate 211 Example 1: Is Network Congestion Affecting My Application? 211 Example 2: What Services Are Required to Serve an API Endpoint? 211 Organizing the Data 212 A Strawperson Solution 212 What About the Trade-offs? 214 Sampling for Aggregate Analysis 214 The Processing Pipeline 215 Incorporating Heterogeneous Data 217 viii | Table of Contents Custom Functions 217 Joining with Other Data Sources 218 Recap and Case Study 219 The Value of Traces in Aggregate 219 Organizing the Data 220 Sampling for Aggregate Analysis 220 The Processing Pipeline 221 Incorporating Heterogeneous Data 221 12. Beyond Spans. 223 Why Spans Have Prevailed 223 Visibility 223 Pragmatism 224 Portability 224 Compatibility 225 Flexibility 225 Why Spans Aren’t Enough 225 Graphs, Not Trees 226 Inter-Request Dependencies 227 Decoupled Dependencies 228 Distributed Dataflow 229 Machine Learning 230 Low-Level Performance Metrics 231 New Abstractions 232 Seeing Causality 234 13. Beyond Distributed Tracing. 237 Limitations of Distributed Tracing 238 Challenge 1: Anticipating Problems 239 Challenge 2: Completeness Versus Costs 240 Challenge 3: Open-Ended Use Cases 240 Other Tools Like Distributed Tracing 241 Census 241 A Motivating Example 242 A Distributed Tracing Solution? 243 Tag Propagation and Local Metric Aggregation 244 Comparison to Distributed Tracing 245 Pivot Tracing 246 Dynamic Instrumentation 246 Recurring Problems 247 How Does It Work? 247 Dynamic Context 248 Table of Contents | ix Comparison to Distributed Tracing 248 Pythia 249 Performance Regressions 249 Design 251 Overheads 251 Comparison to Distributed Tracing 251 14. The Future of Context Propagation. 253 Cross-Cutting Tools 253 Use Cases 254 Distributed Tracing 254 Cross-Component Metrics 255 Cross-Component Resource Management 255 Managing Data Quality Trade-offs 256 Failure Testing of Microservices 257 Enforcing Cross-System Consistency 258 Request Duplication 258 Record Lineage in Stream Processing Systems 259 Auditing Security Policies 259 Testing in Production 259 Common Themes 260 Should You Care? 260 The Tracing Plane 261 Is Baggage Enough? 262 Beyond Key-Value Pairs 264 Compiling BDL 265 BaggageContext 266 Merging 266 Overheads 266 A. The State of Distributed Tracing Circa 2020. 269 B. Context Propagation in OpenTelemetry. 275 Bibliography. 281 Index. 285 x | Table of Contents Foreword Human beings have struggled to understand production software for exactly as long as human beings have had production software. We have these marvelously fast machines, but they don’t speak our language and—despite their speed and all of the hype about artificial intelligence—they are still entirely unreflective and opaque. For many (many) decades, our efforts to understand production software ultimately boiled down to two types of telemetry data: log data and time series statistics. The time series data—also known as metrics—helped us understand that “something ter‐ rible” was happening inside of our computers. If we were lucky, the logging data would help us understand specifically what that terrible thing was. But then everything changed: our software needed more than just one computer. In fact, it needed thousands of them. We broke the software into tiny,
Load more

Recommended publications
  • Hacks, Cracks, and Crime: an Examination of the Subculture and Social Organization of Computer Hackers Thomas Jeffrey Holt University of Missouri-St
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by University of Missouri, St. Louis University of Missouri, St. Louis IRL @ UMSL Dissertations UMSL Graduate Works 11-22-2005 Hacks, Cracks, and Crime: An Examination of the Subculture and Social Organization of Computer Hackers Thomas Jeffrey Holt University of Missouri-St. Louis, [email protected] Follow this and additional works at: https://irl.umsl.edu/dissertation Part of the Criminology and Criminal Justice Commons Recommended Citation Holt, Thomas Jeffrey, "Hacks, Cracks, and Crime: An Examination of the Subculture and Social Organization of Computer Hackers" (2005). Dissertations. 616. https://irl.umsl.edu/dissertation/616 This Dissertation is brought to you for free and open access by the UMSL Graduate Works at IRL @ UMSL. It has been accepted for inclusion in Dissertations by an authorized administrator of IRL @ UMSL. For more information, please contact [email protected]. Hacks, Cracks, and Crime: An Examination of the Subculture and Social Organization of Computer Hackers by THOMAS J. HOLT M.A., Criminology and Criminal Justice, University of Missouri- St. Louis, 2003 B.A., Criminology and Criminal Justice, University of Missouri- St. Louis, 2000 A DISSERTATION Submitted to the Graduate School of the UNIVERSITY OF MISSOURI- ST. LOUIS In partial Fulfillment of the Requirements for the Degree DOCTOR OF PHILOSOPHY in Criminology and Criminal Justice August, 2005 Advisory Committee Jody Miller, Ph. D. Chairperson Scott H. Decker, Ph. D. G. David Curry, Ph. D. Vicki Sauter, Ph. D. Copyright 2005 by Thomas Jeffrey Holt All Rights Reserved Holt, Thomas, 2005, UMSL, p.
    [Show full text]
  • Advanced-Game-Design-With-Html5
    www.allitebooks.com For your convenience Apress has placed some of the front matter material after the index. Please use the Bookmarks and Contents at a Glance links to access them. www.allitebooks.com Contents at a Glance About the Author .....................................................................................................xv About the Technical Reviewers .............................................................................xvii Acknowledgments ..................................................................................................xix Introduction ............................................................................................................xxi ■ Chapter 1: Level Up! .............................................................................................. 1 ■ Chapter 2: The Canvas Drawing API .................................................................... 59 ■ Chapter 3: Working with Game Assets ................................................................ 93 ■ Chapter 4: Making Sprites and a Scene Graph .................................................. 111 ■ Chapter 5: Making Things Move ........................................................................ 165 ■ Chapter 6: Interactivity ...................................................................................... 189 ■ Chapter 7: Collision Detection ........................................................................... 239 ■ Chapter 8: Juice It Up .......................................................................................
    [Show full text]
  • Computations in Algebraic Geometry with Macaulay 2
    Computations in algebraic geometry with Macaulay 2 Editors: D. Eisenbud, D. Grayson, M. Stillman, and B. Sturmfels Preface Systems of polynomial equations arise throughout mathematics, science, and engineering. Algebraic geometry provides powerful theoretical techniques for studying the qualitative and quantitative features of their solution sets. Re- cently developed algorithms have made theoretical aspects of the subject accessible to a broad range of mathematicians and scientists. The algorith- mic approach to the subject has two principal aims: developing new tools for research within mathematics, and providing new tools for modeling and solv- ing problems that arise in the sciences and engineering. A healthy synergy emerges, as new theorems yield new algorithms and emerging applications lead to new theoretical questions. This book presents algorithmic tools for algebraic geometry and experi- mental applications of them. It also introduces a software system in which the tools have been implemented and with which the experiments can be carried out. Macaulay 2 is a computer algebra system devoted to supporting research in algebraic geometry, commutative algebra, and their applications. The reader of this book will encounter Macaulay 2 in the context of concrete applications and practical computations in algebraic geometry. The expositions of the algorithmic tools presented here are designed to serve as a useful guide for those wishing to bring such tools to bear on their own problems. A wide range of mathematical scientists should find these expositions valuable. This includes both the users of other programs similar to Macaulay 2 (for example, Singular and CoCoA) and those who are not interested in explicit machine computations at all.
    [Show full text]
  • Cracking Software
    Cracking software click here to download Using this, you can completely bypass the registration process by making it skip the application's key code verification process without using a valid key. In this Null Byte, let's go over how cracking could work in practice by looking at an example program (a program that serves. A few password cracking tools use a dictionary that contains passwords. These tools .. Looking for SSH login password crack software. Reply. This is just for learning. Softwares used: W32Dasm HIEW32 My New Tutorial Link - (Intro to Crackin using. HashCat claims to be the fastest and most advanced password cracking software available. Released as a free and open source software. In order to crack most software, you will need to have a good grasp on assembly, which is a low-level programming language. Assembly is derived from machine. The term crack is also commonly applied to the files used in software cracking programs, which enable illegal copying and the use of commercial software by. If your losst ypur passwords, you can try to crack your operating system and application passwords with various password‐cracking tools. LiveCD available to simplify the cracking.» Dumps and loads hashes from encrypted SAM recovered from a Windows partition.» Free and open source software. Hacking and Cracking -Software key and many more. 33K likes. HACKING TRICKS #PATCH SOFTWARE #ANDROID APPS PRO #MAC OS #WINDOWS APPS. Not everyone can crack a software because doing that requires a lot of computer knowledge but here we show you a logical method on how to. Crack means the act of breaking into a computer system.
    [Show full text]
  • Warez All That Pirated Software Coming From?
    Articles http://www.informit.com/articles/printerfriendly.asp?p=29894 Warez All that Pirated Software Coming From? Date: Nov 1, 2002 By Seth Fogie. Article is provided courtesy of Prentice Hall PTR. In this world of casual piracy, many people have forgotten or just never realized where many software releases originate. Seth Fogie looks at the past, present, and future of the warez industry; and illustrates the simple fact that "free" software is here to stay. NOTE The purpose of this article is to provide an educational overview of warez. The author is not taking a stance on the legality, morality, or any other *.ality on the issues surrounding the subject of warez and pirated software. In addition, no software was pirated, cracked, or otherwise illegally obtained during the writing of this article. Software piracy is one of the hottest subjects in today's computerized culture. With the upheaval of Napster and the subsequent spread of peer-to-peer programs, the casual sharing of software has become a world-wide pastime. All it takes is a few minutes on a DSL connection, and KazaA (or KazaA-Lite for those people who don't want adware) and any 10-year-old kid can have the latest pop song hit in their possession. As if deeply offending the music industry isn't enough, the same avenues taken to obtain cheap music also holds a vast number of software games and applications—some worth over $10,000. While it may be common knowledge that these items are available online, what isn't commonly known is the complexity of the process that many of these "releases" go through before they hit the file-sharing mainstream.
    [Show full text]
  • The Pretzel User Manual
    The PretzelBook second edition Felix G¨artner June 11, 1998 2 Contents 1 Introduction 5 1.1 Do Prettyprinting the Pretzel Way . 5 1.2 History . 6 1.3 Acknowledgements . 6 1.4 Changes to second Edition . 6 2 Using Pretzel 7 2.1 Getting Started . 7 2.1.1 A first Example . 7 2.1.2 Running Pretzel . 9 2.1.3 Using Pretzel Output . 9 2.2 Carrying On . 10 2.2.1 The Two Input Files . 10 2.2.2 Formatted Tokens . 10 2.2.3 Regular Expressions . 10 2.2.4 Formatted Grammar . 11 2.2.5 Prettyprinting with Format Instructions . 12 2.2.6 Formatting Instructions . 13 2.3 Writing Prettyprinting Grammars . 16 2.3.1 Modifying an existing grammar . 17 2.3.2 Writing a new Grammar from Scratch . 17 2.3.3 Context Free versus Context Sensitive . 18 2.3.4 Available Grammars . 19 2.3.5 Debugging Prettyprinting Grammars . 19 2.3.6 Experiences . 21 3 Pretzel Hacking 23 3.1 Adding C Code to the Rules . 23 3.1.1 Example for Tokens . 23 3.1.2 Example for Grammars . 24 3.1.3 Summary . 25 3.1.4 Tips and Tricks . 26 3.2 The Pretzel Interface . 26 3.2.1 The Prettyprinting Scanner . 27 3.2.2 The Prettyprinting Parser . 28 3.2.3 Example . 29 3.3 Building a Pretzel prettyprinter by Hand . 30 3.4 Obtaining a Pretzel Prettyprinting Module . 30 3.4.1 The Prettyprinting Scanner . 30 3.4.2 The Prettyprinting Parser . 31 3.5 Multiple Pretzel Modules in the same Program .
    [Show full text]
  • Foundations for Music-Based Games
    Die approbierte Originalversion dieser Diplom-/Masterarbeit ist an der Hauptbibliothek der Technischen Universität Wien aufgestellt (http://www.ub.tuwien.ac.at). The approved original version of this diploma or master thesis is available at the main library of the Vienna University of Technology (http://www.ub.tuwien.ac.at/englweb/). MASTERARBEIT Foundations for Music-Based Games Ausgeführt am Institut für Gestaltungs- und Wirkungsforschung der Technischen Universität Wien unter der Anleitung von Ao.Univ.Prof. Dipl.-Ing. Dr.techn. Peter Purgathofer und Univ.Ass. Dipl.-Ing. Dr.techn. Martin Pichlmair durch Marc-Oliver Marschner Arndtstrasse 60/5a, A-1120 WIEN 01.02.2008 Abstract The goal of this document is to establish a foundation for the creation of music-based computer and video games. The first part is intended to give an overview of sound in video and computer games. It starts with a summary of the history of game sound, beginning with the arguably first documented game, Tennis for Two, and leading up to current developments in the field. Next I present a short introduction to audio, including descriptions of the basic properties of sound waves, as well as of the special characteristics of digital audio. I continue with a presentation of the possibilities of storing digital audio and a summary of the methods used to play back sound with an emphasis on the recreation of realistic environments and the positioning of sound sources in three dimensional space. The chapter is concluded with an overview of possible categorizations of game audio including a method to differentiate between music-based games.
    [Show full text]
  • Hitachi Unified Storage File Module File Services Administration Guide
    Hitachi Unified Storage File Module File Services Administration Guide Release 12.5 MK-92HUSF004-09 December 2015 © 2011-2015 Hitachi, Ltd. All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording, or stored in a database or retrieval system for any purpose without the express written permission of Hitachi, Ltd. Hitachi, Ltd., reserves the right to make changes to this document at any time without notice and assumes no responsibility for its use. This document contains the most current information available at the time of publication. When new or revised information becomes available, this entire document will be updated and distributed to all registered users. Some of the features described in this document might not be currently available. Refer to the most recent product announcement for information about feature and product availability, or contact Hitachi Data Systems Corporation at https://portal.hds.com. Notice: Hitachi, Ltd., products and services can be ordered only under the terms and conditions of the applicable Hitachi Data Systems Corporation agreements. The use of Hitachi, Ltd., products is governed by the terms of your agreements with Hitachi Data Systems Corporation. 2 Hitachi Unified Storage File Module File Services Administration Guide Hitachi Data Systems products and services can be ordered only under the terms and conditions of Hitachi Data Systems’ applicable agreements. The use of Hitachi Data Systems products is governed by the terms of your agreements with Hitachi Data Systems. By using this software, you agree that you are responsible for: a) Acquiring the relevant consents as may be required under local privacy laws or otherwise from employees and other individuals to access relevant data; and b) Verifying that data continues to be held, retrieved, deleted, or otherwise processed in accordance with relevant laws.
    [Show full text]
  • Software Import and Contacts Between European and American Cracking Scenes
    “Amis and Euros.” Software Import and Contacts Between European and American Cracking Scenes Patryk Wasiak Dr., Researcher [email protected] Institute for Cultural Studies, University of Wroclaw, Poland Abstract This article explores practices of so-called “software import” between Europe and the US by the Commodore 64 cracking scene and reflects on how it was related to the establishment and expression of cultural identities of its members. While discussing the establishment of computer games’ transatlantic routes of distribution with Bulletin Board System (BBS) boards, I will explore how this phenomenon, providing the sceners with new challenges and, subsequently, new roles such as importer, modem trader or NTSC-fixer, influenced the rise of new cultural identities within the cracking scene. Furthermore, I will analyze how the import of software and the new scene roles influenced the expression of cultural distinction between American and European sceners, referred to as “Euros” and “Amis” on the scene forum. Keywords: Europe, United States, crackers, cultural identity, modem, software piracy Introduction The aim of this article is to provide a historical inquiry into cultural practices related to the so called “software import” between Europe and the US, within the framework of the Commodore 64 cracking scene. In the mid-1980s several crackers from European countries and the US established direct and regular contacts with the computer modem Bulletin Board System (BBS) boards. In the existing studies the cracking scene is considered primarily as a social world of crackers (Vuorinen 2007), a particular form of hacker culture (Thomas 2003; Sterling 1992). This study intends to broaden the existing academic work by considering how the transatlantic circulation of cracked software and its appropriation influenced the establishment of cultural identities.
    [Show full text]
  • Meanings That Hackers Assign to Their Being a Hacker - Orly Turgeman-Goldschmidt
    Meanings that Hackers Assign to their Being a Hacker - Orly Turgeman-Goldschmidt Copyright © 2008 International Journal of Cyber Criminology (IJCC) ISSN: 0974 – 2891 July - December 2008, Vol 2 (2): 382–396 This is an Open Access article distributed under the terms of the Creative Commons Attribution-Non-Commercial-Share Alike License, which permits unrestricted non- commercial use, distribution, and reproduction in any medium, provided the original work is properly cited. This license does not permit commercial exploitation or the creation of derivative works without specific permission. Meanings that Hackers Assign to their Being a Hacker 1 Orly Turgeman-Goldschmidt Bar-Ilan University, Israel Abstract This study analyzes the ways in which hackers interpret their lives, behavior, and beliefs, as well as their perceptions of how society treats them. The study was based on unstructured, face-to-face interviews with fifty-four Israeli hackers who were asked to tell their life stories. Analysis of the data reveals differences in the hackers’ self-presentation and the extent of their hacking activity. Although these differences imply the importance of informal labeling since childhood, it seems that hackers succeed in avoiding both, the effects of labeling and secondary deviance and that they feel no shame. Furthermore, they structure their identities as positive deviants and acquire the identity of breakers of boundaries, regardless of the number and severity of the computer offenses they have committed. Keywords: hackers; crackers; hacking; labeling; positive deviant; construction of identity. Introduction Computer-related deviance has not been sufficiently studied, especially from the perspective of the perpetrators themselves (Yar, 2005). The present study analyzes the ways in which hackers interpret their lives, behavior, and beliefs, as well as their perceptions of how society treats them.
    [Show full text]
  • Files I Have Known
    FILES I HAVE KNOWN FOREWORD The data reminiscences that follow are probably an unrewarding read. Published reminiscences stand as monuments to the ego of their authors. However, the true purpose of this text is to experimentally substitute digital files with written memories of those files, prompted by metadata. Can the original essence of a data file be recreated purely by words? Many of the files chosen here are obscure and not necessarily available online. It’s likely that the files themselves are of little interest if discovered without their context, forming only fragments of nonsensicality in the wider noise of online data. These reminiscences too may appear as yet further noise flung onto an internet already overpopulated with noise vying for visibility. But by intimately analysing one’s own files in terms of their emotional impact, the reader may come to appreciate that every file has a story. In our daily dealings with data, we partake in the creation of such stories, and the stories also create us. FILES I HAVE KNOWN CHAPTER I “dontryathome.zip” Size: 345KB (345,410 bytes) Created: 31 December 2004 13:53 Format: ZIP Archive “dontryathome.zip” was a compressed archive containing images randomly taken from the web. It’s a file I no longer possess, so all that exists of it now is the above scrap of metadata, the text of the email it was attached to, and the memory. This may well be the eventual fate of all data – future generations might only be left with metadata and memories to reminiscence over if either (a) the pace of technology outstrips the means to preserve and open obsolete file formats, or (b) a sudden cataclysmic environmental or economic global disaster annuls electronic technologies.
    [Show full text]
  • GOOL: a Generic Object-Oriented Language (Extended Version)
    GOOL: A Generic Object-Oriented Language (extended version) Jacques Carette Brooks MacLachlan W. Spencer Smith Department of Computing and Department of Computing and Department of Computing and Software Software Software McMaster University McMaster University McMaster University Hamilton, Ontario, Canada Hamilton, Ontario, Canada Hamilton, Ontario, Canada [email protected] [email protected] [email protected] Abstract it to the others. Of course, this can be made to work — one We present GOOL, a Generic Object-Oriented Language. It could engineer a multi-language compiler (such as gcc) to demonstrates that a language, with the right abstractions, de-compile its Intermediate Representation (IR) into most can capture the essence of object-oriented programs. We of its input languages. The end-results would however be show how GOOL programs can be used to generate human- wildly unidiomatic; roughly the equivalent of a novice in a readable, documented and idiomatic source code in multi- new (spoken) language “translating” word-by-word. ple languages. Moreover, in GOOL, it is possible to express What if, instead, there was a single meta-language de- common programming idioms and patterns, from simple signed to embody the common semantic concepts of a num- library-level functions, to simple tasks (command-line argu- ber of OO languages, encoded so that the necessary informa- ments, list processing, printing), to more complex patterns, tion for translation is present? This source language could such as methods with a mixture of input, output and in-out be agnostic about what eventual target language will be used parameters, and finally Design Patterns (such as Observer, – and free of the idiosyncratic details of any given language.
    [Show full text]