DOCSLIB.ORG

Persistent Memory Programming Made Easy with Pmemkv

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Persistent Memory Programming Made Easy with Pmemkv Persistent Memory Programming Made Easy with pmemkv Andy Rudoff Intel Corporation 2019 Storage Developer Conference. © Intel Corporation. All Rights Reserved. 1 Agenda . Why pmemkv? . pmemkv Design . Engines . Language bindings . Performance 2019 Storage Developer Conference. © Intel Corporation. All Rights Reserved. 2 Why pmemkv? 2019 Storage Developer Conference. © Intel Corporation. All Rights Reserved. 3 Ways to Use Persistent Memory . Memory Mode . Transparent to application . Transparent to OS . Volatile . …but not a match for every use case . As Storage… 2019 Storage Developer Conference. © Intel Corporation. All Rights Reserved. 4 The SNIA NVM Programming Model file memory Management UI Application Application Application space user Standard Standard Standard Load/ Raw Device File API File API Store Management Library Access “DAX” File System space kernel pmem-Aware MMU File System Mappings Generic NVDIMM Driver Persistent Memory 2019 Storage Developer Conference. © Intel Corporation. All Rights Reserved. 5 With Great Direct Access Comes Great Responsibility Flushing… RAS… Program Initialization Program Initialization DAX mapped file? Dirty Shutdown? (OS provides info) no yes yes no Data set is potentially inconsistent. Use standard API for flushing Recover. (msync/fsync or FlushFileBuffers) CPU caches Known Poison Blocks considered persistent? (ACPI provides info) yes no no yes Repair data set Normal Operation Stores considered persistent CLWB? when globally-visible (CPU_ID provides info) no yes Use CLWB+SFENCE Consistency… CLFLUSHOPT? for flushing (CPU_ID provides info) no yes open(…); mmap(…); Use CLFLUSHOPT+SFENCE Use CLFLUSH for flushing for flushing strcpy(pmem, "Hello, World!"); msync(…); Crash 2019 Storage Developer Conference. © Intel Corporation. All Rights Reserved. 6 PMDK “make pmem programming easier” PCJ – Persistent Collection for PCJ/ C++ C Python pmemkv Java LLPL C+ Java JS Ruby C + Application Load/Store Standard File API User Space PMDK Interface to create a Interface for persistent memory Transaction persistent memory resident allocation, transactions and general Interface to create arrays of facilities Support log file pmem-resident blocks, of same size, atomically updated MMU pmem-Aware Mappings libpmemlog libpmemobj libpmemblk File System Kernel Space Support for volatile memory usage Low level support for NVDIMM Low level support for remote access to memkind local persistent memory persistent memory Low-level support vmemcache libpmem librpmem In Development 2019 Storage Developer Conference. © Intel Corporation. All Rights Reserved. 7 libpmemobj Performance Across Versions B-tree Benchmark 549.66% 511.33% 377.12% 510.59% 167.49% 351.09% 100.00% 199.07% 2019 Storage Developer Conference. © Intel Corporation. All Rights Reserved. 8 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 Why pmemkv? . We have Memory Mode and Storage . for legacy use cases . We have libpmem . for raw access . We have libpmemobj . For transactions, allocation, containers . How about a simple put/get API! . Tuned and validated to product quality 2019 Storage Developer Conference. © Intel Corporation. All Rights Reserved. 9 Why pmemkv? • Key-value store can take advantage of access-in-place 2019 Storage Developer Conference. © Intel Corporation. All Rights Reserved. 10 pmemkv design 2019 Storage Developer Conference. © Intel Corporation. All Rights Reserved. 11 pmemkv design goals Technical: Community: • Local key/value store (no networking) • Idiomatic language bindings • Open source, developed in the open and • Simple, familiar, bulletproof API friendly licensing • Easily extended with new engines • https://github.com/pmem/pmemkv • Optimized for persistent memory • Outside contributions are welcome (limit copying to/from DRAM) • Intel provides stewardship, validation on real • Flexible configuration, not limited to a single hardware, and code reviews storage algorithm • Standard/comparable benchmarks • Generic tests 2019 Storage Developer Conference. © Intel Corporation. All Rights Reserved. 12 pmemkv architecture applications • pmemkv core is a frontend for engines C++ JavaScript Java Ruby C applications applications applications applications applications • Core implementation written in C++, Node.js not related to Persistent Memory bindings • Pluggable engines • Some engines are implemented in bindings NAPI Java Ruby bindings bindings pmemkv, some engines are imported C++ API (header from external projects only) JNI FFI • Persistent engines are implemented with libpmemobj (PMDK) C API pmemkv • Native API for pmemkv is written C/C++ pmemkv core (C++) • pmemkv design allows for easy integration with high-level language libpmemobj++ pmemkv memkind TBB “native” bindings libpmemobj engines PMDK pluggable engines 2019 Storage Developer Conference. © Intel Corporation. All Rights Reserved. 13 pmemkv configuration • Flexible configuration API Typical config structure example for libpmemobj-based engines • Works with different kinds of engines • Every engine has documented supported config config cfg; parameters individually • Unordered map status s = cfg.put_string("path", path); assert(s == status::OK); • Takes name configuration value as a k-v pair • Supported configuration types: s = cfg.put_uint64("size", SIZE); • int64/uint64/double assert(s == status::OK); • string s = cfg.put_uint64("force_create", 1); • Arbitrary data (pointer and size) assert(s == status::OK); • Resides on stack • Takes optional destructor as an additional parameter if custom configuration parameter allocates memory 2019 Storage Developer Conference. © Intel Corporation. All Rights Reserved. 14 pmemkv design Application start DRAM Persistent Memory 2019 Storage Developer Conference. © Intel Corporation. All Rights Reserved. 15 pmemkv design config cfg; status s = cfg.put_string("path", "/daxfs/file"); DRAM Persistent Memory assert(s == status::OK); cfg s = cfg.put_uint64("size", SIZE); assert(s == status::OK); config structure, resides on stack 2019 Storage Developer Conference. © Intel Corporation. All Rights Reserved. 16 pmemkv design config cfg; status s = cfg.put_string("path", "/daxfs/file"); DRAM Persistent Memory assert(s == status::OK); cfg s = cfg.put_uint64("size", SIZE); assert(s == status::OK); kv db *kv = new db(); db object – volatile object for managing engine 2019 Storage Developer Conference. © Intel Corporation. All Rights Reserved. 17 pmemkv design config cfg; status s = cfg.put_string("path", "/daxfs/file"); DRAM Persistent Memory assert(s == status::OK); kv s = cfg.put_uint64("size", SIZE); /daxfs/file assert(s == status::OK); k v cfg cmap db *kv = new db(); if (kv->open("cmap", cfg) != status::OK) { std::cerr << db::errormsg() << std::endl; return 1; } kv.open() - creates/opens persistent memory pool - checks consistency and perform recovery - takes ownership for cfg structure 2019 Storage Developer Conference. © Intel Corporation. All Rights Reserved. 18 pmemkv design config cfg; status s = cfg.put_string("path", "/daxfs/file"); DRAM Persistent Memory assert(s == status::OK); kv s = cfg.put_uint64("size", SIZE); /daxfs/file assert(s == status::OK); k v cfg cmap db *kv = new db(); if (kv->open("cmap", cfg) != status::OK) { std::cerr << db::errormsg() << std::endl; return 1; } // do work here 2019 Storage Developer Conference. © Intel Corporation. All Rights Reserved. 19 pmemkv design config cfg; status s = cfg.put_string("path", "/daxfs/file"); DRAM Persistent Memory assert(s == status::OK); kv s = cfg.put_uint64("size", SIZE); /daxfs/file assert(s == status::OK); k v cfg cmap db *kv = new db(); if (kv->open("cmap", cfg) != status::OK) { std::cerr << db::errormsg() << std::endl; return 1; } // do work here kv.close() - close database connection kv->close(); - Persistent Memory data remain saved 2019 Storage Developer Conference. © Intel Corporation. All Rights Reserved. 20 pmemkv design config cfg; status s = cfg.put_string("path", "/daxfs/file"); DRAM Persistent Memory assert(s == status::OK); s = cfg.put_uint64("size", SIZE); /daxfs/file assert(s == status::OK); k v cmap db *kv = new db(); if (kv->open("cmap", cfg) != status::OK) { std::cerr << db::errormsg() << std::endl; return 1; } // do work here kv->close(); Safe deletion of volatile data delete kv; 2019 Storage Developer Conference. © Intel Corporation. All Rights Reserved. 21 Engines 2019 Storage Developer Conference. © Intel Corporation. All Rights Reserved. 22 Engines Engine Name Description Experimental? Persistent? Concurrent? Sorted? blackhole Accepts everything, returns nothing No - - - cmap Concurrent hash map No Yes Yes No vsmap Volatile sorted hash map No No No Yes vcmap Volatile concurrent hash map No No Yes No tree3 Persistent B+ tree Yes Yes No No stree Sorted persistent B+ tree Yes Yes No Yes Caching for remote Memcached or caching Yes Yes No - Redis server Sorted concurrent map csmap Yes Yes Yes Yes (under development) 2019 Storage Developer Conference. © Intel Corporation. All Rights Reserved. 23 Engines • Engine contributions are welcome! • Types: • ordered/unordered • persistent/volatile • concurrent/single threaded • Engines are optimized for different workloads & capabilities • All engines work with all language bindings • Generic tests for engines include: • memcheck • helgrind/drd • pmemcheck • pmemreorder 2019 Storage Developer Conference. © Intel Corporation. All Rights Reserved. 24 Multiple Engines Within the Same Memory Pool • pmemkv does not limit you to a single engine to a single memory pool Persistent Memory • Engines are reachable from root object /daxfs/file
Load more

Recommended publications
  • Debreach: Selective Dictionary Compression to Prevent BREACH and CRIME
    debreach: Selective Dictionary Compression to Prevent BREACH and CRIME A THESIS SUBMITTED TO THE FACULTY OF THE GRADUATE SCHOOL OF THE UNIVERSITY OF MINNESOTA BY Brandon Paulsen IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE Professor Peter A.H. Peterson July 2017 © Brandon Paulsen 2017 Acknowledgements First, I’d like to thank my advisor Peter Peterson and my lab mate Jonathan Beaulieu for their insights and discussion throughout this research project. Their contributions have undoubtedly improved this work. I’d like to thank Peter specifically for renew- ing my motivation when my project appeared to be at a dead–end. I’d like to thank Jonathan specifically for being my programming therapist. Next, I’d like to thank my family and friends for constantly supporting my aca- demic goals. In particular, I’d like to thank my mom and dad for their emotional support and encouragement. I’d like to thank my brother Derek and my friend Paul “The Wall” Vaynshenk for being great rock climbing partners, which provided me the escape from work that I needed at times. I’d like to again thank Jonathan Beaulieu and Xinru Yan for being great friends and for many games of Settlers of Catan. I’d like to thank Laura Krebs for helping me to discover my passion for academics and learning. Finally, I’d like to thank my fellow graduate students and the computer science faculty of UMD for an enjoyable graduate program. I’d also like to thank Professor Bethany Kubik and Professor Haiyang Wang for serving on my thesis committee.
    [Show full text]
  • Turbo.Lua Documentation Release 2.1.2
    Turbo.lua Documentation Release 2.1.2 John Abrahamsen Nov 02, 2018 Contents 1 Hello World 3 2 Supported Architectures 5 3 Supported Operating Systems7 4 Installation 9 5 Object oriented Lua 11 6 Packaging 13 7 Dependencies 15 8 License 17 9 Tutorials 19 9.1 Get Started With Turbo.......................................... 19 9.1.1 Installing Turbo......................................... 19 9.1.2 Hello World........................................... 20 9.1.3 Request parameters....................................... 20 9.1.4 Routes.............................................. 20 9.1.5 Serving Static Files....................................... 21 9.1.6 JSON Output.......................................... 22 9.2 Asynchronous modules......................................... 22 9.2.1 Overview............................................ 22 9.2.2 Example module........................................ 24 10 API documentation 27 10.1 Turbo.lua API Versioning........................................ 27 10.1.1 Preliminaries.......................................... 27 10.1.2 Module Version......................................... 27 10.2 turbo.web – Core web framework.................................... 28 10.2.1 RequestHandler class...................................... 28 10.2.2 HTTPError class........................................ 32 10.2.3 StaticFileHandler class..................................... 32 10.2.4 RedirectHandler class...................................... 33 10.2.5 Application class........................................ 33
    [Show full text]
  • Fundamental Data Structures Contents
    Fundamental Data Structures Contents 1 Introduction 1 1.1 Abstract data type ........................................... 1 1.1.1 Examples ........................................... 1 1.1.2 Introduction .......................................... 2 1.1.3 Defining an abstract data type ................................. 2 1.1.4 Advantages of abstract data typing .............................. 4 1.1.5 Typical operations ...................................... 4 1.1.6 Examples ........................................... 5 1.1.7 Implementation ........................................ 5 1.1.8 See also ............................................ 6 1.1.9 Notes ............................................. 6 1.1.10 References .......................................... 6 1.1.11 Further ............................................ 7 1.1.12 External links ......................................... 7 1.2 Data structure ............................................. 7 1.2.1 Overview ........................................... 7 1.2.2 Examples ........................................... 7 1.2.3 Language support ....................................... 8 1.2.4 See also ............................................ 8 1.2.5 References .......................................... 8 1.2.6 Further reading ........................................ 8 1.2.7 External links ......................................... 9 1.3 Analysis of algorithms ......................................... 9 1.3.1 Cost models ......................................... 9 1.3.2 Run-time analysis
    [Show full text]
  • Language and Framework Support for Reviewably- Secure Software Systems
    Language and Framework Support for Reviewably- Secure Software Systems Adrian Mettler Electrical Engineering and Computer Sciences University of California at Berkeley Technical Report No. UCB/EECS-2012-244 http://www.eecs.berkeley.edu/Pubs/TechRpts/2012/EECS-2012-244.html December 13, 2012 Copyright © 2012, by the author(s). All rights reserved. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission. Language and Framework Support for Reviewably-Secure Software Systems by Adrian Matthew Mettler A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Computer Science in the Graduate Division of the University of California, Berkeley Committee in charge: Professor David Wagner, Chair Professor Deirdre Mulligan Professor Dawn Song Fall 2012 Language and Framework Support for Reviewably-Secure Software Systems Copyright 2012 by Adrian Matthew Mettler 1 Abstract Language and Framework Support for Reviewably-Secure Software Systems by Adrian Matthew Mettler Doctor of Philosophy in Computer Science University of California, Berkeley Professor David Wagner, Chair My thesis is that languages and frameworks can and should be designed to make it easier for programmers to write reviewably secure systems. A system is reviewably secure if its security is easy for an experienced programmer to verify, given access to the source code.
    [Show full text]
  • The Checker Framework Manual: Custom Pluggable Types for Java
    The Checker Framework Manual: Custom pluggable types for Java https://checkerframework.org/ Version 2.5.7 (4 Nov 2018) For the impatient: Section 1.3 (page 13) describes how to install and use pluggable type-checkers. Contents 1 Introduction 12 1.1 How to read this manual . 13 1.2 How it works: Pluggable types . 13 1.3 Installation . 13 1.4 Example use: detecting a null pointer bug . 13 2 Using a checker 15 2.1 Writing annotations . 15 2.2 Running a checker . 16 2.2.1 Using annotated libraries . 16 2.2.2 Distributing your annotated project . 17 2.2.3 Summary of command-line options . 17 2.2.4 Checker auto-discovery . 19 2.2.5 Shorthand for built-in checkers . 19 2.3 What the checker guarantees . 19 2.4 Tips about writing annotations . 20 2.4.1 Write annotations before you run a checker . 20 2.4.2 How to get started annotating legacy code . 20 2.4.3 Annotations indicate non-exceptional behavior . 22 2.4.4 Subclasses must respect superclass annotations . 22 2.4.5 Annotations on constructor invocations . 23 2.4.6 What to do if a checker issues a warning about your code . 24 3 Nullness Checker 26 3.1 What the Nullness Checker checks . 26 3.2 Nullness annotations . 27 3.2.1 Nullness qualifiers . 27 3.2.2 Nullness method annotations . 28 3.2.3 Initialization qualifiers . 28 3.2.4 Map key qualifiers . 28 3.3 Writing nullness annotations . 29 3.3.1 Implicit qualifiers . 29 3.3.2 Default annotation .
    [Show full text]
  • Improving Scalability of Symbolic Execution for Software with Complex Environment Interfaces
    Improving Scalability of Symbolic Execution for Software with Complex Environment Interfaces THÈSE NO 6719 (2015) PRÉSENTÉE LE 13 JUILLET 2015 À LA FACULTÉ INFORMATIQUE ET COMMUNICATIONS LABORATOIRE DES SYSTEMES FIABLES PROGRAMME DOCTORAL EN INFORMATIQUE ET COMMUNICATIONS ÉCOLE POLYTECHNIQUE FÉDÉRALE DE LAUSANNE POUR L'OBTENTION DU GRADE DE DOCTEUR ÈS SCIENCES PAR Stefan, BUCUR acceptée sur proposition du jury: Prof. J. R. Larus, président du jury Prof. G. Candea, directeur de thèse Prof. V. Adve, rapporteur Prof. J. Kinder, rapporteur Prof. W. Zwaenepoel, rapporteur Suisse 2015 2 Abstract (German) Manuelles Testen von Software ist aufwändig und fehleranfällig. Dennoch ist es die unter Fach- leuten beliebteste Methode zur Qualitätssicherung. Die Automatisierung des Testprozesses ver- spricht eine höhere Effektivität insbesondere zum Auffinden von Fehlern in Randfällen. Sym- bolische Softwareausführung zeichnet sich als automatische Testtechnik dadurch aus, dass sie keine falsch positiven Resultate hat, mögliche Programmausführungen abschliessend aufzählt, und besonders interessante Ausführungen prioritisieren kann. In der Praxis erschwert jedoch die so- genannte Path Explosion – die Tatsache, dass die Anzahl Programmausführungen im Verhältnis zur Programmgrösse exponentiell ansteigt – die Anwendung von Symbolischer Ausführung, denn Software besteht heutzutage oft aus Millionen von Zeilen Programmcode. Um Software effizient symbolisch zu testen, nutzen Entwickler die Modularität der Software und testen die einzelnen Systemkomponenten separat. Eine Komponente benötigt jedoch eine Umgebung, in der sie ihre Aufgabe erfüllen kann. Die Schnittstelle zu dieser Umgebung muss von der symbolischen Ausführungsplattform bereitgestellt werden, und zwar möglichst effizient, präzis und komplett. Dies ist das Umgebungsproblem. Es ist schwierig, das Umgebungsprob- lem ein für alle mal zu lösen, denn seine Natur hängt von der gegebenen Schnittstelle und ihrer Implementierung ab.
    [Show full text]
  • Thirty Ways to Improve the Performance of Your Java™ Programs
    Thirty Ways to Improve the Performance of Your Java™ Programs Glen McCluskey E-mail: [email protected] Version 1.0, October 1999 Copyright © 1999 Glen McCluskey & Associates LLC. All Rights Reserved. Java and all Java-based marks are trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. and other countries. Throughout this paper, when the Java trademark appears alone it is a reference to the Java programming language. When the JDK trademark appears alone it is a reference to the Java Development Kit. The author would like to thank Steve Buroff, Craig Hondo, John Spicer, and Clay Wilson for assistance with proofreading. Thirty Ways to Improve the Performance of Your Java™ Programs Contents 1 Introduction........................................................................................................................................... 4 1.1 When to Worry About Performance................................................................................................ 4 1.2 Performance Issues Not Covered in This Paper ............................................................................. 4 1.3 Just-in-Time Compilers and Java Virtual Machines....................................................................... 4 1.4 Environment and Tools Used in Code Examples............................................................................ 5 1.5 How Examples Were Timed ............................................................................................................ 5 1.6 Performance Analysis Tools ..........................................................................................................
    [Show full text]
  • Simple Java - Foreword 86
    1 | 183 Contents I Freface 7 II Java Questions 11 1 How to Check if an Array Contains a Value in Java Efficiently? 13 2 Top 10 Questions about Java Exceptions 16 3 Why Field Can’t Be Overridden? 18 4 Constructors of Sub and Super Classes in Java? 20 5 Java Enum Examples 23 6 Java Access Level for Members: public, protected, private 24 7 The Interface and Class Hierarchy Diagram of Java Collections 25 8 Top 9 questions about Java Maps 28 9 Java equals() and hashCode() Contract 33 10 What does a Java array look like in memory? 35 11 The Introduction of Java Memory Leaks 38 12 Frequently Used Methods of Java HashMap 40 13 How Java Compiler Generate Code for Overloaded and Overridden Methods? 42 14 String is passed by “reference” in Java 43 15 FileOutputStream vs. FileWriter 46 16 HashSet vs. TreeSet vs. LinkedHashSet 46 17 How to Write a File Line by Line in Java? 52 18 What can we learn from Java HelloWorld? 54 19 Top 10 questions of Java Strings 58 20 How does Java handle aliasing? 61 21 How Static Type Checking Works in Java? 63 22 Interview Question - Use Java Thread to Do Math Calculation 64 23 Why String is immutable in Java ? 65 24 ArrayList vs. LinkedList vs. Vector 67 25 Java Varargs Examples 72 26 HashMap vs. TreeMap vs. Hashtable vs. LinkedHashMap 73 27 What is Instance Initializer in Java? 80 28 Top 10 Methods for Java Arrays 81 29 Java Type Erasure Mechanism 84 30 Simple Java - Foreword 86 31 Top 10 Mistakes Java Developers Make 86 32 How to make a method thread-safe in Java? 92 33 What Is Inner Interface in Java? 94 34 Top 10 questions about Java Collections 96 35 Set vs.
    [Show full text]
  • Prototyping Symbolic Execution Engines for Interpreted Languages
    Prototyping Symbolic Execution Engines for Interpreted Languages Stefan Bucur Johannes Kinder George Candea École Polytechnique Fédérale de Lausanne École Polytechnique Fédérale de Lausanne École Polytechnique Fédérale de Lausanne stefan.bucur@epfl.ch Royal Holloway, University of London george.candea@epfl.ch [email protected] Abstract 1. Introduction Symbolic execution is being successfully used to automat- Developers spend much of their time writing unit and perfor- ically test statically compiled code [4, 7, 9, 15]. However, mance tests and manually tracing hard-to-reproduce bugs; increasingly more systems and applications are written in ideally, these tasks would be automated. Symbolic execution dynamic interpreted languages like Python. Building a new is a particularly successful technique for exploring multiple symbolic execution engine is a monumental effort, and so execution paths fully automatically. It has been used to find is keeping it up-to-date as the target language evolves. Fur- bugs and to generate high-coverage test suites for Windows thermore, ambiguous language specifications lead to their media format parsers [16], Windows device drivers [19], implementation in a symbolic execution engine potentially and the Linux Coreutils [7], and it has helped to reproduce differing from the production interpreter in subtle ways. crashes for debugging [29]. Symbolic execution enumerates We address these challenges by flipping the problem and feasible execution paths by using a constraint solver to syn- using the interpreter itself as a specification of the language thesize inputs that drive the program down paths that have semantics. We present a recipe and tool (called CHEF)for not been covered before.
    [Show full text]
  • Transient Typechecks Are (Almost) Free
    Transient Typechecks Are (Almost) Free Richard Roberts School of Engineering and Computer Science, Victoria University of Wellington, New Zealand [email protected] Stefan Marr School of Computing, University of Kent, UK [email protected] Michael Homer School of Engineering and Computer Science, Victoria University of Wellington, New Zealand [email protected] James Noble School of Engineering and Computer Science, Victoria University of Wellington, New Zealand [email protected] Abstract Transient gradual typing imposes run-time type tests that typically cause a linear slowdown. This performance impact discourages the use of type annotations because adding types to a program makes the program slower. A virtual machine can employ standard just-in-time optimizations to reduce the overhead of transient checks to near zero. These optimizations can give gradually-typed languages performance comparable to state-of-the-art dynamic languages, so programmers can add types to their code without affecting their programs’ performance. 2012 ACM Subject Classification Software and its engineering → Just-in-time compilers; Software and its engineering → Object oriented languages; Software and its engineering → Interpreters Keywords and phrases dynamic type checking, gradual types, optional types, Grace, Moth, object- oriented programming Digital Object Identifier 10.4230/LIPIcs.ECOOP.2019.5 Funding This work is supported by the Royal Society of New Zealand Marsden Fund. 1 Introduction “It is a truth universally acknowledged, that a dynamic language in possession of a good user base, must be in want of a type system.” with apologies to Jane Austen. Dynamic languages are increasingly prominent in the software industry.
    [Show full text]
  • Concolic Execution of Nmap Scripts for Honeyfarm
    Concolic Execution of NMap Scripts for Honeyfarm Generation Zhe Li Bo Chen Department of Computer Science Department of Computer Science Portland State University Portland State University Portland, Oregon, USA Portland, Oregon, USA [email protected] [email protected] Wu-chang Feng Fei Xie Department of Computer Science Department of Computer Science Portland State University Portland State University Portland, Oregon, USA Portland, Oregon, USA [email protected] [email protected] ABSTRACT and honeynets) or to selectively terminate the operation of the script Attackers rely upon a vast array of tools for automating attacks by denying access (such as with web application firewalls). Unfortu- against vulnerable servers and services. It is often the case that nately, due to the massive code bases being used and the volume of when vulnerabilities are disclosed, scripts for detecting and exploit- vulnerabilities that are being discovered, it is difficult to keep such ing them in tools such as Nmap and Metasploit are released soon approaches up to date and to scale them to the number of vulnerabil- after, leading to the immediate identification and compromise of ities that are being disclosed. Thus, it is important that automated vulnerable systems. Honeypots, honeynets, tarpits, and other decep- defenses keep up with this arms race and attempt to make some tive techniques can be used to slow attackers down, however, such of the most common tasks an adversary relies upon more difficult approaches have difficulty keeping up with the sheer number of and time-consuming. In particular, as reconnaissance and targeting vulnerabilities being discovered and attacking scripts that are being are critical in an attack, slowing down or degrading this capability released.
    [Show full text]
  • Fundamental Data Structures Zuyd Hogeschool, ICT Contents
    Fundamental Data Structures Zuyd Hogeschool, ICT Contents 1 Introduction 1 1.1 Abstract data type ........................................... 1 1.1.1 Examples ........................................... 1 1.1.2 Introduction .......................................... 2 1.1.3 Defining an abstract data type ................................. 2 1.1.4 Advantages of abstract data typing .............................. 5 1.1.5 Typical operations ...................................... 5 1.1.6 Examples ........................................... 6 1.1.7 Implementation ........................................ 7 1.1.8 See also ............................................ 8 1.1.9 Notes ............................................. 8 1.1.10 References .......................................... 8 1.1.11 Further ............................................ 9 1.1.12 External links ......................................... 9 1.2 Data structure ............................................. 9 1.2.1 Overview ........................................... 10 1.2.2 Examples ........................................... 10 1.2.3 Language support ....................................... 11 1.2.4 See also ............................................ 11 1.2.5 References .......................................... 11 1.2.6 Further reading ........................................ 11 1.2.7 External links ......................................... 12 2 Sequences 13 2.1 Array data type ............................................ 13 2.1.1 History ...........................................
    [Show full text]