Virtual Memory
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Memory Deduplication: An
1 Memory Deduplication: An Effective Approach to Improve the Memory System Yuhui Deng1,2, Xinyu Huang1, Liangshan Song1, Yongtao Zhou1, Frank Wang3 1Department of Computer Science, Jinan University, Guangzhou, 510632, P. R. China {[email protected];[email protected];[email protected];[email protected]} 2Key Laboratory of Computer System and Architecture, Chinese Academy of Sciences Beijing, 100190, PR China 3 School of Computing,University of Kent, CT27NF, UK [email protected] Abstract— Programs now have more aggressive demands of memory to hold their data than before. This paper analyzes the characteristics of memory data by using seven real memory traces. It observes that there are a large volume of memory pages with identical contents contained in the traces. Furthermore, the unique memory content accessed are much less than the unique memory address accessed. This is incurred by the traditional address-based cache replacement algorithms that replace memory pages by checking the addresses rather than the contents of those pages, thus resulting in many identical memory contents with different addresses stored in the memory. For example, in the same file system, opening two identical files stored in different directories, or opening two similar files that share a certain amount of contents in the same directory, will result in identical data blocks stored in the cache due to the traditional address-based cache replacement algorithms. Based on the observations, this paper evaluates memory compression and memory deduplication. As expected, memory deduplication greatly outperforms memory compression. For example, the best deduplication ratio is 4.6 times higher than the best compression ratio. -
Murciano Soto, Joan; Rexachs Del Rosario, Dolores Isabel, Dir
This is the published version of the bachelor thesis: Murciano Soto, Joan; Rexachs del Rosario, Dolores Isabel, dir. Anàlisi de presta- cions de sistemes d’emmagatzematge per IA. 2021. (958 Enginyeria Informàtica) This version is available at https://ddd.uab.cat/record/248510 under the terms of the license TFG EN ENGINYERIA INFORMATICA,` ESCOLA D’ENGINYERIA (EE), UNIVERSITAT AUTONOMA` DE BARCELONA (UAB) Analisi` de prestacions de sistemes d’emmagatzematge per IA Joan Murciano Soto Resum– Els programes d’Intel·ligencia` Artificial (IA) son´ programes que fan moltes lectures de fitxers per la seva naturalesa. Aquestes lectures requereixen moltes crides a dispositius d’emmagatzematge, i aquestes poden comportar endarreriments en l’execucio´ del programa. L’ample de banda per transportar dades de disc a memoria` o viceversa pot esdevenir en un bottleneck, incrementant el temps d’execucio.´ De manera que es´ important saber detectar en aquest tipus de programes, si les entrades/sortides (E/S) del nostre sistema se saturen. En aquest treball s’estudien diferents programes amb altes quantitats de lectures a disc. S’utilitzen eines de monitoritzacio,´ les quals ens informen amb metriques` relacionades amb E/S a disc. Tambe´ veiem l’impacte que te´ el swap, el qual tambe´ provoca un increment d’operacions d’E/S. Aquest document preten´ mostrar la metodologia utilitzada per a realitzar l’analisi` descrivint les eines i els resultats obtinguts amb l’objectiu de que serveixi de guia per a entendre el comportament i l’efecte de les E/S i el swap. Paraules clau– E/S, swap, IA, monitoritzacio.´ Abstract– Artificial Intelligence (IA) programs make many file readings by nature. -
Strict Memory Protection for Microcontrollers
Master Thesis Spring 2019 Strict Memory Protection for Microcontrollers Erlend Sveen Supervisor: Jingyue Li Co-supervisor: Magnus Själander Sunday 17th February, 2019 Abstract Modern desktop systems protect processes from each other with memory protection. Microcontroller systems, which lack support for memory virtualization, typically only uses memory protection for areas that the programmer deems necessary and does not separate processes completely. As a result the application still appears monolithic and only a handful of errors may be detected. This thesis presents a set of solutions for complete separation of processes, unleash- ing the full potential of the memory protection unit embedded in modern ARM-based microcontrollers. The operating system loads multiple programs from disk into indepen- dently protected portions of memory. These programs may then be started, stopped, modified, crashed etc. without affecting other running programs. When loading pro- grams, a new allocation algorithm is used that automatically aligns the memories for use with the protection hardware. A pager is written to satisfy additional run-time demands of applications, and communication primitives for inter-process communication is made available. Since every running process is unable to get access to other running processes, not only reliability but also security is improved. An application may be split so that unsafe or error-prone code is separated from mission-critical code, allowing it to be independently restarted when an error occurs. With executable and writeable memory access rights being mutually exclusive, code injection is made harder to perform. The solution is all transparent to the programmer. All that is required is to split an application into sub-programs that operates largely independently. -
PC-BSD 9 Turns a New Page
CONTENTS Dear Readers, Here is the November issue. We are happy that we didn’t make you wait for it as long as for October one. Thanks to contributors and supporters we are back and ready to give you some usefull piece of knowledge. We hope you will Editor in Chief: Patrycja Przybyłowicz enjoy it as much as we did by creating the magazine. [email protected] The opening text will tell you What’s New in BSD world. It’s a review of PC-BSD 9 by Mark VonFange. Good reading, Contributing: especially for PC-BSD users. Next in section Get Started you Mark VonFange, Toby Richards, Kris Moore, Lars R. Noldan, will �nd a great piece for novice – A Beginner’s Guide To PF Rob Somerville, Erwin Kooi, Paul McMath, Bill Harris, Jeroen van Nieuwenhuizen by Toby Richards. In Developers Corner Kris Moore will teach you how to set up and maintain your own repository on a Proofreaders: FreeBSD system. It’s a must read for eager learners. Tristan Karstens, Barry Grumbine, Zander Hill, The How To section in this issue is for those who enjoy Christopher J. Umina experimenting. Speed Daemons by Lars R Noldan is a very good and practical text. By reading it you can learn Special Thanks: how to build a highly available web application server Denise Ebery with advanced networking mechanisms in FreeBSD. The Art Director: following article is the �nal one of our GIS series. The author Ireneusz Pogroszewski will explain how to successfully manage and commission a DTP: complex GIS project. -
Hiding Process Memory Via Anti-Forensic Techniques
DIGITAL FORENSIC RESEARCH CONFERENCE Hiding Process Memory via Anti-Forensic Techniques By: Frank Block (Friedrich-Alexander Universität Erlangen-Nürnberg (FAU) and ERNW Research GmbH) and Ralph Palutke (Friedrich-Alexander Universität Erlangen-Nürnberg) From the proceedings of The Digital Forensic Research Conference DFRWS USA 2020 July 20 - 24, 2020 DFRWS is dedicated to the sharing of knowledge and ideas about digital forensics research. Ever since it organized the first open workshop devoted to digital forensics in 2001, DFRWS continues to bring academics and practitioners together in an informal environment. As a non-profit, volunteer organization, DFRWS sponsors technical working groups, annual conferences and challenges to help drive the direction of research and development. https://dfrws.org Forensic Science International: Digital Investigation 33 (2020) 301012 Contents lists available at ScienceDirect Forensic Science International: Digital Investigation journal homepage: www.elsevier.com/locate/fsidi DFRWS 2020 USA d Proceedings of the Twentieth Annual DFRWS USA Hiding Process Memory Via Anti-Forensic Techniques Ralph Palutke a, **, 1, Frank Block a, b, *, 1, Patrick Reichenberger a, Dominik Stripeika a a Friedrich-Alexander Universitat€ Erlangen-Nürnberg (FAU), Germany b ERNW Research GmbH, Heidelberg, Germany article info abstract Article history: Nowadays, security practitioners typically use memory acquisition or live forensics to detect and analyze sophisticated malware samples. Subsequently, malware authors began to incorporate anti-forensic techniques that subvert the analysis process by hiding malicious memory areas. Those techniques Keywords: typically modify characteristics, such as access permissions, or place malicious data near legitimate one, Memory subversion in order to prevent the memory from being identified by analysis tools while still remaining accessible. -
Virtual Memory - Paging
Virtual memory - Paging Johan Montelius KTH 2020 1 / 32 The process code heap (.text) data stack kernel 0x00000000 0xC0000000 0xffffffff Memory layout for a 32-bit Linux process 2 / 32 Segments - a could be solution Processes in virtual space Address translation by MMU (base and bounds) Physical memory 3 / 32 one problem Physical memory External fragmentation: free areas of free space that is hard to utilize. Solution: allocate larger segments ... internal fragmentation. 4 / 32 another problem virtual space used code We’re reserving physical memory that is not used. physical memory not used? 5 / 32 Let’s try again It’s easier to handle fixed size memory blocks. Can we map a process virtual space to a set of equal size blocks? An address is interpreted as a virtual page number (VPN) and an offset. 6 / 32 Remember the segmented MMU MMU exception no virtual addr. offset yes // < within bounds index + physical address segment table 7 / 32 The paging MMU MMU exception virtual addr. offset // VPN available ? + physical address page table 8 / 32 the MMU exception exception virtual address within bounds page available Segmentation Paging linear address physical address 9 / 32 a note on the x86 architecture The x86-32 architecture supports both segmentation and paging. A virtual address is translated to a linear address using a segmentation table. The linear address is then translated to a physical address by paging. Linux and Windows do not use use segmentation to separate code, data nor stack. The x86-64 (the 64-bit version of the x86 architecture) has dropped many features for segmentation. -
Chapter 4 Memory Management and Virtual Memory Asst.Prof.Dr
Chapter 4 Memory management and Virtual Memory Asst.Prof.Dr. Supakit Nootyaskool IT-KMITL Object • To discuss the principle of memory management. • To understand the reason that memory partitions are importance for system organization. • To describe the concept of Paging and Segmentation. 4.1 Difference in main memory in the system • The uniprogramming system (example in DOS) allows only a program to be present in memory at a time. All resources are provided to a program at a time. Example in a memory has a program and an OS running 1) Operating system (on Kernel space) 2) A running program (on User space) • The multiprogramming system is difference from the mentioned above by allowing programs to be present in memory at a time. All resource must be organize and sharing to programs. Example by two programs are in the memory . 1) Operating system (on Kernel space) 2) Running programs (on User space + running) 3) Programs are waiting signals to execute in CPU (on User space). The multiprogramming system uses memory management to organize location to the programs 4.2 Memory management terms Frame Page Segment 4.2 Memory management terms Frame Page A fixed-lengthSegment block of main memory. 4.2 Memory management terms Frame Page A fixed-length block of data that resides in secondary memory. A page of data may temporarily beSegment copied into a frame of main memory. A variable-lengthMemory management block of data that residesterms in secondary memory. A segment may temporarily be copied into an available region of main memory or the segment may be divided into pages which can be individuallyFrame copied into mainPage memory. -
Module 4: Memory Management the Von Neumann Principle for the Design and Operation of Computers Requires That a Program Has to Be Primary Memory Resident to Execute
Operating Systems/Memory management Lecture Notes Module 4: Memory Management The von Neumann principle for the design and operation of computers requires that a program has to be primary memory resident to execute. Also, a user requires to revisit his programs often during its evolution. However, due to the fact that primary memory is volatile, a user needs to store his program in some non-volatile store. All computers provide a non-volatile secondary memory available as an online storage. Programs and files may be disk resident and downloaded whenever their execution is required. Therefore, some form of memory management is needed at both primary and secondary memory levels. Secondary memory may store program scripts, executable process images and data files. It may store applications, as well as, system programs. In fact, a good part of all OS, the system programs which provide services (the utilities for instance) are stored in the secondary memory. These are requisitioned as needed. The main motivation for management of main memory comes from the support for multi- programming. Several executables processes reside in main memory at any given time. In other words, there are several programs using the main memory as their address space. Also, programs move into, and out of, the main memory as they terminate, or get suspended for some IO, or new executables are required to be loaded in main memory. So, the OS has to have some strategy for main memory management. In this chapter we shall discuss the management issues and strategies for both main memory and secondary memory. -
OS Paging and Buffer Management
OS Paging and Bu↵er Management Using the Virtual-Memory Paging Mechanism to Balance Hot/Cold Data in Memory and HDD Jana Lampe TU Kaiserslautern LGIS Seminar SS 2014 Optimizing Data Management on New Hardware 1 Introduction 1.1 Motivation Traditional database systems follow the design principle that all data is stored on hard disk where records are accessible via indexing structures and loaded into main-memory as soon as they are requested. Since memory prices have dropped significantly during the last three decades, new database systems have been designed that allow faster access to the data by storing all data inside the main-memory, e.g. H-Store [9]. Notwithstanding the above, also durable storage technologies have made con- siderable progress towards larger capacity, more input/output operations per second as well as reduced cost per storage unit at the same time. Solid-state drives (SSDs) have entered the market and, using the non-volatile NAND-flash technology, convince by their reading and writing speed, although they are still about ten times more expensive than common HDDs. Another promising tech- nology is phase-change memory (PCM), but it is still in research state. Now one could ask, why all the data is stored in memory. If there are records that are not accessed that frequently, would it not be cheaper to store them on a secondary storage device such as HDD or SSD? Moreover, energy consump- tion would decrease and more space would be available to temporarily store intermediate results thereby accelerating query processing. To answer this question, Gray and Putzolu proposed in [5] the 5-minute rule, which says, that a record should be kept in memory if it has accessed at least every 5 minutes (the break-even interval). -
Fluidmem: Open Source Full Memory Disaggregation
FluidMem: Open Source Full Memory Disaggregation by Blake Caldwell B.S., University of Colorado at Boulder, 2014 M.S., University of Colorado at Boulder, 2015 A thesis submitted to the Faculty of the Graduate School of the University of Colorado in partial fulfillment of the requirements for the degree of Doctor of Philosophy Department of Computer Science 2019 This thesis entitled: FluidMem: Open Source Full Memory Disaggregation written by Blake Caldwell has been approved for the Department of Computer Science Prof. Richard Han Prof. Eric Keller Prof. Sangtae Ha Prof. Kenneth Anderson Date The final copy of this thesis has been examined by the signatories, and we find that both the content and the form meet acceptable presentation standards of scholarly work in the above mentioned discipline. iii Caldwell, Blake (Ph.D., Computer Science) FluidMem: Open Source Full Memory Disaggregation Thesis directed by Prof. Richard Han To satisfy the performance demands of memory-intensive applications facing DRAM short- ages, the focus of previous work has been on incorporating remote memory to expand capacity. However, the emergence of resource balancing as a priority for cloud computing requires the capa- bility to dynamically size virtual machine memory up and down. Furthermore, hardware-based or kernel space implementations hamper flexibility with respect to making customizations or integrat- ing the continuing open source advancements in software infrastructure for the data center. This thesis presents an architecture to meet the performance, bi-directional sizing, and flexibility challenges of memory disaggregation in the cloud. The implementation, called FluidMem, is open source software [35] that integrates with the Linux kernel, KVM hypervisor, and multiple key- values stores. -
SLAKE: Facilitating Slab Manipulation for Exploiting
SLAKE: Facilitating Slab Manipulation for Exploiting Vulnerabilities in the Linux Kernel Yueqi Chen Xinyu Xing [email protected] [email protected] The Pennsylvania State University The Pennsylvania State University ABSTRACT 1 INTRODUCTION To determine the exploitability for a kernel vulnerability, a secu- Despite extensive code review, a Linux kernel, like all other soft- rity analyst usually has to manipulate slab and thus demonstrate ware, still inevitably contains a large number of bugs and vulnerabil- the capability of obtaining the control over a program counter or ities [42]. Compared with vulnerabilities in user-level applications, performing privilege escalation. However, this is a lengthy process a vulnerability in kernel code is generally more disconcerting be- because (1) an analyst typically has no clue about what objects cause kernel code runs with a higher privilege and the successful and system calls are useful for kernel exploitation and (2) he lacks exploitation of a kernel vulnerability could provide an attacker with the knowledge of manipulating a slab and obtaining the desired full root access. layout. In the past, researchers have proposed various techniques One straightforward solution to minimize the damage of Linux to facilitate exploit development. Unfortunately, none of them can kernel defects is to have software developers and security analysts be easily applied to address these challenges. On the one hand, this patch all the bugs and vulnerabilities immediately. However, even is because of the complexity of the Linux kernel. On the other hand, though allowing anyone to contribute to code development and this is due to the dynamics and non-deterministic of slab variations. -
A Guide to Kernel Exploitation Attacking the Core (2011
A Guide to Kernel Exploitation This page intentionally left blank A Guide to Kernel Exploitation Attacking the Core Enrico Perla Massimiliano Oldani Technical Editor Graham Speake AMSTERDAM • BOSTON • HEIDELBERG • LONDON • • • NEW YORK OXFORD PARIS SAN DIEGO SYNGRESS SAN FRANCISCO • SINGAPORE • SYDNEY • TOKYO ® Syngress is an imprint of Elsevier Acquiring Editor: Rachel Roumeliotis Development Editor: Matthew Cater Project Manager: Julie Ochs Designer: Alisa Andreola Syngress is an imprint of Elsevier 30 Corporate Drive, Suite 400, Burlington, MA 01803, USA © 2011 Elsevier Inc. All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage and retrieval system, without permission in writing from the publisher. Details on how to seek permission, further information about the Publisher’s permissions policies and our arrangements with organizations such as the Copyright Clearance Center and the Copyright Licensing Agency, can be found at our website: www.elsevier.com/permissions. This book and the individual contributions contained in it are protected under copyright by the Publisher (other than as may be noted herein). Notices Knowledge and best practice in this field are constantly changing. As new research and experience broaden our understanding, changes in research methods or professional practices, may become necessary. Practitioners and researchers must always rely on their own experience and knowledge in evaluating and using any information or methods described herein. In using such information or methods they should be mindful of their own safety and the safety of others, including parties for whom they have a professional responsibility.