DOCSLIB.ORG

An Analysis of How Static and Shared Libraries Affect Memory Usage on an IP-STB

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
An Analysis of How Static and Shared Libraries Affect Memory Usage on an IP-STB An analysis of how static and shared libraries affect memory usage on an IP-STB Master of Science Thesis in the Programme Networks and Distributed Systems Dongping Huang Chalmers University of Technology University of Gothenburg Department of Computer Science and Engineering Göteborg, Sweden, September 2010 The Author grants to Chalmers University of Technology and University of Gothenburg the non-exclusive right to publish the Work electronically and in a non-commercial purpose make it accessible on the Internet. The Author warrants that he/she is the author to the Work, and warrants that the Work does not contain text, pictures or other material that violates copyright law. The Author shall, when transferring the rights of the Work to a third party (for example a publisher or a company), acknowledge the third party about this agreement. If the Author has signed a copyright agreement with a third party regarding the Work, the Author warrants hereby that he/she has obtained any necessary permission from this third party to let Chalmers University of Technology and University of Gothenburg store the Work electronically and make it accessible on the Internet. An analysis of how code locality affects the dynamic memory usage on an embedded set- top box running Linux Dongping Huang © Dongping Huang, September 2010. Examiner: Björn von Sydow Chalmers University of Technology University of Gothenburg Department of Computer Science and Engineering SE-412 96 Göteborg Sweden Telephone + 46 (0)31-772 1000 Department of Computer Science and Engineering Göteborg, Sweden September 2010 Abstract An IP-STB is a small and stand alone embedded system which is able to provide a series of advanced entertaining services. However, like an ordinary embedded system, an IP-STB still has to bear with extremely main memory constraints and no disk space. Normally, advanced features provided by an IP-STB require a large amount of limited main memory, therefore how to maintain a small static memory footprint and low memory usage becomes very important. Libraries composed of shared code contribute to a significant amount of memory consumption. Libraries can be categorized into two types: static libraries and shared libraries. Both of them have their pros and cons in terms of memory usage and system performance. However, there is no existing generic method to determine when and why to choose one type over the other one. To address this problem, a series of tools were implemented to give a quantitative analysis of how different types of libraries affect the system memory usage. Besides, their advantages and disadvantages were discussed and compared. In addition to the above, the optimization of shared libraries was also investigated. Acknowledgement First of all, I would like to express my deepest gratitude to Motorola for giving me this opportunity to accomplish this thesis. I would particularly like to acknowledge my supervisor Patrik Almqvist for his patient guidance and assistance. Finally, I would like to thank my examiner Björn von Sydow at Chalmers University of Technology for his valuable guidance and feedback. Table of Contents 1 Introduction..............................................................................................................................1 1.1 Background.......................................................................................................................1 1.2 Problems description........................................................................................................1 1.3 Objective...........................................................................................................................1 1.4 Glossary............................................................................................................................2 1.5 Thesis organization...........................................................................................................3 2 The System...............................................................................................................................4 2.1 Hardware..........................................................................................................................4 2.2 Software............................................................................................................................4 2.3 IP-STB installation packages............................................................................................4 2.4 Boot images......................................................................................................................4 2.5 Boot procedure.................................................................................................................5 2.6 File systems......................................................................................................................5 3 Libraries and tools....................................................................................................................7 3.1 ELF...................................................................................................................................7 3.2 Static libraries...................................................................................................................9 3.3 Shared libraries...............................................................................................................10 3.4 Linux memory mapping.................................................................................................11 3.5 GNU compiler and binutils.............................................................................................11 3.5.1 The /proc file system...............................................................................................12 4 Measuring memory usage: methodology ..............................................................................15 4.1 Prerequisites....................................................................................................................15 4.2 Listing dependencies......................................................................................................15 4.3 Measuring memory usage...............................................................................................16 4.4 The specific procedure of measurement.........................................................................18 5 Discussion..............................................................................................................................20 5.1 Shortcomings of static libraries......................................................................................20 5.2 Shortcomings of shared libraries....................................................................................21 5.3 Static libraries or shared libraries ..................................................................................25 5.3.1 Memory usage.........................................................................................................25 5.3.2 System performance................................................................................................25 5.3.3 Compatibility and extendibility..............................................................................26 5.3.4 General proposals ...................................................................................................26 5.4 How to remove unused material.....................................................................................27 5.5 Position independent code (PIC)....................................................................................28 5.5.1 Differences between -fPIC and -fpic .....................................................................28 5.5.2 Relocation processing.............................................................................................29 5.5.3 Cost of position independent code..........................................................................29 5.5.4 Why not use PIC for all applications?....................................................................31 6 Summary................................................................................................................................32 6.1 Conclusion......................................................................................................................32 6.2 Future work.....................................................................................................................32 7 References..............................................................................................................................33 8 Appendix A............................................................................................................................34 9 Appendix B.............................................................................................................................35 1 Introduction 1.1 Background An ordinary set-top box is a device that connects to a television to display digital TV and receives digital signal from external sources such as satellites, cables and antennas. An IP based set-top box (IP-STB), as its name implies, connects and receives digital signals from IP based networks. Taking full advantages of broadband networks, an IP-STB is able to provide a range of advanced services such as video on demand (VOD), time-shifting video, muliticast TV, as well as some Internet applications, including web surfing and so on. The IP-STB used
Load more

Recommended publications
  • An Event-Driven Embedded Operating System for Miniature Robots
    This is a repository copy of OpenSwarm: an event-driven embedded operating system for miniature robots. White Rose Research Online URL for this paper: http://eprints.whiterose.ac.uk/110437/ Version: Accepted Version Proceedings Paper: Trenkwalder, S.M., Lopes, Y.K., Kolling, A. et al. (3 more authors) (2016) OpenSwarm: an event-driven embedded operating system for miniature robots. In: Proceedings of IROS 2016. 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems, 09-14 Oct 2016, Daejeon, Korea. IEEE . ISBN 978-1-5090-3762-9 https://doi.org/10.1109/IROS.2016.7759660 © 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works. Reuse Items deposited in White Rose Research Online are protected by copyright, with all rights reserved unless indicated otherwise. They may be downloaded and/or printed for private study, or other acts as permitted by national copyright laws. The publisher or other rights holders may allow further reproduction and re-use of the full text version. This is indicated by the licence information on the White Rose Research Online record for the item. Takedown If you consider content in White Rose Research Online to be in breach of UK law, please notify us by emailing [email protected] including the URL of the record and the reason for the withdrawal request.
    [Show full text]
  • What Are the Problems with Embedded Linux?
    What Are the Problems with Embedded Linux? Every Operating System Has Benefits and Drawbacks Linux is ubiquitous. It runs most internet servers, is inside Android* smartphones, and is used on millions of embedded systems that, in the past, ran Real-Time Operating Systems (RTOSes). Linux can (and should) be used were possible for embedded projects, but while it gives you extreme choice, it also presents the risk of extreme complexity. What, then, are the trade-offs between embedded Linux and an RTOS? In this article, we cover some key considerations when evaluating Linux for a new development: ■ Design your system architecture first ■ What is Linux? ■ Linux vs. RTOSes ■ Free software is about liberty—not price ■ How much does Embedded Linux cost? ■ Why pay for Embedded Linux? ■ Should you buy Embedded Linux or roll-your-own (RYO)? ■ To fork or not to fork? ■ Software patching ■ Open source licensing ■ Making an informed decision The important thing is not whether Linux or an RTOS is “the best,” but whether either operating system—or both together—makes the most technical and financial sense for your project. We hope this article helps you make an informed decision. Design Your System Architecture First It is important to design your system architecture first—before choosing either Linux or an RTOS—because both choices can limit architectural freedom. You may discover that aspects of your design require neither Linux nor an RTOS, making your design a strong candidate for a heterogeneous approach that includes one or more bare-metal environments (with possibly a Linux and/or RTOS environment as well).
    [Show full text]
  • Co-Optimizing Performance and Memory Footprint Via Integrated CPU/GPU Memory Management, an Implementation on Autonomous Driving Platform
    2020 IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS) Co-Optimizing Performance and Memory Footprint Via Integrated CPU/GPU Memory Management, an Implementation on Autonomous Driving Platform Soroush Bateni*, Zhendong Wang*, Yuankun Zhu, Yang Hu, and Cong Liu The University of Texas at Dallas Abstract—Cutting-edge embedded system applications, such as launches the OpenVINO toolkit for the edge-based deep self-driving cars and unmanned drone software, are reliant on learning inference on its integrated HD GPUs [4]. integrated CPU/GPU platforms for their DNNs-driven workload, Despite the advantages in SWaP features presented by the such as perception and other highly parallel components. In this work, we set out to explore the hidden performance im- integrated CPU/GPU architecture, our community still lacks plication of GPU memory management methods of integrated an in-depth understanding of the architectural and system CPU/GPU architecture. Through a series of experiments on behaviors of integrated GPU when emerging autonomous and micro-benchmarks and real-world workloads, we find that the edge intelligence workloads are executed, particularly in multi- performance under different memory management methods may tasking fashion. Specifically, in this paper we set out to explore vary according to application characteristics. Based on this observation, we develop a performance model that can predict the performance implications exposed by various GPU memory system overhead for each memory management method based management (MM) methods of the integrated CPU/GPU on application characteristics. Guided by the performance model, architecture. The reason we focus on the performance impacts we further propose a runtime scheduler.
    [Show full text]
  • Performance Study of Real-Time Operating Systems for Internet Of
    IET Software Research Article ISSN 1751-8806 Performance study of real-time operating Received on 11th April 2017 Revised 13th December 2017 systems for internet of things devices Accepted on 13th January 2018 E-First on 16th February 2018 doi: 10.1049/iet-sen.2017.0048 www.ietdl.org Rafael Raymundo Belleza1 , Edison Pignaton de Freitas1 1Institute of Informatics, Federal University of Rio Grande do Sul, Av. Bento Gonçalves, 9500, CP 15064, Porto Alegre CEP: 91501-970, Brazil E-mail: [email protected] Abstract: The development of constrained devices for the internet of things (IoT) presents lots of challenges to software developers who build applications on top of these devices. Many applications in this domain have severe non-functional requirements related to timing properties, which are important concerns that have to be handled. By using real-time operating systems (RTOSs), developers have greater productivity, as they provide native support for real-time properties handling. Some of the key points in the software development for IoT in these constrained devices, like task synchronisation and network communications, are already solved by this provided real-time support. However, different RTOSs offer different degrees of support to the different demanded real-time properties. Observing this aspect, this study presents a set of benchmark tests on the selected open source and proprietary RTOSs focused on the IoT. The benchmark results show that there is no clear winner, as each RTOS performs well at least on some criteria, but general conclusions can be drawn on the suitability of each of them according to their performance evaluation in the obtained results.
    [Show full text]
  • Program-Library-HOWTO.Pdf
    Program Library HOWTO David A. Wheeler version 1.20, 11 April 2003 This HOWTO for programmers discusses how to create and use program libraries on Linux. This includes static libraries, shared libraries, and dynamically loaded libraries. Program Library HOWTO Table of Contents 1. Introduction.....................................................................................................................................................1 2. Static Libraries................................................................................................................................................2 3. Shared Libraries.............................................................................................................................................3 3.1. Conventions......................................................................................................................................3 3.1.1. Shared Library Names.............................................................................................................3 3.1.2. Filesystem Placement..............................................................................................................4 3.2. How Libraries are Used....................................................................................................................4 3.3. Environment Variables.....................................................................................................................5 3.3.1. LD_LIBRARY_PATH............................................................................................................5
    [Show full text]
  • Opencobol Manual for Opencobol 2.0
    OpenCOBOL Manual for OpenCOBOL 2.0 Keisuke Nishida / Roger While Edition 2.0 Updated for OpenCOBOL 2.0 26 January 2012 OpenCOBOL is an open-source COBOL compiler, which translates COBOL programs to C code and compiles it using GCC or other C compiler system. This manual corresponds to OpenCOBOL 2.0. Copyright c 2002,2003,2004,2005,2006 Keisuke Nishida Copyright c 2007-2012 Roger While Permission is granted to make and distribute verbatim copies of this manual provided the copy- right notice and this permission notice are preserved on all copies. Permission is granted to copy and distribute modified versions of this manual under the condi- tions for verbatim copying, provided that the entire resulting derived work is distributed under the terms of a permission notice identical to this one. Permission is granted to copy and distribute translations of this manual into another language, under the above conditions for modified versions, except that this permission notice maybe stated in a translation approved by the Free Software Foundation. i Table of Contents 1 Getting Started :::::::::::::::::::::::::::::::::::::::::::::::: 1 1.1 Hello World!::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 1 2 Compile :::::::::::::::::::::::::::::::::::::::::::::::::::::::: 2 2.1 Compiler Options ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 2 2.1.1 Help Options ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 2 2.1.2 Built Target ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
    [Show full text]
  • Our Tas Top 11 Technologies of the Decade Tinyos and Nesc Hardware Evolu On
    Our TAs Top 11 Technologies of the Decade Mo Sha Yong Fu 1.! Smartphones 7.! Drone Aircra • [email protected]" •! [email protected]" •! TinyOS tutorial." •! Grade critiques." 2.! Social Networking 8.! Planetary Rovers •! Help students with projects." •! Office Hour: by appointment." 3.! Voice over IP 9.! Flexible AC •! Manage motes." •! Bryan 502D Transmission •! Grade projects." 4.! LED Lighng •! Office Hour: Tue/Thu 5:30-6." 5.! Mul@core CPUs 10.!Digital Photography •! Bryan 502A 6.! Cloud Compung 11.!Class-D Audio Chenyang Lu 1 Chenyang Lu 2 TinyOS and nesC Hardware Evoluon ! TinyOS: OS for wireless sensor networks. ! Miniature devices manufactured economically ! nesC: programming language for TinyOS. ! Microprocessors ! Sensors/actuators ! Wireless chips 4.5’’X2.4’’ 1’’X1’’ 1 mm2 1 nm2 Chenyang Lu 4 Chenyang Lu 3 Mica2 Mote Hardware Constraints ! Processor ! Microcontroller: 7.4 MHz, 8 bit Severe constraints on power, size, and cost ! Memory: 4KB data, 128 KB program ! slow microprocessor ! Radio ! low-bandwidth radio ! Max 38.4 Kbps ! limited memory ! Sensors ! limited hardware parallelism CPU hit by many interrupts! ! Light, temperature, acceleraon, acous@c, magne@c… ! manage sleep modes in hardware components ! Power ! <1 week on two AA baeries in ac@ve mode ! >1 year baery life on sleep modes! Chenyang Lu 5 Chenyang Lu 6 Soware Challenges Tradional OS ! Small memory footprint ! Mul@-threaded ! Efficiency - power and processing ! Preemp@ve scheduling ! Concurrency-intensive operaons ! Threads: ! Diversity in applicaons & plaorm efficient modularity ! ready to run; ! Support reconfigurable hardware and soJware executing ! execu@ng on the CPU; ! wai@ng for data. gets CPU preempted needs data gets data ready waiting needs data Chenyang Lu 7 Chenyang Lu 8 Pros and Cons of Tradional OS Example: Preempve Priority Scheduling ! Each process has a fixed priority (1 highest); ! Mul@-threaded + preemp@ve scheduling ! P1: priority 1; P2: priority 2; P3: priority 3.
    [Show full text]
  • Static Typing Where Possible, Dynamic Typing When Needed: the End of the Cold War Between Programming Languages
    Intended for submission to the Revival of Dynamic Languages Static Typing Where Possible, Dynamic Typing When Needed: The End of the Cold War Between Programming Languages Erik Meijer and Peter Drayton Microsoft Corporation Abstract Advocates of dynamically typed languages argue that static typing is too rigid, and that the softness of dynamically lan- This paper argues that we should seek the golden middle way guages makes them ideally suited for prototyping systems with between dynamically and statically typed languages. changing or unknown requirements, or that interact with other systems that change unpredictably (data and application in- tegration). Of course, dynamically typed languages are indis- 1 Introduction pensable for dealing with truly dynamic program behavior such as method interception, dynamic loading, mobile code, runtime <NOTE to="reader"> Please note that this paper is still very reflection, etc. much work in progress and as such the presentation is unpol- In the mother of all papers on scripting [16], John Ousterhout ished and possibly incoherent. Obviously many citations to argues that statically typed systems programming languages related and relevant work are missing. We did however do our make code less reusable, more verbose, not more safe, and less best to make it provocative. </NOTE> expressive than dynamically typed scripting languages. This Advocates of static typing argue that the advantages of static argument is parroted literally by many proponents of dynami- typing include earlier detection of programming mistakes (e.g. cally typed scripting languages. We argue that this is a fallacy preventing adding an integer to a boolean), better documenta- and falls into the same category as arguing that the essence of tion in the form of type signatures (e.g.
    [Show full text]
  • Fluidmem: Open Source Full Memory Disaggregation
    FluidMem: Open Source Full Memory Disaggregation by Blake Caldwell B.S., University of Colorado at Boulder, 2014 M.S., University of Colorado at Boulder, 2015 A thesis submitted to the Faculty of the Graduate School of the University of Colorado in partial fulfillment of the requirements for the degree of Doctor of Philosophy Department of Computer Science 2019 This thesis entitled: FluidMem: Open Source Full Memory Disaggregation written by Blake Caldwell has been approved for the Department of Computer Science Prof. Richard Han Prof. Eric Keller Prof. Sangtae Ha Prof. Kenneth Anderson Date The final copy of this thesis has been examined by the signatories, and we find that both the content and the form meet acceptable presentation standards of scholarly work in the above mentioned discipline. iii Caldwell, Blake (Ph.D., Computer Science) FluidMem: Open Source Full Memory Disaggregation Thesis directed by Prof. Richard Han To satisfy the performance demands of memory-intensive applications facing DRAM short- ages, the focus of previous work has been on incorporating remote memory to expand capacity. However, the emergence of resource balancing as a priority for cloud computing requires the capa- bility to dynamically size virtual machine memory up and down. Furthermore, hardware-based or kernel space implementations hamper flexibility with respect to making customizations or integrat- ing the continuing open source advancements in software infrastructure for the data center. This thesis presents an architecture to meet the performance, bi-directional sizing, and flexibility challenges of memory disaggregation in the cloud. The implementation, called FluidMem, is open source software [35] that integrates with the Linux kernel, KVM hypervisor, and multiple key- values stores.
    [Show full text]
  • Dynamic C++ Classes a Lightweight Mechanism to Update Code in a Running Program
    The following paper was originally published in the Proceedings of the USENIX Annual Technical Conference (NO 98) New Orleans, Louisiana, June 1998 Dynamic C++ Classes A lightweight mechanism to update code in a running program Gísli Hjálmtÿsson AT&T Labs - Research Robert Gray Dartmouth College For more information about USENIX Association contact: 1. Phone: 510 528-8649 2. FAX: 510 548-5738 3. Email: [email protected] 4. WWW URL:http://www.usenix.org/ Dynamic C++ Classes A lightweight mechanism to update code in a running program Gísli Hjálmtýsson Robert Gray AT&T Labs – Research Thayer School of Engineering1 180 Park Avenue Dartmouth College Florham Park, NJ 07932 Hanover, NH 03755 [email protected] [email protected] has transformed many industries, continuous change has Abstract become just as important as continuous operation. Techniques for dynamically adding new code to a run- Rapid introduction of new functionality and dynamic ning program already exist in various operating sys- adaptation to volatile needs is essential. Systems, par- tems, programming languages and runtime environ- ticularly telecommunications systems, must be custom- ments. Most of these systems have not found their way ized on a very fine time scale, either due to user demand 1 into common use, however, since they require pro- or to load and usage fluctuations. grammer retraining and invalidate previous software An effective way to allow both continuous operation investments. In addition, many of the systems are too and continuous change is through dynamic code up- high-level for performance-critical applications. This dates. New code is added to a running program without paper presents an implementation of dynamic classes halting the program, thus introducing new functionality for the C++ language.
    [Show full text]
  • Reprogramming Embedded Systems at Run-Time
    Proceedings of the 8th International Conference on Sensing Technology, Sep. 2-4, 2014, Liverpool, UK Reprogramming Embedded Systems at Run-Time Richard Oliver, Adriana Wilde IEEE(S) and Ed Zaluska SMIEEE Electronics and Computer Science University of Southampton, United Kingdom {rjo2g10,agw106,ejz}@ecs.soton.ac.uk Abstract—The dynamic re-programming of embedded systems is load mechanisms. This paper aims to address the problem of a long-standing problem in the field. With the advent of wireless dynamic reprogramming of such systems, and will offer sensor networks and the ‘Internet of Things’ it has now become comparison and a critical appraisal of the methods currently necessary to be able to reprogram at run-time due to the being used in the field. difficulty of gaining access to such systems once deployed. The issues of power consumption, flexibility, and operating system The remainder of this paper is structured as follows. protections are examined for a range of approaches, and a Section II explains code execution on embedded devices, critical comparison is given. A combination of approaches is section III discusses various existing methods, focusing on the recommended for the implementation of real-world systems and issues of power consumption, flexibility of approach, and OS areas where further work is required are highlighted. protection. Section IV follows with conclusions and suggests future work in the area. Keywords-Wireless sensor networks; Programming; Runtime, Energy consumption II. BACKGROUND I. INTRODUCTION This section will cover the execution of code on microcontrollers. This is typically of no concern when flashing Embedded systems are pervasive in the modern world, a static image to a microcontroller but a greater understanding being found in systems as widely ranging as fridges, cars, is required to implement dynamic code loading.
    [Show full text]
  • Linux Standard Base Development Kit for Application Building/Porting
    Linux Standard Base Development Kit for application building/porting Rajesh Banginwar Nilesh Jain Intel Corporation Intel Corporation [email protected] [email protected] Abstract validating binaries and RPM packages for LSB conformance. We conclude with a couple of case studies that demonstrate usage of the build The Linux Standard Base (LSB) specifies the environment as well as the associated tools de- binary interface between an application and a scribed in the paper. runtime environment. This paper discusses the LSB Development Kit (LDK) consisting of a build environment and associated tools to assist software developers in building/porting their applications to the LSB interface. Developers 1 Linux Standard Base Overview will be able to use the build environment on their development machines, catching the LSB porting issues early in the development cycle The Linux* Standard Base (LSB)[1] specifies and reducing overall LSB conformance testing the binary interface between an application and time and cost. Associated tools include appli- a runtime environment. The LSB Specifica- cation and package checkers to test for LSB tion consists of a generic portion, gLSB, and conformance of application binaries and RPM an architecture-specific portion, archLSB. As packages. the names suggest, gLSB contains everything that is common across all architectures, and This paper starts with the discussion of ad- archLSBs contain the things that are specific vantages the build environment provides by to each processor architecture, such as the ma- showing how it simplifies application develop- chine instruction set and C library symbol ver- ment/porting for LSB conformance. With the sions. availability of this additional build environment from LSB working group, the application de- As much as possible, the LSB builds on ex- velopers will find the task of porting applica- isting standards, including the Single UNIX tions to LSB much easier.
    [Show full text]