Monitoring Virtual Memory Using Vmstat

Total Page:16

File Type:pdf, Size:1020Kb

Monitoring Virtual Memory Using Vmstat AdminAdmin Monitoring Virtual Memory Using vmstat Just using a lot of swap space doesn’t necessarily mean that you need more memory. Here’s how to tell when Linux is happy with the available memory and when it needs more... inux novices often find virtual portions of physical memory when possible. memory mysterious, but with a All processes use memory, of course, but grasp of the fundamental each process doesn’t need all its allocated concepts, it’s easy to understand. memory all the time. Taking advantage of this L With this knowledge, you can fact, the kernel frees up physical memory by monitor your system’s memory utilization writing some or all of a process’ memory to using vmstat and detect problems that can disk until it’s needed again. adversely affect system performance. The kernel uses paging and swapping to perform this memory management. Paging Getting the basics right refers to writing portions, termed pages, of a Physical memory—the actual RAM process’ memory to disk. Swapping, strictly installed—is a finite resource on any system. speaking, refers to writing the entire process, The Linux memory handler manages the not just part, to disk. In Linux, true swapping allocation of that limited resource by freeing is exceedingly rare, but the terms paging and www.linuxforu.com | LINUX FOR YOU | DECEMBER 2005 87 C M Y K Admin swapping often are used interchangeably. 0 0 0 29232 116972 4524 244900 0 0 0 0 0 0 0 0 0 When pages are written to disk, the event is called a 0 0 0 29232 116972 4524 244900 0 0 0 0 2560 6 0 1 99 page-out, and when pages are returned to physical 0 0 0 29232 116972 4524 244900 0 0 0 0 2574 10 0 2 98 memory, the event is called a page-in. A page fault occurs when the kernel needs a page, finds it doesn’t exist in All fields in this output are well explained in the physical memory because it has been paged-out, and re- vmstat man page, but the most important reads it in from disk. columns for this article are free, si and so. The Page-ins are common, normal and are not a cause for free column shows the amount of free memory, si shows concern. For example, when an application first starts up, page-ins and so shows page-outs. In this example, the its executable image and data are paged-in. This is normal so column is zero consistently, indicating there are no behaviour. page-outs. Page-outs, however, can be a sign of trouble. When the kernel detects that memory is running low, it attempts to The abbreviations so and si are used instead of the free up memory by paging out. Although this may happen more accurate po and pi for historical reasons. briefly from time to time, if page-outs are plentiful and Here’s an example of a system with paging activity: constant, the kernel can reach a point where it’s actually spending more time managing paging activity than running procs memory swap io system cpu the applications, and system performance suffers. This r b w swpd free buff cache si so bi bo in cs us sy id woeful state is referred to as thrashing. Using swap space is not inherently bad. Rather, it’s 1 0 0 13344 1444 1308 19692 0 168 129 42 1505 713 20 11 69 intense paging activity that’s problematic. For instance, if 1 0 0 13856 1640 1308 18524 64 516 379 129 4341 646 24 34 42 your most-memory-intensive application is idle, it’s fine for 3 0 0 13856 1084 1308 18316 56 64 14 0 320 1022 84 9 8 portions of it to be set aside when another large job is active. Memory pages belonging to an idle application are Notice the nonzero so values indicating there is not better set aside so the kernel can use physical memory for enough physical memory and the kernel is paging out. You disk buffering. can use top and ps to identify the processes that are using the most memory. Using vmstat You also can use top to show memory and swap vmstat, as its name suggests, reports virtual memory statistics. Here is an example of the uppermost portion of a statistics. It shows how much virtual memory there is, how typical top report: much is free and paging activity. Most importantly, you can observe page-ins and page-outs as they happen. This is 14:23:19 up 348 days, 3:02, 1 user, load average: 0.00, 0.00, 0.00 extremely useful. 55 processes: 54 sleeping, 1 running, 0 zombie, 0 stopped To monitor the virtual memory activity on your system, CPU states: 0.0% user, 2.4% system, 0.0% nice, 97.6% idle it’s best to use vmstat with a delay. A delay is the number Mem: 481076K total, 367508K used, 113568K free, 4712K buffers of seconds between updates. If you don’t supply a delay, Swap: 1004052K total, 29852K used, 974200K free, 244396K cached vmstat reports the averages since the last boot and quit. Five seconds is the recommended delay interval. For more information about top, see the top man page. To run vmstat with a five-second delay, issue the command: This can be viewed by issuing the command man top at the command prompt. vmstat 5 Conclusion You also can specify a count, which indicates how many It isn’t necessarily bad for your system to be using some of updates you want to see before vmstat quits. If you don’t its swap space. But if you discover your system is often specify a count, the count defaults to infinity, but you can running low on physical memory and paging is causing stop output with Ctrl-C. performance to suffer, add more memory. If you can’t add To run vmstat with ten updates, five seconds apart, more memory, run memory-intensive jobs at different issue the command: times of the day and avoid running non-essential jobs when memory demand is high or distribute jobs across multiple vmstat 5 10 systems if possible. Here’s an example of the output generated by a system By: Brian K Tanaka. The author has been a UNIX system free of paging activity: administrator since 1994 with companies such as SGI, Intuit and RealNetworks. He can be reached at [email protected]. procs memory swap io system cpu Courtesy: Linux Journal r b w swpd free buff cache si so bi bo in cs us sy id 88 DECEMBER 2005 | LINUX FOR YOU | www.linuxforu.com C M Y K.
Recommended publications
  • Web Vmstat Any Distros, Especially Here’S Where Web Vmstat Comes Those Targeted at In
    FOSSPICKS Sparkling gems and new releases from the world of FOSSpicks Free and Open Source Software Mike Saunders has spent a decade mining the internet for free software treasures. Here’s the result of his latest haul… Shiny statistics in a browser Web VMStat any distros, especially Here’s where Web VMStat comes those targeted at in. It’s a system monitor that runs Madvanced users, ship an HTTP server, so you can connect with shiny system monitoring tools to it via a web browser and see on the desktop. Conky is one such fancy CSS-driven charts. Before you tool, while GKrellM was all the rage install it, you’ll need to get the in the last decade, and they are websocketd utility, which you can genuinely useful for keeping tabs find at https://github.com/ on your boxes, especially when joewalnes/websocketd. Helpfully, you’re an admin in charge of the developer has made pre- various servers. compiled executables available, so Now, pretty much all major you can just grab the 32-bit or distros include a useful command 64-bit tarball, extract it and there line tool for monitoring system you have it: websocketd. (Of course, Here’s the standard output for vmstat – not very interesting, right? resource usage: vmstat. Enter if you’re especially security vmstat 1 in a terminal window and conscious, you can compile it from copy the aforementioned you’ll see a regularly updating (once its source code.) websocketd into the same place. per second) bunch of statistics, Next, clone the Web VMStat Git Then just enter: showing CPU usage, free RAM, repository (or grab the Zip file and ./run swap usage and so forth.
    [Show full text]
  • Performance, Scalability on the Server Side
    Performance, Scalability on the Server Side John VanDyk Presented at Des Moines Web Geeks 9/21/2009 Who is this guy? History • Apple // • Macintosh • Windows 3.1- Server 2008R2 • Digital Unix (Tru64) • Linux (primarily RHEL) • FreeBSD Systems Iʼve worked with over the years. Languages • Perl • Userland Frontier™ • Python • Java • Ruby • PHP Languages Iʼve worked with over the years (Userland Frontier™ʼs integrated language is UserTalk™) Open source developer since 2000 Perl/Python/PHP MySQL Apache Linux The LAMP stack. Time to Serve Request Number of Clients Performance vs. scalability. network in network out RAM CPU Storage These are the basic laws of physics. All bottlenecks are caused by one of these four resources. Disk-bound •To o l s •iostat •vmstat Determine if you are disk-bound by measuring throughput. vmstat (BSD) procs memory page disk faults cpu r b w avm fre flt re pi po fr sr tw0 in sy cs us sy id 0 2 0 799M 842M 27 0 0 0 12 0 23 344 2906 1549 1 1 98 3 3 0 869M 789M 5045 0 0 0 406 0 10 1311 17200 5301 12 4 84 3 5 0 923M 794M 5219 0 0 0 5178 0 27 1825 21496 6903 35 8 57 1 2 0 931M 784M 909 0 0 0 146 0 12 955 9157 3570 8 4 88 blocked plenty of RAM, idle processes no swapping CPUs A disk-bound FreeBSD machine. b = blocked for resources fr = pages freed/sec cs = context switches avm = active virtual pages in = interrupts flt = memory page faults sy = system calls per interval vmstat (RHEL5) # vmstat -S M 5 25 procs ---------memory-------- --swap- ---io--- --system- -----cpu------ r b swpd free buff cache si so bi bo in cs us sy id wa st 1 0 0 1301 194 5531 0 0 0 29 1454 2256 24 20 56 0 0 3 0 0 1257 194 5531 0 0 0 40 2087 2336 34 27 39 0 0 2 0 0 1183 194 5531 0 0 0 53 1658 2763 33 28 39 0 0 0 0 0 1344 194 5531 0 0 0 34 1807 2125 29 19 52 0 0 no blocked busy but not processes overloaded CPU in = interrupts/sec cs = context switches/sec wa = time waiting for I/O Solving disk bottlenecks • Separate spindles (logs and databases) • Get rid of atime updates! • Minimize writes • Move temp writes to /dev/shm Overview of what weʼre about to dive into.
    [Show full text]
  • Linux Performance Tools
    Linux Performance Tools Brendan Gregg Senior Performance Architect Performance Engineering Team [email protected] @brendangregg This Tutorial • A tour of many Linux performance tools – To show you what can be done – With guidance for how to do it • This includes objectives, discussion, live demos – See the video of this tutorial Observability Benchmarking Tuning Stac Tuning • Massive AWS EC2 Linux cloud – 10s of thousands of cloud instances • FreeBSD for content delivery – ~33% of US Internet traffic at night • Over 50M subscribers – Recently launched in ANZ • Use Linux server tools as needed – After cloud monitoring (Atlas, etc.) and instance monitoring (Vector) tools Agenda • Methodologies • Tools • Tool Types: – Observability – Benchmarking – Tuning – Static • Profiling • Tracing Methodologies Methodologies • Objectives: – Recognize the Streetlight Anti-Method – Perform the Workload Characterization Method – Perform the USE Method – Learn how to start with the questions, before using tools – Be aware of other methodologies My system is slow… DEMO & DISCUSSION Methodologies • There are dozens of performance tools for Linux – Packages: sysstat, procps, coreutils, … – Commercial products • Methodologies can provide guidance for choosing and using tools effectively • A starting point, a process, and an ending point An#-Methodologies • The lack of a deliberate methodology… Street Light An<-Method 1. Pick observability tools that are: – Familiar – Found on the Internet – Found at random 2. Run tools 3. Look for obvious issues Drunk Man An<-Method • Tune things at random until the problem goes away Blame Someone Else An<-Method 1. Find a system or environment component you are not responsible for 2. Hypothesize that the issue is with that component 3. Redirect the issue to the responsible team 4.
    [Show full text]
  • Java Bytecode Manipulation Framework
    Notice About this document The following copyright statements and licenses apply to software components that are distributed with various versions of the OnCommand Performance Manager products. Your product does not necessarily use all the software components referred to below. Where required, source code is published at the following location: ftp://ftp.netapp.com/frm-ntap/opensource/ 215-09632 _A0_ur001 -Copyright 2014 NetApp, Inc. All rights reserved. 1 Notice Copyrights and licenses The following component is subject to the ANTLR License • ANTLR, ANother Tool for Language Recognition - 2.7.6 © Copyright ANTLR / Terence Parr 2009 ANTLR License SOFTWARE RIGHTS ANTLR 1989-2004 Developed by Terence Parr Partially supported by University of San Francisco & jGuru.com We reserve no legal rights to the ANTLR--it is fully in the public domain. An individual or company may do whatever they wish with source code distributed with ANTLR or the code generated by ANTLR, including the incorporation of ANTLR, or its output, into commerical software. We encourage users to develop software with ANTLR. However, we do ask that credit is given to us for developing ANTLR. By "credit", we mean that if you use ANTLR or incorporate any source code into one of your programs (commercial product, research project, or otherwise) that you acknowledge this fact somewhere in the documentation, research report, etc... If you like ANTLR and have developed a nice tool with the output, please mention that you developed it using ANTLR. In addition, we ask that the headers remain intact in our source code. As long as these guidelines are kept, we expect to continue enhancing this system and expect to make other tools available as they are completed.
    [Show full text]
  • UNIX OS Agent User's Guide
    IBM Tivoli Monitoring Version 6.3.0 UNIX OS Agent User's Guide SC22-5452-00 IBM Tivoli Monitoring Version 6.3.0 UNIX OS Agent User's Guide SC22-5452-00 Note Before using this information and the product it supports, read the information in “Notices” on page 399. This edition applies to version 6, release 3 of IBM Tivoli Monitoring (product number 5724-C04) and to all subsequent releases and modifications until otherwise indicated in new editions. © Copyright IBM Corporation 1994, 2013. US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Tables ...............vii Solaris System CPU Workload workspace ....28 Solaris Zone Processes workspace .......28 Chapter 1. Using the monitoring agent . 1 Solaris Zones workspace ..........28 System Details workspace .........28 New in this release ............2 System Information workspace ........29 Components of the monitoring agent ......3 Top CPU-Memory %-VSize Details workspace . 30 User interface options ...........4 UNIX OS workspace ...........30 UNIX Detail workspace ..........31 Chapter 2. Requirements for the Users workspace ............31 monitoring agent ...........5 Enabling the Monitoring Agent for UNIX OS to run Chapter 4. Attributes .........33 as a nonroot user .............7 Agent Availability Management Status attributes . 36 Securing your IBM Tivoli Monitoring installation 7 Agent Active Runtime Status attributes .....37 Setting overall file ownership and permissions for AIX AMS attributes............38
    [Show full text]
  • System Analysis and Tuning Guide System Analysis and Tuning Guide SUSE Linux Enterprise Server 15 SP1
    SUSE Linux Enterprise Server 15 SP1 System Analysis and Tuning Guide System Analysis and Tuning Guide SUSE Linux Enterprise Server 15 SP1 An administrator's guide for problem detection, resolution and optimization. Find how to inspect and optimize your system by means of monitoring tools and how to eciently manage resources. Also contains an overview of common problems and solutions and of additional help and documentation resources. Publication Date: September 24, 2021 SUSE LLC 1800 South Novell Place Provo, UT 84606 USA https://documentation.suse.com Copyright © 2006– 2021 SUSE LLC and contributors. All rights reserved. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or (at your option) version 1.3; with the Invariant Section being this copyright notice and license. A copy of the license version 1.2 is included in the section entitled “GNU Free Documentation License”. For SUSE trademarks, see https://www.suse.com/company/legal/ . All other third-party trademarks are the property of their respective owners. Trademark symbols (®, ™ etc.) denote trademarks of SUSE and its aliates. Asterisks (*) denote third-party trademarks. All information found in this book has been compiled with utmost attention to detail. However, this does not guarantee complete accuracy. Neither SUSE LLC, its aliates, the authors nor the translators shall be held liable for possible errors or the consequences thereof. Contents About This Guide xii 1 Available Documentation xiii
    [Show full text]
  • A Hitchhiker's Guide for Performance Assessment and Benchmarking SAS® Applications
    SAS Global Forum 2012 Systems Architecture and Administration Paper 367-2012 A Hitchhiker's guide for performance assessment & benchmarking SAS® applications Viraj Kumbhakarna, JPMorgan Chase, Columbus, OH Anurag Katare, Cognizant Technology Solutions, Teaneck, NJ ABSTRACT Almost every IT department today, needs some kind of an IT infrastructure to support different business processes in the organization. For a typical IT organization the ensemble of hardware, software, networking facilities may constitute the IT infrastructure. IT infrastructure is setup in order to develop, test, deliver, monitor, control or support IT services. Sometimes multiple applications may be hosted on a common server platform. With a continual increase in the user base, ever increasing volume of data and perpetual increase in number of processes required to support the growing business need, there often arises a need to upgrade the IT infrastructure. The paper discusses a stepwise approach to conduct a performance assessment and a performance benchmarking exercise required to assess the current state of the IT infrastructure (constituting the hardware and software) prior to an upgrade. It considers the following steps to be followed in order to proceed with a planned approach to implement process improvement. 1) Phase I: Assessment & Requirements gathering a) Understand ASIS process b) Assess AIX UNIX server configuration 2) Phase II: Performance assessment and benchmarking a) Server performance i) Server Utilization ii) Memory Utilization iii) Disk Utilization iv) Network traffic v) Resource Utilization b) Process Performance i) CPU Usage ii) Memory usage iii) Disc space 3) Phase III: Interpretation of results for performance improvement INTRODUCTION The paper discusses a practical assessment exercise that was carried out for benchmarking performance of SAS® processes on a UNIX AIX® 64 server having the SAS® 9.2 Foundation package installed on it.
    [Show full text]
  • SUSE Linux Enterprise Server 12 SP4 System Analysis and Tuning Guide System Analysis and Tuning Guide SUSE Linux Enterprise Server 12 SP4
    SUSE Linux Enterprise Server 12 SP4 System Analysis and Tuning Guide System Analysis and Tuning Guide SUSE Linux Enterprise Server 12 SP4 An administrator's guide for problem detection, resolution and optimization. Find how to inspect and optimize your system by means of monitoring tools and how to eciently manage resources. Also contains an overview of common problems and solutions and of additional help and documentation resources. Publication Date: September 24, 2021 SUSE LLC 1800 South Novell Place Provo, UT 84606 USA https://documentation.suse.com Copyright © 2006– 2021 SUSE LLC and contributors. All rights reserved. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or (at your option) version 1.3; with the Invariant Section being this copyright notice and license. A copy of the license version 1.2 is included in the section entitled “GNU Free Documentation License”. For SUSE trademarks, see https://www.suse.com/company/legal/ . All other third-party trademarks are the property of their respective owners. Trademark symbols (®, ™ etc.) denote trademarks of SUSE and its aliates. Asterisks (*) denote third-party trademarks. All information found in this book has been compiled with utmost attention to detail. However, this does not guarantee complete accuracy. Neither SUSE LLC, its aliates, the authors nor the translators shall be held liable for possible errors or the consequences thereof. Contents About This Guide xii 1 Available Documentation xiii
    [Show full text]
  • Whitepaper Why Nagios and Server Monitoring Are Failing Modern Apps
    An AppDynamics Business White Paper Why Nagios and Server Monitoring Are Failing Modern Apps Server monitoring is an important part of any data center monitoring architecture, but too often it becomes a crutch and a deterrent to successfully building out a holistic monitoring platform. Server status is only one indicator of application performance, so relying exclusively on server monitoring tools leaves organizations with large blind spots and unhappy end users. In this paper we will explore what server monitoring is and how it can (and should) fit into a larger application performance management platform. What is Server Monitoring? Server monitoring consists of monitoring operating system and associated hardware metrics for the servers that run your application. It’s the view of the world from the perspective of the server, but never from inside the running processes. Basic server monitoring metrics include CPU sys time, CPU wait time, used memory, free memory, disk queue length, % disk used, network collisions, adapter transmit rate, etc. Server monitoring is used by every IT organization in some shape or form. The 9s are an (Unintentional) Lie IT organizations are usually held to a standard of three, four, or five “nines” of availability, referring to the number of nines in. The table below defines each of the “nines” and translates their meaning into acceptable minutes of server downtime per year. Max Yearly Downtime The Nines Yearly Uptime (Minutes) (Minutes) 99.9% (3 nines) 525,074.5 525.5 99.99% (4 nines) 525,547.5 52.5 99.999% (5 nines) 525,594.8 5.2 Availability is usually measured at the server level by checking if the server is responding to requests.
    [Show full text]
  • Licensing Information User Manual Oracle® ILOM
    Licensing Information User Manual ® Oracle ILOM Firmware Release 3.2.x October 2018 Part No: E62005-12 October 2018 Licensing Information User Manual Oracle ILOM Firmware Release 3.2.x Part No: E62005-12 Copyright © 2016, 2018, Oracle and/or its affiliates. License Restrictions Warranty/Consequential Damages Disclaimer This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. Warranty Disclaimer The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing. Restricted Rights Notice If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, then the following notice is applicable: U.S. GOVERNMENT END USERS: Oracle programs (including any operating system, integrated software, any programs embedded, installed or activated on delivered hardware, and modifications of such programs) and Oracle computer documentation or other Oracle data delivered to or accessed by U.S. Government
    [Show full text]
  • Embedded Android
    Embedded Android 1 These slides are made available to you under a Creative Commons Delivered and/or customized by Share-Alike 3.0 license. The full terms of this license are here: https://creativecommons.org/licenses/by-sa/3.0/ Attribution requirements and misc., PLEASE READ: ● This slide must remain as-is in this specific location (slide #2), everything else you are free to change; including the logo :-) ● Use of figures in other documents must feature the below “Originals at” URL immediately under that figure and the below copyright notice where appropriate. ● You are free to fill in the “Delivered and/or customized by” space on the right as you see fit. ● You are FORBIDEN from using the default “About” slide as-is or any of its contents. (C) Copyright 2010-2017, Opersys inc. These slides created by: Karim Yaghmour Originals at: www.opersys.com/training/embedded-android 2 About ● Author of: ● Introduced Linux Trace Toolkit in 1999 ● Originated Adeos and relayfs (kernel/relay.c) ● Training, Custom Dev, Consulting, ... 3 About Android ● Huge ● Fast moving ● Stealthy 4 About Android ● Huge ● Fast moving ● Stealthy Mainly: ● Internals-specifics are subject to change Therefore: ● Must learn to relearn every new release 5 Goals ● Master the intricacies of all components making up Android, including kernel Androidisms ● Get hands-on experience in building and customizing Android-based embedded systems ● Learn basics of Android app development ● Familiarize with the Android ecosystem 6 Format ● Tracks: ● Lecture ● Exercises ● Fast pace ● Lots
    [Show full text]
  • Informix Best Practices Configuration, ONCONFIG, CPU and Memory Usage
    Informix Best Practices Configuration, ONCONFIG, CPU and Memory Usage Webcast – February 23, 2017 by Lester Knutsen Lester Knutsen Lester Knutsen is President of Advanced DataTools Corporation, and has been building large Data Warehouse and Business Systems using Informix Database software since 1983. Lester focuses on large database performance tuning, training and consulting. Lester is a member of the IBM Gold Consultant program and was presented with one of the Inaugural IBM Data Champion awards by IBM. Lester was one of the founders of the International Informix Users Group and the Washington Area Informix User Group. [email protected] www.advancedatatools.com 703-256-0267 x102 Informix Best 2 Practices Overview • CPU Recommendations and Best Practices • Memory Recommendations and Best Practices • ONCONFIG Recommendations and Best Practices – Basic Settings – Additional Key Settings Informix Best 3 Practices CPU – Central Processor Unit Recommendations for Informix and Best Practices CPU Terms • Socket = One Chip or Processor • Cores per Socket = How many cores run on a chip. A core only runs one process at a time. • Hyper-Threads or SMT threads per Core = Many Cores have the ability to run multiple threads. No matter how many threads run on a Core, only one thread can run at a time on a core. Hyper-Threads will appear as additional Virtual Cores. • Chip speed is measured in gigahertz (GHz); this is the speed of a single core of your processor. • PVU - IBM Processor Value Unit = A unit of measure used to differentiate licensing
    [Show full text]