Rosetta Stone for Unix.Pdf
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Freenas® 11.0 User Guide
FreeNAS® 11.0 User Guide June 2017 Edition FreeNAS® IS © 2011-2017 iXsystems FreeNAS® AND THE FreeNAS® LOGO ARE REGISTERED TRADEMARKS OF iXsystems FreeBSD® IS A REGISTERED TRADEMARK OF THE FreeBSD Foundation WRITTEN BY USERS OF THE FreeNAS® network-attached STORAGE OPERATING system. VERSION 11.0 CopYRIGHT © 2011-2017 iXsystems (https://www.ixsystems.com/) CONTENTS WELCOME....................................................1 TYPOGRAPHIC Conventions...........................................2 1 INTRODUCTION 3 1.1 NeW FeaturES IN 11.0..........................................3 1.2 HarDWARE Recommendations.....................................4 1.2.1 RAM...............................................5 1.2.2 The OperATING System DeVICE.................................5 1.2.3 StorAGE Disks AND ContrOLLERS.................................6 1.2.4 Network INTERFACES.......................................7 1.3 Getting Started WITH ZFS........................................8 2 INSTALLING AND UpgrADING 9 2.1 Getting FreeNAS® ............................................9 2.2 PrEPARING THE Media.......................................... 10 2.2.1 On FreeBSD OR Linux...................................... 10 2.2.2 On WindoWS.......................................... 11 2.2.3 On OS X............................................. 11 2.3 Performing THE INSTALLATION....................................... 12 2.4 INSTALLATION TROUBLESHOOTING...................................... 18 2.5 UpgrADING................................................ 19 2.5.1 Caveats:............................................ -
Kutrace: Where Have All the Nanoseconds Gone?
KUTrace: Where have all the nanoseconds gone? Richard Sites, Invited Professor EPFL 2017.10.27 Tracing Summit 2017, Prague Richard Sites 2017.10.27 Outline Observability problem statement KUTrace solution Tracing design goals Goals drive the design Results Display Design Comparisons Conclusions Richard Sites 2017.10.27 Observability problem statement Richard Sites 2017.10.27 3 Problem statement This talk is about tail latency in real user-facing datacenter transactions. It is not about batch processing throughput, nor about benchmarks. Context: A datacenter of perhaps 20,000 servers running software services that spread work for a user-facing transaction across a few hundred or thousand machines in parallel. Each server handles hundreds of transactions per second. Some transactions are unusually slow, but not repeatably. Slow transactions occur unpredictably, but there are several per minute. We wish to observe where all the time goes in such transactions, and observe why they are slow. Richard Sites 2017.10.27 Problem statement ☞ Some transactions are unusually slow, but not repeatably. ∴ There is some source of interference just before or during a slow transaction. Understanding tail latency requires complete traces of CPU events over a few minutes, with small enough CPU and memory overhead to be usable under busiest-hour live load. Existing tracing tools have much-too-high overhead. Problem: build better tail-latency observation tools Richard Sites 2017.10.27 KUTrace solution Richard Sites 2017.10.27 6 KUTrace solution KUTrace uses minimal Linux kernel patches on a single server to trace every transition between kernel- and user-mode execution, on every CPU core, with small enough overhead to use routinely on live loads. -
Debugging Kernel Problems
Debugging Kernel Problems by GregLehey Edition for AsiaBSDCon 2004 Taipei, 13 March 2004 Debugging Kernel Problems by GregLehey([email protected]) Copyright © 1995-2004 GregLehey 3Debugging Kernel Problems Preface Debugging kernel problems is a black art. Not manypeople do it, and documentation is rare, in- accurate and incomplete. This document is no exception: faced with the choice of accuracyand completeness, I chose to attempt the latter.Asusual, time was the limiting factor,and this draft is still in beta status. This is a typical situation for the whole topic of kernel debugging: building debug tools and documentation is expensive,and the people who write them are also the people who use them, so there'satendencytobuild as much of the tool as necessary to do the job at hand. If the tool is well-written, it will be reusable by the next person who looks at a particular area; if not, it might fall into disuse. Consider this book a starting point for your own develop- ment of debugging tools, and remember: more than anywhere else, this is an area with ``some as- sembly required''. Debugging Kernel Problems 4 1 Introduction Operating systems fail. All operating systems contain bugs, and theywill sometimes cause the system to behave incorrectly.The BSD kernels are no exception. Compared to most other oper- ating systems, both free and commercial, the BSD kernels offer a large number of debugging tools. This tutorial examines the options available both to the experienced end user and also to the developer. In this tutorial, we’ll look at the following topics: • Howand whykernels fail. -
Tigersharc DSP Hardware Specification, Revision 1.0.2, Direct Memory Access
7 DIRECT MEMORY ACCESS Figure 7-0. Table 7-0. Listing 7-0. Overview Direct Memory Access (DMA) is a mechanism for transferring data with- out core being involved. The TigerSHARC® DSP’s on-chip DMA controller relieves the core processor of the burden of moving data between internal memory and an external device, external memory, or between link ports and internal or external memory. The fully-integrated DMA controller allows the TigerSHARC® DSP core processor, or an external device, to specify data transfer operations and return to normal processing while the DMA controller carries out the data transfers in the background. The TigerSHARC® DSP DMA competes with other masters for internal memory access. For more information, see “Architecture and Microarchi- tecture Overview” on page 6-7. This conflict is minimized due to the large internal memory bandwidth that is available. The DMA includes 14 DMA channels, four of which are dedicated to external memory devices, eight to link ports, and two to AutoDMA registers. TigerSHARC DSP Hardware Specification 7 - 1 Overview Figure 7-1 shows a block diagram of the TigerSHARC® DSP’s DMA controller. TRANSMITTER RECEIVER TCB TCB REGISTERS REGISTERS Internal DMA DMA CONTROLLER Bus Requests Interface Figure 7-1. DMA Block Diagram Data Transfers — General Information The DMA controller can perform several types of data transfers: • Internal memory ⇒ external memory and memory-mapped periph- erals • Internal memory ⇒ internal memory of other TigerSHARC® DSPs residing on the cluster bus • Internal memory ⇒ host processor • Internal memory ⇒ link port I/O • External memory ⇒ external peripherals 7 - 2 TigerSHARC DSP Hardware Specification Direct Memory Access • External memory ⇒ internal memory • External memory ⇒ link port I/O • Link port I/O ⇒ internal memory • Link port I/O ⇒ external memory • Cluster bus master via AutoDMA registers ⇒ internal memory Internal-to-internal memory transfers are not directly supported. -
Openbsd and Soekris
OpenBSD and Soekris UUASC meeting June 3, 2004 Presented by Arild Jensen Outline ● What is OpenBSD and where do I get it? ● Built-in security features ● Maintaining an OpenBSD system ● The PF packet filter Outline (cont'd) ● What is Soekris and where do I get it? ● Different models and accessories ● Getting OpenBSD onto a Soekris box ● Maintaining a Soekris/OpenBSD solution What is OpenBSD? History BSD Net/2 (4.3BSD Lite) NetBSD 0.8 386BSD 0.0 NetBSD 0.9 386BSD 0.1 4.4BSD Lite 1 NetBSD 1.0 386BSD 1.0 NetBSD 1.1 OpenBSD FreeBSD What is OpenBSD? From the creators: “...freely available, multi- platform 4.4BSD-based UNIX-like OS.” Emphasis on: ● Portability ● Standardization ● Correctness ● Proactive Security ● Integrated Cryptography ...and where do I get it? www.openbsd.org CD sales only No .iso downloads $40 Portability ● i386 ● MVME68k ● Sparc ● MVME88k ● Sparc64 ● AMD64 ● HP300 ● CATS (ARM) ● Mac68k ● MacPPC ● HPPA Standardization The Story of CARP ● Firewall failover desired ● IEEE VRRP (Virtual router redundancy protocol) ● Cisco patents involved, HSRP protocol ● Cisco and Alcatel dispute ● Birth of CARP (Common address redundancy protocol ● Early implementation included in OpenBSD 3.5 Correctness The Audit Process ● 6-12 member security team ● Continuous audit of code multiple times by different people ● Security holes and common errors ● Result: Newly discovered bugs often already fixed in OpenBSD Pro-active Security Source Code Run Time ● ProPolice ● Privilege Separation – Buffer overflow – Avoid running as root protection – Dual-process setup – Similar to Stackguard – Daemons being ● W^X converted – Write xor Execute ● Chroot – Fine-grained memory – Apache /var/www permission layout – BIND /var/named – Only on some architectures Cryptography ● Based outside of U.S. -
Performance, Scalability on the Server Side
Performance, Scalability on the Server Side John VanDyk Presented at Des Moines Web Geeks 9/21/2009 Who is this guy? History • Apple // • Macintosh • Windows 3.1- Server 2008R2 • Digital Unix (Tru64) • Linux (primarily RHEL) • FreeBSD Systems Iʼve worked with over the years. Languages • Perl • Userland Frontier™ • Python • Java • Ruby • PHP Languages Iʼve worked with over the years (Userland Frontier™ʼs integrated language is UserTalk™) Open source developer since 2000 Perl/Python/PHP MySQL Apache Linux The LAMP stack. Time to Serve Request Number of Clients Performance vs. scalability. network in network out RAM CPU Storage These are the basic laws of physics. All bottlenecks are caused by one of these four resources. Disk-bound •To o l s •iostat •vmstat Determine if you are disk-bound by measuring throughput. vmstat (BSD) procs memory page disk faults cpu r b w avm fre flt re pi po fr sr tw0 in sy cs us sy id 0 2 0 799M 842M 27 0 0 0 12 0 23 344 2906 1549 1 1 98 3 3 0 869M 789M 5045 0 0 0 406 0 10 1311 17200 5301 12 4 84 3 5 0 923M 794M 5219 0 0 0 5178 0 27 1825 21496 6903 35 8 57 1 2 0 931M 784M 909 0 0 0 146 0 12 955 9157 3570 8 4 88 blocked plenty of RAM, idle processes no swapping CPUs A disk-bound FreeBSD machine. b = blocked for resources fr = pages freed/sec cs = context switches avm = active virtual pages in = interrupts flt = memory page faults sy = system calls per interval vmstat (RHEL5) # vmstat -S M 5 25 procs ---------memory-------- --swap- ---io--- --system- -----cpu------ r b swpd free buff cache si so bi bo in cs us sy id wa st 1 0 0 1301 194 5531 0 0 0 29 1454 2256 24 20 56 0 0 3 0 0 1257 194 5531 0 0 0 40 2087 2336 34 27 39 0 0 2 0 0 1183 194 5531 0 0 0 53 1658 2763 33 28 39 0 0 0 0 0 1344 194 5531 0 0 0 34 1807 2125 29 19 52 0 0 no blocked busy but not processes overloaded CPU in = interrupts/sec cs = context switches/sec wa = time waiting for I/O Solving disk bottlenecks • Separate spindles (logs and databases) • Get rid of atime updates! • Minimize writes • Move temp writes to /dev/shm Overview of what weʼre about to dive into. -
Chapter 1. Origins of Mac OS X
1 Chapter 1. Origins of Mac OS X "Most ideas come from previous ideas." Alan Curtis Kay The Mac OS X operating system represents a rather successful coming together of paradigms, ideologies, and technologies that have often resisted each other in the past. A good example is the cordial relationship that exists between the command-line and graphical interfaces in Mac OS X. The system is a result of the trials and tribulations of Apple and NeXT, as well as their user and developer communities. Mac OS X exemplifies how a capable system can result from the direct or indirect efforts of corporations, academic and research communities, the Open Source and Free Software movements, and, of course, individuals. Apple has been around since 1976, and many accounts of its history have been told. If the story of Apple as a company is fascinating, so is the technical history of Apple's operating systems. In this chapter,[1] we will trace the history of Mac OS X, discussing several technologies whose confluence eventually led to the modern-day Apple operating system. [1] This book's accompanying web site (www.osxbook.com) provides a more detailed technical history of all of Apple's operating systems. 1 2 2 1 1.1. Apple's Quest for the[2] Operating System [2] Whereas the word "the" is used here to designate prominence and desirability, it is an interesting coincidence that "THE" was the name of a multiprogramming system described by Edsger W. Dijkstra in a 1968 paper. It was March 1988. The Macintosh had been around for four years. -
Xcode Package from App Store
KH Computational Physics- 2016 Introduction Setting up your computing environment Installation • MAC or Linux are the preferred operating system in this course on scientific computing. • Windows can be used, but the most important programs must be installed – python : There is a nice package ”Enthought Python Distribution” http://www.enthought.com/products/edudownload.php – C++ and Fortran compiler – BLAS&LAPACK for linear algebra – plotting program such as gnuplot Kristjan Haule, 2016 –1– KH Computational Physics- 2016 Introduction Software for this course: Essentials: • Python, and its packages in particular numpy, scipy, matplotlib • C++ compiler such as gcc • Text editor for coding (for example Emacs, Aquamacs, Enthought’s IDLE) • make to execute makefiles Highly Recommended: • Fortran compiler, such as gfortran or intel fortran • BLAS& LAPACK library for linear algebra (most likely provided by vendor) • open mp enabled fortran and C++ compiler Useful: • gnuplot for fast plotting. • gsl (Gnu scientific library) for implementation of various scientific algorithms. Kristjan Haule, 2016 –2– KH Computational Physics- 2016 Introduction Installation on MAC • Install Xcode package from App Store. • Install ‘‘Command Line Tools’’ from Apple’s software site. For Mavericks and lafter, open Xcode program, and choose from the menu Xcode -> Open Developer Tool -> More Developer Tools... You will be linked to the Apple page that allows you to access downloads for Xcode. You wil have to register as a developer (free). Search for the Xcode Command Line Tools in the search box in the upper left. Download and install the correct version of the Command Line Tools, for example for OS ”El Capitan” and Xcode 7.2, Kristjan Haule, 2016 –3– KH Computational Physics- 2016 Introduction you need Command Line Tools OS X 10.11 for Xcode 7.2 Apple’s Xcode contains many libraries and compilers for Mac systems. -
Optimization of System's Performance with Kernel Tracing by Cohort Intelligence
I.J. Information Technology and Computer Science, 2017, 6, 59-66 Published Online June 2017 in MECS (http://www.mecs-press.org/) DOI: 10.5815/ijitcs.2017.06.08 Optimization of System’s Performance with Kernel Tracing by Cohort Intelligence Aniket B. Tate Dept. of Computer Engineering, Vishwakarma Institute of Information Technology, Pune, 411048, India E-mail: [email protected] Laxmi A. Bewoor Dept. of Computer Engineering, Vishwakarma Institute of Information Technology, Pune, 411048, India E-mail: [email protected] Abstract—Linux tracing tools are used to record the execution time etc. which shows better results. While events running in the background on the system. But scheduling a process on one of the cores, the scheduler these tools lack to analyze the log data. In the field of considers the average waiting time, turnaround time, time Artificial Intelligence Cohort Intelligence (CI) is recently quantum for a process, number of context switches, proposed technique, which works on the principle of self- earliness, the tardiness of process etc. But the scheduler learning within a cohort. This paper presents an approach does not take CPU load into consideration. As a result of to optimize the performance of the system by tracing the this, the cores get unevenly loaded and many of the cores system, then extract the information from trace data and will be kept in ideal state. Cohort Intelligence (CI) is pass it to cohort intelligence algorithm. The output of recently introduced meta-heuristics [6][7] it works on cohort intelligence algorithm shows, how the load of the self-supervised learning behavior in a cohort. -
BSD UNIX Toolbox 1000+ Commands for Freebsd, Openbsd
76034ffirs.qxd:Toolbox 4/2/08 12:50 PM Page iii BSD UNIX® TOOLBOX 1000+ Commands for FreeBSD®, OpenBSD, and NetBSD®Power Users Christopher Negus François Caen 76034ffirs.qxd:Toolbox 4/2/08 12:50 PM Page ii 76034ffirs.qxd:Toolbox 4/2/08 12:50 PM Page i BSD UNIX® TOOLBOX 76034ffirs.qxd:Toolbox 4/2/08 12:50 PM Page ii 76034ffirs.qxd:Toolbox 4/2/08 12:50 PM Page iii BSD UNIX® TOOLBOX 1000+ Commands for FreeBSD®, OpenBSD, and NetBSD®Power Users Christopher Negus François Caen 76034ffirs.qxd:Toolbox 4/2/08 12:50 PM Page iv BSD UNIX® Toolbox: 1000+ Commands for FreeBSD®, OpenBSD, and NetBSD® Power Users Published by Wiley Publishing, Inc. 10475 Crosspoint Boulevard Indianapolis, IN 46256 www.wiley.com Copyright © 2008 by Wiley Publishing, Inc., Indianapolis, Indiana Published simultaneously in Canada ISBN: 978-0-470-37603-4 Manufactured in the United States of America 10 9 8 7 6 5 4 3 2 1 Library of Congress Cataloging-in-Publication Data is available from the publisher. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600. Requests to the Publisher for permis- sion should be addressed to the Legal Department, Wiley Publishing, Inc., 10475 Crosspoint Blvd., Indianapolis, IN 46256, (317) 572-3447, fax (317) 572-4355, or online at http://www.wiley.com/go/permissions. -
Absolute BSD—The Ultimate Guide to Freebsd Table of Contents Absolute BSD—The Ultimate Guide to Freebsd
Absolute BSD—The Ultimate Guide to FreeBSD Table of Contents Absolute BSD—The Ultimate Guide to FreeBSD............................................................................1 Dedication..........................................................................................................................................3 Foreword............................................................................................................................................4 Introduction........................................................................................................................................5 What Is FreeBSD?...................................................................................................................5 How Did FreeBSD Get Here?..................................................................................................5 The BSD License: BSD Goes Public.......................................................................................6 The Birth of Modern FreeBSD.................................................................................................6 FreeBSD Development............................................................................................................7 Committers.........................................................................................................................7 Contributors........................................................................................................................8 Users..................................................................................................................................8 -
The Complete Freebsd
The Complete FreeBSD® If you find errors in this book, please report them to Greg Lehey <grog@Free- BSD.org> for inclusion in the errata list. The Complete FreeBSD® Fourth Edition Tenth anniversary version, 24 February 2006 Greg Lehey The Complete FreeBSD® by Greg Lehey <[email protected]> Copyright © 1996, 1997, 1999, 2002, 2003, 2006 by Greg Lehey. This book is licensed under the Creative Commons “Attribution-NonCommercial-ShareAlike 2.5” license. The full text is located at http://creativecommons.org/licenses/by-nc-sa/2.5/legalcode. You are free: • to copy, distribute, display, and perform the work • to make derivative works under the following conditions: • Attribution. You must attribute the work in the manner specified by the author or licensor. • Noncommercial. You may not use this work for commercial purposes. This clause is modified from the original by the provision: You may use this book for commercial purposes if you pay me the sum of USD 20 per copy printed (whether sold or not). You must also agree to allow inspection of printing records and other material necessary to confirm the royalty sums. The purpose of this clause is to make it attractive to negotiate sensible royalties before printing. • Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under a license identical to this one. • For any reuse or distribution, you must make clear to others the license terms of this work. • Any of these conditions can be waived if you get permission from the copyright holder. Your fair use and other rights are in no way affected by the above.