DOCSLIB.ORG

Advanced Bash-Scripting Guide

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Advanced Bash-Scripting Guide Advanced Bash−Scripting Guide An in−depth exploration of the art of shell scripting Mendel Cooper <[email protected]> 3.8 26 February 2006 Revision History Revision 3.6 28 Aug 2005 Revised by: mc 'POKEBERRY' release: Bugfix Update. Revision 3.7 23 Oct 2005 Revised by: mc 'WHORTLEBERRY' release: Bugfix Update. Revision 3.8 26 Feb 2006 Revised by: mc 'BLAEBERRY' release: Minor Update. This tutorial assumes no previous knowledge of scripting or programming, but progresses rapidly toward an intermediate/advanced level of instruction . all the while sneaking in little snippets of UNIX® wisdom and lore. It serves as a textbook, a manual for self−study, and a reference and source of knowledge on shell scripting techniques. The exercises and heavily−commented examples invite active reader participation, under the premise that the only way to really learn scripting is to write scripts. This book is suitable for classroom use as a general introduction to programming concepts. The latest update of this document, as an archived, bzip2−ed "tarball" including both the SGML source and rendered HTML, may be downloaded from the author's home site. A pdf version is also available. See the change log for a revision history. Dedication For Anita, the source of all the magic Advanced Bash−Scripting Guide Table of Contents Chapter 1. Why Shell Programming?...............................................................................................................1 Chapter 2. Starting Off With a Sha−Bang.......................................................................................................3 2.1. Invoking the script............................................................................................................................6 2.2. Preliminary Exercises.......................................................................................................................6 Part 2. Basics.......................................................................................................................................................7 Chapter 3. Special Characters...........................................................................................................................8 Chapter 4. Introduction to Variables and Parameters..................................................................................25 4.1. Variable Substitution......................................................................................................................25 4.2. Variable Assignment.......................................................................................................................27 4.3. Bash Variables Are Untyped..........................................................................................................29 4.4. Special Variable Types...................................................................................................................30 Chapter 5. Quoting...........................................................................................................................................34 5.1. Quoting Variables...........................................................................................................................34 5.2. Escaping..........................................................................................................................................35 Chapter 6. Exit and Exit Status.......................................................................................................................41 Chapter 7. Tests................................................................................................................................................43 7.1. Test Constructs...............................................................................................................................43 7.2. File test operators............................................................................................................................49 7.3. Other Comparison Operators..........................................................................................................52 7.4. Nested if/then Condition Tests.......................................................................................................57 7.5. Testing Your Knowledge of Tests..................................................................................................57 Chapter 8. Operations and Related Topics....................................................................................................59 8.1. Operators.........................................................................................................................................59 8.2. Numerical Constants.......................................................................................................................65 Part 3. Beyond the Basics.................................................................................................................................67 Chapter 9. Variables Revisited........................................................................................................................68 9.1. Internal Variables............................................................................................................................68 9.2. Manipulating Strings.......................................................................................................................84 9.2.1. Manipulating strings using awk............................................................................................90 9.2.2. Further Discussion.................................................................................................................90 9.3. Parameter Substitution....................................................................................................................90 9.4. Typing variables: declare or typeset...............................................................................................99 9.5. Indirect References to Variables...................................................................................................101 9.6. $RANDOM: generate random integer..........................................................................................104 9.7. The Double Parentheses Construct...............................................................................................113 i Advanced Bash−Scripting Guide Table of Contents Chapter 10. Loops and Branches..................................................................................................................115 10.1. Loops..........................................................................................................................................115 10.2. Nested Loops..............................................................................................................................126 10.3. Loop Control...............................................................................................................................126 10.4. Testing and Branching................................................................................................................130 Chapter 11. Internal Commands and Builtins.............................................................................................137 11.1. Job Control Commands..............................................................................................................161 Chapter 12. External Filters, Programs and Commands...........................................................................166 12.1. Basic Commands........................................................................................................................166 12.2. Complex Commands...................................................................................................................171 12.3. Time / Date Commands..............................................................................................................180 12.4. Text Processing Commands........................................................................................................183 12.5. File and Archiving Commands...................................................................................................202 12.6. Communications Commands......................................................................................................218 12.7. Terminal Control Commands.....................................................................................................232 12.8. Math Commands.........................................................................................................................233 12.9. Miscellaneous Commands..........................................................................................................242 Chapter 13. System and Administrative Commands..................................................................................254 13.1. Analyzing a System Script..........................................................................................................279 Chapter 14. Command Substitution.............................................................................................................281 Chapter 15. Arithmetic Expansion................................................................................................................287 Chapter 16. I/O Redirection...........................................................................................................................288
Load more

Recommended publications
  • Version 7.8-Systemd
    Linux From Scratch Version 7.8-systemd Created by Gerard Beekmans Edited by Douglas R. Reno Linux From Scratch: Version 7.8-systemd by Created by Gerard Beekmans and Edited by Douglas R. Reno Copyright © 1999-2015 Gerard Beekmans Copyright © 1999-2015, Gerard Beekmans All rights reserved. This book is licensed under a Creative Commons License. Computer instructions may be extracted from the book under the MIT License. Linux® is a registered trademark of Linus Torvalds. Linux From Scratch - Version 7.8-systemd Table of Contents Preface .......................................................................................................................................................................... vii i. Foreword ............................................................................................................................................................. vii ii. Audience ............................................................................................................................................................ vii iii. LFS Target Architectures ................................................................................................................................ viii iv. LFS and Standards ............................................................................................................................................ ix v. Rationale for Packages in the Book .................................................................................................................... x vi. Prerequisites
    [Show full text]
  • Powerview Command Reference
    PowerView Command Reference TRACE32 Online Help TRACE32 Directory TRACE32 Index TRACE32 Documents ...................................................................................................................... PowerView User Interface ............................................................................................................ PowerView Command Reference .............................................................................................1 History ...................................................................................................................................... 12 ABORT ...................................................................................................................................... 13 ABORT Abort driver program 13 AREA ........................................................................................................................................ 14 AREA Message windows 14 AREA.CLEAR Clear area 15 AREA.CLOSE Close output file 15 AREA.Create Create or modify message area 16 AREA.Delete Delete message area 17 AREA.List Display a detailed list off all message areas 18 AREA.OPEN Open output file 20 AREA.PIPE Redirect area to stdout 21 AREA.RESet Reset areas 21 AREA.SAVE Save AREA window contents to file 21 AREA.Select Select area 22 AREA.STDERR Redirect area to stderr 23 AREA.STDOUT Redirect area to stdout 23 AREA.view Display message area in AREA window 24 AutoSTOre ..............................................................................................................................
    [Show full text]
  • Bash Guide for Beginners
    Bash Guide for Beginners Machtelt Garrels Garrels BVBA <tille wants no spam _at_ garrels dot be> Version 1.11 Last updated 20081227 Edition Bash Guide for Beginners Table of Contents Introduction.........................................................................................................................................................1 1. Why this guide?...................................................................................................................................1 2. Who should read this book?.................................................................................................................1 3. New versions, translations and availability.........................................................................................2 4. Revision History..................................................................................................................................2 5. Contributions.......................................................................................................................................3 6. Feedback..............................................................................................................................................3 7. Copyright information.........................................................................................................................3 8. What do you need?...............................................................................................................................4 9. Conventions used in this
    [Show full text]
  • Rhd256.Bowez Notes.2006-07-20.Txt Thu Jul 20 15:31
    ../rhd256.bowez_notes.2006-07-20.txt Thu Jul 20 15:31:29 2006 1 ==> ./01_intro/notes.txt <== ---------------------------------------------------------------------- ---------------------------------------------------------------------- 2.3 Bash Special Characaters 1.3 Prerequisates ---------------------------------------------------------------------- ---------------------------------------------------------------------- ------------------------------------------------------------------ Familiarity: Redirection (Stream Manipulation) ------------------------------------------------------------------ the classroom setup (server1, station1, etc...) the "pub" directory every process has three default file descriptors (streams): Common Tools: 0: stdin 1: stdout terminal: 2: stderr changing fonts, etc.... - by default, all are connected to the terminal. editors: - In RHL, can be accessed by /dev/std{in,out,err} emacs vi, vim, gvim < redirect descriptor input pico -w, gedit > redirect descriptor output (create or clobber) (less pager) >> redirect descriptor output (create or append) finding information: 2>&1 bind one descriptor to another (i.e., combine stderr and stdout) --help, -h >& bash shortcut for the same man pages (chapters) info pages (info, pinfo, nautilus) examples: /usr/share/doc grep root /etc/* rpm (-qf filename, -ql, -qi) grep root /etc/* > /tmp/out grep root /etc/* 2> /tmp/err ---------------------------------------------------------------------- grep root /etc/* > /tmp/out2 2> /tmp/err2 grep root /etc/* >& /tmp/all 1.4 Procedures
    [Show full text]
  • Computer Security 03
    Computer Security 03. Program Hijacking & Code Injection Paul Krzyzanowski Rutgers University Spring 2019 September 25, 2019 CS 419 © 2019 Paul Krzyzanowski 1 Top vulnerability concerns for 2019 MITRE, a non-profit organization that manages federally-funded research & development centers, publishes a list of top security weaknesses Rank Name Score 1 Improper Restriction of Operations within the Bounds of a Memory Buffer 75.56 2 Cross-site Scripting 45.69 3 Improper Input Validation 43.61 4 Information Exposure 32.12 5 Out-of-bounds Read 26.53 6 SQL Injection 24.54 7 Use After Free 17.94 8 Integer Overflow or Wraparound 17.35 9 Cross-Site Request Forgery (CSRF) 15.54 10 14.10 Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal') https://cwe.mitre.org/top25/archive/2019/2019_cwe_top25.html September 25, 2019 CS 419 © 2019 Paul Krzyzanowski 2 Hijacking Getting software to do something different from what the user or developer expected Examples: • Redirect web browser to a malicious site • Change DNS (IP address lookup) results • Change search engine • Change search paths to load different libraries or have different programs run • Intercept & alter messages Code injection Getting a program to process data in a way that it changes the execution of a program September 25, 2019 CS 419 © 2019 Paul Krzyzanowski 3 Bugs and mistakes • Most attacks are due to – Social engineering: getting a legitimate user to do something – Or bugs: using a program in a way it was not intended • Attacked system may be further weakened because
    [Show full text]
  • 11.6. Tempfile — Generate Temporary Files and Directories — Python V2
    Navigation • index • modules | • next | • previous | • Python v2.6.4 documentation » • The Python Standard Library » • 11. File and Directory Access » 11.6. tempfile — Generate temporary files and directories¶ This module generates temporary files and directories. It works on all supported platforms. In version 2.3 of Python, this module was overhauled for enhanced security. It now provides three new functions, NamedTemporaryFile(), mkstemp(), and mkdtemp(), which should eliminate all remaining need to use the insecure mktemp() function. Temporary file names created by this module no longer contain the process ID; instead a string of six random characters is used. Also, all the user-callable functions now take additional arguments which allow direct control over the location and name of temporary files. It is no longer necessary to use the global tempdir and template variables. To maintain backward compatibility, the argument order is somewhat odd; it is recommended to use keyword arguments for clarity. The module defines the following user-callable functions: tempfile.TemporaryFile([mode='w+b'[, bufsize=-1[, suffix=''[, prefix='tmp'[, dir=None]]]]])¶ Return a file-like object that can be used as a temporary storage area. The file is created using mkstemp(). It will be destroyed as soon as it is closed (including an implicit close when the object is garbage collected). Under Unix, the directory entry for the file is removed immediately after the file is created. Other platforms do not support this; your code should not rely on a temporary file created using this function having or not having a visible name in the file system. The mode parameter defaults to 'w+b' so that the file created can be read and written without being closed.
    [Show full text]
  • GNU Coreutils Cheat Sheet (V1.00) Created by Peteris Krumins ([email protected], -- Good Coders Code, Great Coders Reuse)
    GNU Coreutils Cheat Sheet (v1.00) Created by Peteris Krumins ([email protected], www.catonmat.net -- good coders code, great coders reuse) Utility Description Utility Description arch Print machine hardware name nproc Print the number of processors base64 Base64 encode/decode strings or files od Dump files in octal and other formats basename Strip directory and suffix from file names paste Merge lines of files cat Concatenate files and print on the standard output pathchk Check whether file names are valid or portable chcon Change SELinux context of file pinky Lightweight finger chgrp Change group ownership of files pr Convert text files for printing chmod Change permission modes of files printenv Print all or part of environment chown Change user and group ownership of files printf Format and print data chroot Run command or shell with special root directory ptx Permuted index for GNU, with keywords in their context cksum Print CRC checksum and byte counts pwd Print current directory comm Compare two sorted files line by line readlink Display value of a symbolic link cp Copy files realpath Print the resolved file name csplit Split a file into context-determined pieces rm Delete files cut Remove parts of lines of files rmdir Remove directories date Print or set the system date and time runcon Run command with specified security context dd Convert a file while copying it seq Print sequence of numbers to standard output df Summarize free disk space setuidgid Run a command with the UID and GID of a specified user dir Briefly list directory
    [Show full text]
  • Constraints in Dynamic Symbolic Execution: Bitvectors Or Integers?
    Constraints in Dynamic Symbolic Execution: Bitvectors or Integers? Timotej Kapus, Martin Nowack, and Cristian Cadar Imperial College London, UK ft.kapus,m.nowack,[email protected] Abstract. Dynamic symbolic execution is a technique that analyses programs by gathering mathematical constraints along execution paths. To achieve bit-level precision, one must use the theory of bitvectors. However, other theories might achieve higher performance, justifying in some cases the possible loss of precision. In this paper, we explore the impact of using the theory of integers on the precision and performance of dynamic symbolic execution of C programs. In particular, we compare an implementation of the symbolic executor KLEE using a partial solver based on the theory of integers, with a standard implementation of KLEE using a solver based on the theory of bitvectors, both employing the popular SMT solver Z3. To our surprise, our evaluation on a synthetic sort benchmark, the ECA set of Test-Comp 2019 benchmarks, and GNU Coreutils revealed that for most applications the integer solver did not lead to any loss of precision, but the overall performance difference was rarely significant. 1 Introduction Dynamic symbolic execution is a popular program analysis technique that aims to systematically explore all the paths in a program. It has been very successful in bug finding and test case generation [3, 4]. The research community and industry have produced many tools performing symbolic execution, such as CREST [5], FuzzBALL [9], KLEE [2], PEX [14], and SAGE [6], among others. To illustrate how dynamic symbolic execution works, consider the program shown in Figure 1a.
    [Show full text]
  • NSWI 0138: Advanced Unix Programming
    NSWI 0138: Advanced Unix programming (c) 2011-2016 Vladim´ırKotal (c) 2009-2010 Jan Pechanec, Vladim´ırKotal SISAL MFF UK, Malostransk´en´am.25, 118 00 Praha 1 Charles University Czech Republic Vladim´ırKotal [email protected] March 10, 2016 1 Vladim´ırKotal NSWI 0138 (Advanced Unix programming) Contents 1 Overview 5 1.1 What is this lecture about? . .5 1.2 The lecture will cover... .5 1.3 A few notes on source code files . .6 2 Testing 6 2.1 Why?...........................................6 2.2 When ? . .6 2.3 Types of testing . .7 3 Debugging 8 3.1 Debuging in general . .8 3.2 Observing . .9 3.3 Helper tools . .9 3.3.1 ctags . .9 3.3.2 cscope . 10 3.3.3 OpenGrok . 11 3.3.4 Other tools . 11 3.4 Debugging data . 11 3.4.1 stabs . 11 3.4.2 DWARF . 12 3.4.3 CTF (Compact C Type Format) . 12 3.5 Resource leaks . 13 3.6 libumem . 13 3.6.1 How does libumem work . 14 3.6.2 Using libumem+mdb to find memory leaks . 14 3.6.3 How does ::findleaks work . 16 3.7 watchmalloc . 17 3.8 Call tracing . 18 3.9 Using /proc . 19 3.10 Debugging dynamic libraries . 20 3.11 Debuggers . 20 3.12 Symbol search and interposition . 20 3.13 dtrace . 21 4 Terminals 21 4.1 Terminal I/O Overview . 21 4.2 Terminal I/O Overview (cont.) . 22 4.3 Physical (Hardware) Terminal . 24 4.4 stty(1) command . 24 4.5 TTY Driver Connected To a Phy Terminal .
    [Show full text]
  • Advanced Bash-Scripting Guide
    Advanced Bash−Scripting Guide An in−depth exploration of the art of shell scripting Mendel Cooper <[email protected]> 5.0 24 June 2007 Revision History Revision 4.2 10 Dec 2006 Revised by: mc 'SPARKLEBERRY' release: Important Update. Revision 4.3 29 Apr 2007 Revised by: mc 'INKBERRY' release: Minor Update. Revision 5.0 24 Jun 2007 Revised by: mc 'SERVICEBERRY' release: Major Update. This tutorial assumes no previous knowledge of scripting or programming, but progresses rapidly toward an intermediate/advanced level of instruction . all the while sneaking in little snippets of UNIX® wisdom and lore. It serves as a textbook, a manual for self−study, and a reference and source of knowledge on shell scripting techniques. The exercises and heavily−commented examples invite active reader participation, under the premise that the only way to really learn scripting is to write scripts. This book is suitable for classroom use as a general introduction to programming concepts. The latest update of this document, as an archived, bzip2−ed "tarball" including both the SGML source and rendered HTML, may be downloaded from the author's home site. A pdf version is also available ( pdf mirror site). See the change log for a revision history. Dedication For Anita, the source of all the magic Advanced Bash−Scripting Guide Table of Contents Chapter 1. Why Shell Programming?...............................................................................................................1 Chapter 2. Starting Off With a Sha−Bang.......................................................................................................3
    [Show full text]
  • Bash Guide for Beginners
    Bash Guide for Beginners Machtelt Garrels Xalasys.com <tille wants no spam _at_ xalasys dot com> Version 1.8 Last updated 20060315 Edition Bash Guide for Beginners Table of Contents Introduction.........................................................................................................................................................1 1. Why this guide?...................................................................................................................................1 2. Who should read this book?.................................................................................................................1 3. New versions, translations and availability.........................................................................................2 4. Revision History..................................................................................................................................2 5. Contributions.......................................................................................................................................3 6. Feedback..............................................................................................................................................3 7. Copyright information.........................................................................................................................3 8. What do you need?...............................................................................................................................4 9. Conventions used in this
    [Show full text]
  • GPL-3-Free Replacements of Coreutils 1 Contents
    GPL-3-free replacements of coreutils 1 Contents 2 Coreutils GPLv2 2 3 Alternatives 3 4 uutils-coreutils ............................... 3 5 BSDutils ................................... 4 6 Busybox ................................... 5 7 Nbase .................................... 5 8 FreeBSD ................................... 6 9 Sbase and Ubase .............................. 6 10 Heirloom .................................. 7 11 Replacement: uutils-coreutils 7 12 Testing 9 13 Initial test and results 9 14 Migration 10 15 Due to the nature of Apertis and its target markets there are licensing terms that 1 16 are problematic and that forces the project to look for alternatives packages. 17 The coreutils package is good example of this situation as its license changed 18 to GPLv3 and as result Apertis cannot provide it in the target repositories and 19 images. The current solution of shipping an old version which precedes the 20 license change is not tenable in the long term, as there are no upgrades with 21 bugfixes or new features for such important package. 22 This situation leads to the search for a drop-in replacement of coreutils, which 23 need to provide compatibility with the standard GNU coreutils packages. The 24 reason behind is that many other packages rely on the tools it provides, and 25 failing to do that would lead to hard to debug failures and many custom patches 26 spread all over the archive. In this regard the strict requirement is to support 27 the features needed to boot a target image with ideally no changes in other 28 components. The features currently available in our coreutils-gplv2 fork are a 29 good approximation. 30 Besides these specific requirements, the are general ones common to any Open 31 Source Project, such as maturity and reliability.
    [Show full text]