sign in sign up
<<
Home , Bash (Unix shell), Glob (programming)

The and You

Nathan Jarus

I Slides: http://web.mst.edu/~nmjxv3/articles/shell.pdf I Files: http://web.mst.edu/~nmjxv3/articles/shell/

October 3, 2017

1 / 22 What’s so cool about shells?

Introduction 2 / 22 What’s so cool about shells?

I You can find ‘em on the beach

I They look nice

I Cool ocean sounds

I Sometimes have crabs inside!

Introduction 2 / 22 What’s so cool about shells?

I Pretty fast for some tasks

I Works well over a slow connection

I Can construct complex tools out of simple ones

I Easy to automate tasks!

Introduction 2 / 22 Is PuTTY a shell? Nope, it’s a !

I Terminals connect your keyboard and screen to your shell I They run on your machine and connect to a shell (either on your machine or somewhere else)

I : , -terminal, &. I Mac: Terminal.app I Windows: PuTTY

I Can connect to remote shells via ssh (secure shell)

What is a shell, anyway?

I A shell takes commands, runs them, and shows you the output

I Linux has a bunch of shells: bash, zsh, dash, csh, ,...

I Shells usually provide some tools for connecting programs together, running multiple programs, &c.

Introduction 3 / 22 Nope, it’s a terminal!

I Terminals connect your keyboard and screen to your shell I They run on your machine and connect to a shell (either on your machine or somewhere else)

I Linux: xterm, gnome-terminal, &c. I Mac: Terminal.app I Windows: PuTTY

I Can connect to remote shells via ssh (secure shell)

What is a shell, anyway?

I A shell takes commands, runs them, and shows you the output

I Linux has a bunch of shells: bash, zsh, dash, csh, tcsh,...

I Shells usually provide some tools for connecting programs together, running multiple programs, &c. Is PuTTY a shell?

Introduction 3 / 22 What is a shell, anyway?

I A shell takes commands, runs them, and shows you the output

I Linux has a bunch of shells: bash, zsh, dash, csh, tcsh,...

I Shells usually provide some tools for connecting programs together, running multiple programs, &c. Is PuTTY a shell? Nope, it’s a terminal!

I Terminals connect your keyboard and screen to your shell I They run on your machine and connect to a shell (either on your machine or somewhere else)

I Linux: xterm, gnome-terminal, &c. I Mac: Terminal.app I Windows: PuTTY

I Can connect to remote shells via ssh (secure shell)

Introduction 3 / 22 I : Change Directories

I cd with no arguments takes you to your home directory I cd - takes you to the last directory you were in I ls: List (files and directories)

I -l Display a detailed list of information about each file I -h Display human-readable file sizes I -a Display all files, even hidden ones (files that with a .) Some neat tricks:

I ls *.txt

I ls **/*.cpp

Navigating your filesystem

I : Print

Files ‘n whatnot 4 / 22 I ls: List (files and directories)

I -l Display a detailed list of information about each file I -h Display human-readable file sizes I -a Display all files, even hidden ones (files that start with a .) Some neat tricks:

I ls *.txt

I ls **/*.cpp

Navigating your filesystem

I pwd: Print Working Directory I cd: Change Directories

I cd with no arguments takes you to your home directory I cd - takes you to the last directory you were in

Files ‘n whatnot 4 / 22 Some neat tricks:

I ls *.txt

I ls **/*.cpp

Navigating your filesystem

I pwd: Print Working Directory I cd: Change Directories

I cd with no arguments takes you to your home directory I cd - takes you to the last directory you were in I ls: List (files and directories)

I -l Display a detailed list of information about each file I -h Display human-readable file sizes I -a Display all files, even hidden ones (files that start with a .)

Files ‘n whatnot 4 / 22 Navigating your filesystem

I pwd: Print Working Directory I cd: Change Directories

I cd with no arguments takes you to your home directory I cd - takes you to the last directory you were in I ls: List (files and directories)

I -l Display a detailed list of information about each file I -h Display human-readable file sizes I -a Display all files, even hidden ones (files that start with a .) Some neat tricks:

I ls *.txt

I ls **/*.cpp

Files ‘n whatnot 4 / 22 I mv: Move (or rename) files

I -i: Interactively ask before overwriting files I cp

I -: Recursively copy directories I -i: Interactively ask before overwriting files I rm

I -f: Forcibly remove files (even if write-protected) I -r: Recursively remove directories I -i: Interactively ask before overwriting files Another neat trick: mv bob.{coo,cpp}

Scooting files around

WARNING: These programs will happily destroy all your files if you ask them to

Files ‘n whatnot 5 / 22 I cp

I -r: Recursively copy directories I -i: Interactively ask before overwriting files I rm

I -f: Forcibly remove files (even if write-protected) I -r: Recursively remove directories I -i: Interactively ask before overwriting files Another neat trick: mv bob.{coo,cpp}

Scooting files around

WARNING: These programs will happily destroy all your files if you ask them to I mv: Move (or rename) files

I -i: Interactively ask before overwriting files

Files ‘n whatnot 5 / 22 I rm

I -f: Forcibly remove files (even if write-protected) I -r: Recursively remove directories I -i: Interactively ask before overwriting files Another neat trick: mv bob.{coo,cpp}

Scooting files around

WARNING: These programs will happily destroy all your files if you ask them to I mv: Move (or rename) files

I -i: Interactively ask before overwriting files I cp

I -r: Recursively copy directories I -i: Interactively ask before overwriting files

Files ‘n whatnot 5 / 22 Another neat trick: mv bob.{coo,cpp}

Scooting files around

WARNING: These programs will happily destroy all your files if you ask them to I mv: Move (or rename) files

I -i: Interactively ask before overwriting files I cp

I -r: Recursively copy directories I -i: Interactively ask before overwriting files I rm

I -f: Forcibly remove files (even if write-protected) I -r: Recursively remove directories I -i: Interactively ask before overwriting files

Files ‘n whatnot 5 / 22 Scooting files around

WARNING: These programs will happily destroy all your files if you ask them to I mv: Move (or rename) files

I -i: Interactively ask before overwriting files I cp

I -r: Recursively copy directories I -i: Interactively ask before overwriting files I rm

I -f: Forcibly remove files (even if write-protected) I -r: Recursively remove directories I -i: Interactively ask before overwriting files Another neat trick: mv bob.{coo,cpp}

Files ‘n whatnot 5 / 22 !

I help: Help with built-in Bash commands and features I man: Help with EVERYTHING

I Scroll with arrow keys, j / k , or PgUp and PgDn I q quits I To search for something: /search-term Enter I n goes to next match; N goes to previous match I h shows more navigation hints

Files ‘n whatnot 6 / 22 I head and tail: Display the first or last ten lines of a file

Looking stuff

I cat: Put the contents of one or more files on the screen

I less: Display the contents of a file one page at a

Look around you 7 / 22 Looking at stuff

I cat: Put the contents of one or more files on the screen

I less: Display the contents of a file one page at a time

I head and tail: Display the first or last ten lines of a file

Look around you 7 / 22 I grep: Search for stuff inside files

I -i: Perform case-insensitive match I -v: Invert the match (print lines that don’t match) I -C 5: Show 5 lines of context around matches

Looking for stuff

I : Find files in a directory (and do stuff to them)

I The first argument is a directory to search in I After that you can specify things to search for:

I -name: Search by name or I -: Search for files or directories

I After that you can specify an action:

I -ls: Show file output like ls -l does I -delete: Delete files! I -exec: Execute a command

Just look around you 8 / 22 Looking for stuff

I find: Find files in a directory (and do stuff to them)

I The first argument is a directory to search in I After that you can specify things to search for:

I -name: Search by name or glob I -type: Search for files or directories

I After that you can specify an action:

I -ls: Show file output like ls -l does I -delete: Delete files! I -exec: Execute a command I grep: Search for stuff inside files

I -i: Perform case-insensitive match I -v: Invert the match (print lines that don’t match) I -C 5: Show 5 lines of context around matches

Just look around you 8 / 22 Standard input and output

Every program has one “input stream” (called STDIN) and two “output streams” (called STDOUT and STDERR). In C++, STDIN is connected to cin, STDOUT to cout, and STDERR to cerr.

STDOUT STDIN my program STDERR

Typically STDIN reads from your keyboard and STDOUT (and STDERR) write to the screen.

Plumbing 9 / 22 "Hello there" > hello.txt

STDOUT echo "Hello there" > hello.txt

Writing output to files

You can use > and >> to redirect a command’s output to a file instead!

Plumbing 10 / 22 Writing output to files

You can use > and >> to redirect a command’s output to a file instead!

echo "Hello there" > hello.txt

STDOUT echo "Hello there" > hello.txt

Plumbing 10 / 22 g++ errors.cpp > errors.txt 2>&1

1 > errors.txt g++ errors.cpp 2 2>&1

Capturing STDERR

Typically, you want to capture BOTH standard output and standard error. To do this, redirect standard error into standard output with the incantation 2>&1.

Plumbing 11 / 22 Capturing STDERR

Typically, you want to capture BOTH standard output and standard error. To do this, redirect standard error into standard output with the incantation 2>&1.

g++ errors.cpp > errors.txt 2>&1

1 > errors.txt g++ errors.cpp 2 2>&1

Plumbing 11 / 22 echo "I love to program" | less

STDOUT echo "I love to program" less

Writing output to other programs

You can also redirect one program’s STDOUT to another program’s STDIN! This is done with the pipe (|) character.

Plumbing 12 / 22 Writing output to other programs

You can also redirect one program’s STDOUT to another program’s STDIN! This is done with the pipe (|) character.

echo "I love to program" | less

STDOUT echo "I love to program" less

Plumbing 12 / 22 Shell anatomy

#!/bin/bash

g++ *.cpp -o program ./program

A is comprised of two things: 1. A “” line: starts with #!; contains the command that executes the script 2. A bunch of bash commands

Scripting 13 / 22 Command-line arguments live in numbered variables:

I $0: The name of the program

I $1: The first argument

I $2: The second argument (and so on and so forth) #! /bin/bash

g++ *.cpp -o $1 ./$1

Variables and Arguments

I To assign a value to a variable: num cows=5 (note: NO SPACES)

I To use a variable: $num cows

Scripting 14 / 22 #! /bin/bash

g++ *.cpp -o $1 ./$1

Variables and Arguments

I To assign a value to a variable: num cows=5 (note: NO SPACES)

I To use a variable: $num cows Command-line arguments live in numbered variables:

I $0: The name of the program

I $1: The first argument

I $2: The second argument (and so on and so forth)

Scripting 14 / 22 Variables and Arguments

I To assign a value to a variable: num cows=5 (note: NO SPACES)

I To use a variable: $num cows Command-line arguments live in numbered variables:

I $0: The name of the program

I $1: The first argument

I $2: The second argument (and so on and so forth) #! /bin/bash

g++ *.cpp -o $1 ./$1

Scripting 14 / 22 More special variables:

I $?: The return value of the last program run

I $#: The number of command-line arguments passed

Checking for failures

Our script has two problems right now: 1. If our code doesn’t compile, we probably don’t want it to run! 2. If the user doesn’t offer a program name, the error message is not very nice

Scripting 15 / 22 Checking for failures

Our script has two problems right now: 1. If our code doesn’t compile, we probably don’t want it to run! 2. If the user doesn’t offer a program name, the error message is not very nice More special variables:

I $?: The return value of the last program run

I $#: The number of command-line arguments passed

Scripting 15 / 22 Checking for failures

#! /bin/bash progname="program"

if [[ $# -ge 1 ]]; then progname=$1 fi

g++ *.cpp -o $progname

if [[ $? -eq 0 ]]; then ./$progname fi

Scripting 16 / 22 With a for loop!

#! /bin/bash

for file in *.sh; do echo "Copying $file to $file.bak" cp $file $file.bak done

Doing stuff to a bunch of files

Let’s pretend we don’t know about find for a second. How would we make a backup copy of every one of our shell scripts?

Scripting 17 / 22 Doing stuff to a bunch of files

Let’s pretend we don’t know about find for a second. How would we make a backup copy of every one of our shell scripts?

With a for loop!

#! /bin/bash

for file in *.sh; do echo "Copying $file to $file.bak" cp $file $file.bak done

Scripting 17 / 22 Looping over command-line arguments

Let’s print out each command line argument on its own line: #! /bin/bash

for arg in $@; do echo $arg done

Scripting 18 / 22 When you ask it to run a command, bash looks through all the directories listed in a special variable named PATH. We can add our own directory to this!

PATH=~/bin:$PATH

To get this modification to stick, we need to run this command every time we start our shell. Fortunately, bash runs the /.bashrc script every time you start a new bash shell!

For more variables that control how bash works, see help variables.

Running commands from anywhere

These shell scripts are great, but we always have to give bash the exact path to the script so it knows what to run. How can we fix this?

Customization 19 / 22 To get this modification to stick, we need to run this command every time we start our shell. Fortunately, bash runs the /.bashrc script every time you start a new bash shell!

For more variables that control how bash works, see help variables.

Running commands from anywhere

These shell scripts are great, but we always have to give bash the exact path to the script so it knows what to run. How can we fix this?

When you ask it to run a command, bash looks through all the directories listed in a special variable named PATH. We can add our own directory to this!

PATH=~/bin:$PATH

Customization 19 / 22 For more variables that control how bash works, see help variables.

Running commands from anywhere

These shell scripts are great, but we always have to give bash the exact path to the script so it knows what to run. How can we fix this?

When you ask it to run a command, bash looks through all the directories listed in a special variable named PATH. We can add our own directory to this!

PATH=~/bin:$PATH

To get this modification to stick, we need to run this command every time we start our shell. Fortunately, bash runs the /.bashrc script every time you start a new bash shell!

Customization 19 / 22 Running commands from anywhere

These shell scripts are great, but we always have to give bash the exact path to the script so it knows what to run. How can we fix this?

When you ask it to run a command, bash looks through all the directories listed in a special variable named PATH. We can add our own directory to this!

PATH=~/bin:$PATH

To get this modification to stick, we need to run this command every time we start our shell. Fortunately, bash runs the /.bashrc script every time you start a new bash shell!

For more variables that control how bash works, see help variables.

Customization 19 / 22 rungcc() { progname="program"

if [[ $# -ge 1 ]]; then progname=$1 fi

g++ *.cpp -o $progname

if [[ $? -eq 0 ]]; then ./$progname fi }

Making new commands the quick ‘n easy way

I name=command: Make a shorthand name for an existing command I Bash functions: function-name() { commands }

Customization 20 / 22 Making new commands the quick ‘n easy way

I alias name=command: Make a shorthand name for an existing command I Bash functions: function-name() { commands } rungcc() { progname="program"

if [[ $# -ge 1 ]]; then progname=$1 fi

g++ *.cpp -o $progname

if [[ $? -eq 0 ]]; then ./$progname fi }

Customization 20 / 22 Where to from here?

Take CS 1585! http://web.mst.edu/~nmjxv3/cs1001/

I List of Bash Commands: https://ss64.com/bash/

I Bash Reference Manual: https://www.gnu.org/software/bash/manual/

I All About Pipes: http://www.linfo.org/pipe.html

I Software Carpentry Shell Tutorial: http://swcarpentry.github.io/shell-novice/

I Bash Tutorial: http://tldp.org/LDP/Bash-Beginners-Guide/html/

Conclusion 21 / 22 Pictured: Annie (left) and Lion (right)

Conclusion 22 / 22