At—At, Batch—Execute Commands at a Later Time

Total Page:16

File Type:pdf, Size:1020Kb

Load more

at—at, batch—execute commands at a later time

at [–csm] [–f script] [–qqueue] time [date] [+ increment] at –l [ job...] at –r job... batch

at and batch read commands from standard input to be executed at a later time. at allows you to specify when the commands should be executed, while jobs queued with batch will execute when system load level permits. Executes commands read from stdin or a file at some later time. Unless redirected, the output is mailed to the user.

Example A.1

1 at 6:30am Dec 12 < program 2 at noon tomorrow < program 3 at 1945 pm August 9 < program 4 at now + 3 hours < program 5 at 8:30am Jan 4 < program 6 at -r 83883555320.a

EXPLANATION

1. At 6:30 in the morning on December 12th, start the job. 2. At noon tomorrow start the job. 3. At 7:45 in the evening on August 9th, start the job. 4. In three hours start the job. 5. At 8:30 in the morning of January 4th, start the job.

6. Removes previously scheduled job 83883555320.a.

awk—pattern scanning and processing language

awk [ –fprogram–file ] [ –Fc ] [ prog ] [ parameters ] [ filename...]

awk scans each input filename for lines that match any of a set of patterns specified in prog.

Example A.2

1 awk '{print $1, $2}' file 2 awk '/John/{print $3, $4}' file 3 awk -F: '{print $3}' /etc/passwd 4 date | awk '{print $6}'

EXPLANATION

1. Prints the first two fields of file where fields are separated by whitespace. 2. Prints fields 3 and 4 if the pattern John is found. 3. Using a colon as the field separator, prints the third field of the /etc/passwd file.

4. Sends the output of the date command to awk and prints the sixth field.

banner—make posters

banner prints its arguments (each up to 10 characters long) in large letters on the standard output.

Example A.3

banner Happy Birthday

EXPLANATION

Displays in banner format the string Happy Birthday.

basename—with a directory name delivers portions of the pathname

basename string [ suffix ] dirname string

basename deletes any prefix ending in / (forward slash) and the suffix (if present in string) from string, and prints the result on the standard output.

Example A.4

1 basename /usr/local/bin 2 scriptname="'basename $0'"

EXPLANATION

1. Strips off the prefix /usr/local/ and displays bin.

2. Assigns just the name of the script, $0, to the variable scriptname

bc—processes precision arithmetic

bc [ –c ] [ –l ] [ filename...] bc is an interactive processor for a language that resembles C but provides unlimited precision arithmetic. It takes input from any files given, then reads the standard input.

Example A.5

1 bc << EOF scale=3 4.5 + 5.6 / 3 EOF Output : 6.366 -----------------------------
2 bc ibase=2 5101 (Output) 20 10100 (Output ^D

EXPLANATION

1. This is a here document. From the first EOF to the last EOF input is given to the bc command. The scale specifies the number of digits to the right of the decimal point. The result of the calculation is displayed on the screen.
2. The number base is two. The number is converted to binary (AT&T only).

bdiff—compares two big files

bdiff compares two files that are too large for diff.

cal—displays a calendar

cal [ [ month ] year ] cal prints a calendar for the specified year. If a month is also specified, a calendar just for that month is printed. If neither is specified, a calendar for the present month is printed.

Example A.6

1 cal 1997 2 cal 5 1978

EXPLANATION

1. Prints the calendar year 1997. 2. Prints the month of May for 1978.

cat—concatenates and displays files

cat [ –bnsuvet ] filename... cat reads each filename in sequence and writes it on the standard output. If no input file is given, or if the argument – is encountered, cat reads from the standard input file.

Example A.7

1 cat /etc/passwd 2 cat -n file1 file2 >> file3

EXPLANATION

1. Displays the contents of the /etc/passwd file. 2. Concatenates file1 and file2 and appends output to file3. The –n switch causes each line to be numbered.

chmod—change the permissions mode of a file

chmod [ –fR ] mode filename... chmod [ugoa ]{ + | – | = }[ rwxlsStTugo] filename...

chmod changes or assigns the mode of a file. The mode of a file specifies its permissions and other attributes. The mode may be absolute or symbolic.

Example A.8

1 chmod +x script.file 2 chmod u+x,g-x file 3 chmod 755 *

EXPLANATION

1. Turns on execute permission for user, group, and others on script.file. 2. Turns on execute permission for user, and removes it from group on file. 3. Turns on read, write, and execute for the user, read and execute for the group, and read and execute for others on all files in the current working directory. The value is octal ( 111 101 101 ).

4. rwxxr-xr-x

chown—changes owner of file

chown [ –fhR ] owner filename ... chown changes the owner of the files to owner. The owner may be either a decimal user ID or a login name found in /etc/passwd file. Only the owner of a file (or the superuser) may change the owner of that file.

Example A.9

1 chown john filex 2 chown -R ellie ellie

EXPLANATION

1. Changes the user ID of filex to john. 2. Recursively changes the ownership to ellie for all files in ellie directory.

  • clear—clears the terminal screen
  • cmp—compares two files

cmp [ –l ] [ –s ] filename1 filename2 The two files are compared. cmp makes no comment if the files are the same; if they differ, it announces the byte and line numbers at which the first difference occurred.

Example A.10

cmp file.new file.old

EXPLANATION

If the files differ, the character number and the line number are displayed.

compress—compress, uncompress, zcat compress, uncompress files, or display expanded files

compress [ –cfv ] [ –b bits ] [ filename... ] uncompress [ –cv ] [ filename... ] zcat [ filename... ]

compress reduces the size of the named files using adaptive Lempel-Ziv coding. Whenever possible, each file is replaced by one with a .Z extension. The ownership modes, access time, and modification time will stay the same. If no files are specified, the standard input is compressed to the standard output.

Example A.11

1 compress -v book

book:Compression:35.07% -- replaced with book.Z

2 ls book.Z

EXPLANATION

1. Compresses the book into a file called book.Z and displays the percentage that the file was compressed and its new name.

cp—copies files

cp [ –i ] [ –p ] [ –r ] [ filename ... ] target The cp command copies filename to another target, which is either a file or directory. The filename and target cannot have the same name. If the target is not a directory, only one file may be specified before it; if it is a directory, more than one file may be specified. If target does not exist, cp creates a file named target. If target exists and is not a directory, its contents are overwritten. If target is a directory, the file(s) are copied to that directory.

Example A.12

1 cp file1 file2 2 cp chapter1 book 3 cp -r desktop /usr/bin/tester

EXPLANATION

1. Copies the contents of file1 to file2. 2. Copies the contents of chapter1 to the book directory. In the book directory, chapter1 has its original name. 3. Recursively copies the entire desktop directory into /usr/bin/tester.

cpio—copy file archives in and out

cpio –i [ bBcdfkmrsStuvV6 ] [ –C bufsize ] [ –E filename ]
[ –H header ] [ –I filename [ –M message ] ] [ –R id ] [ pattern ... ] cpio –o [ aABcLvV ] [ –C bufsize ] [ –H header ]
[ –O filename [ –M message ] ] cpio –p [ adlLmuvV ] [ –R id ] directory

Copies file archives according to the modifiers given, usually for backup to a tape or directory.

Example A.13

find . -depth -print | cpio -pdmv /home/john/tmp

EXPLANATION

Starting at the current directory, find descends the directory hierarchy, printing each entries of the directory even if the directory does not have write permission, and sends the filenames to cpio to be copied into the john/tmp directory in the /home partition.

cron—the clock daemon

cron executes commands at specified dates and times. Regularly scheduled jobs can be specified in the /etc/crontab file. In order to use cron, one of the following must be true: (1) you are superuser; (2) you are regular user, but your user ID is listed in the /etc/cron.allow file; (3) you are regular user, but your system contains a file /etc/cron.deny, which is empty.

crypt—encodes or decodes a file

crypt [ password ] crypt encrypts and decrypts the contents of a file. The password is a key that selects a type of transformation.

cut—removes selected fields or characters from each line of a file

cut –clist [ filename ... ] cut –flist [ –dc ] [ –s ] [ filename ... ]

The cut command cuts out columns or characters from a line of a file; if no files are given, uses standard input. The –d option specifies the field delimiter. The default delimiter is a tab.

Example A.14

1 cut -d: -f1,3 /etc/passwd 2 cut -d: -f1-5 /etc/passwd 3 cut -c1-3,8-12 /etc/passwd 4 date | cut -c1-3

EXPLANATION

1. Using the colon as a field delimiter, displays fields 1 and 3 of the /etc/passwd file. 2. Using the colon as a field separator, displays fields 1 through 5 of the etc/passwd file. 3. Cuts and displays characters 1 through 3 and 8 through 12 of each line from the /etc/passwd file. 4. Sends the output of the date command as input to cut. The first three characters are printed.

date—displays the date and time or sets the date

[ –u ] [ –a [ – ] sss.fff ] [ yymmddhhmm [ .ss ] ] [+format ]

Without arguments, the date command displays the date and time. If the command line argument starts with a plus sign, the rest of the argument is used to format the output. If a percent sign is used, the next character is a formatting character to extract a particular part of the date, such as just the year or weekday. To set the date. the command line argument is expressed in digits representing the year, month, day, hours, and minutes.

Example A.15

1 date +%T 2 date +20%y 3 date "+It is now %m/%d /%y"

EXPLANATION

1. Displays the time as 20:25:51 2. Displays 2096. 3. Displays It is now 07/25/96.

diff—compares two files for differences diff [–bitw] [–c | –Cn

Compares two files and displays the differences on a line-by-line basis. Also displays commands that you would use with the ed editor to make changes.

Example A.16

diff file1 file2 1c1 < hello there --- > Hello there. 2a3 > I'm fine.

EXPLANATION

Shows how each line of file1 and file2 differs. The first file is represented by the < symbol, and the second file by the > symbol. Each line is preceded by an ed command indicating the editing command that would be used to make the files the same.

du—summarizes disk usage

du [–arskod] [name ...] The du command reports the number of 512-byte blocks contained in all files and (recursively) directories within each directory and file specified.

Example A.17

1 du -s /desktop 2 du -a

EXPLANATION

1. Displays a summary of the block usage for all the files in /desktop and its subdirectories. 2. Displays block usage for each file in this directory and subdirectories.

echo—echoes arguments

echo [ argument ] ... echo [ –n ] [ argument ]

echo writes its arguments separated by blanks and terminated by a newline on the standard output. System V echo options: \b \c backspace suppress newline
\f form feed \n \r \t newline return tab
\v \\ vertical tab backslash
\0n n is a 1, 2, or 3, octal value

egrep—searches a file for a pattern using full regular expressions

egrep [ –bchilnsv ] [ –e special–expression ][ –f filename ] [ strings ] [ filename ... ]

egrep (expression grep) searches files for a pattern of characters and prints all lines that contain that pattern. egrep uses full regular expressions (expressions with string values that use the full set of alphanumeric and special characters) to match the patterns.

Example A.18

1 egrep 'Tom|John' datafile 2 egrep '^ [A-Z]+' file

EXPLANATION

1. Display all lines in datafile containing the pattern either Tom or John.

2. Display al lines starting with one or more uppercase letters.

expr—evaluates arguments as an expression

expr arguments The arguments are taken as an expression. After evaluation, the result is written to the standard output. The terms of the expression must be separated by blanks. Characters special to the shell must be escaped. Used in Bourne shell scripts for performing simple arithmetic operations.

Example A.19

1 expr 5 + 4 2 expr 5 \* 3 3 num=0 num='expr $num + 1'

EXPLANATION

1. Prints the sum of 5 + 4

2. Prints of result of 5 * 3. The asterisk is protected from shell expansion.

3. After assigning 0 to variable num, the expr command adds 1 to num and result is assigned to num.

fgrep—search a file for a character string

fgrep [ –bchilnsvx ] [ –e special string ] [ –f filename ] [ strings ] [ filename ... ]

fgrep (fast grep) searches files for a character string and prints all lines that contain that string. fgrep is different from grep and egrep because it interprets regular expression metacharacters as literals.

Example A.20

1 fgrep '***' * 2 fgrep '[ ] * ? $' filex

EXPLANATION

1. Displays any line containing three asterisks from each file in the present directory. All characters are treated as themselves; i.e., metacharacters are not special.
2. Displays any lines in filex containing the string enclosed in quotes.

file—determines the type of a file by looking at its contents

file [[ –f ffile ] [ –cl ] [ –m mfile ] filename... file performs a series of tests on each filename in an attempt to determine what it contains. If the contents of the file appear to be ASCII text, file examines the first 512 bytes and tries to guess its language.

Example A.21

1 file bin/ls

/bin/ls:sparc pure dynamically linked executable

2 file go

go: executable shell script

3 file junk

junk: English text

EXPLANATION

1. ls is a binary file dynamically linked when executed.

2. go is an executable shell script. 3. junk is a file containing ASCII text.

find—finds files

find path–name–list expression find recursively descends the directory hierarchy for each pathname in the pathname list (i.e., one or more pathnames) seeking files that match options. First argument is the path where the search starts. The rest of the arguments specify some criteria by which to find the files, such as name, size, owner, permissions, etc. Check the UNIX manual pages for different syntax.

Example A.22

1 find . -name \*.c -print 2 find .. -type f -print 3 find . -type d -print 4 find / -size 0 - exec rm "{}" \; 5 find ~ -perm 644 -print 6 find . -type f -size +500c -atime +21 -ok rm -f "{}" \; 7 find . -name core -print 2> /dev/null (Bourne and Korn Shells)
( find . -name core -print > /dev/tty ) >& /dev/null ( C shell)
8 find / -user ellie xdev -print 9 find ~ -atime +31 -exec mv {} /old/{} \; -print

EXPLANATION

1. Starting at the present working directory (dot), finds all files ending in .c and prints the full pathname of the file. 2. Starting at the parent directory (dot dot), finds all files of type file; i.e., files that are not directories. 3. Starting at the present directory (dot), finds all directory files. 4. Starting at the root directory, finds all files of size zero and removes them. The {} are used as a place holder for the name of each file as it is found.
5. Starting at the user's home directory ~ (Korn and C shells), finds all files that have permissions 644 (read and write for the owner, and read permission for the group and others).
6. Starting at the present working directory, finds files that are over 500 bytes and have not been accessed in the last 21 days and asks if it is okay to remove them.
7. Starting at the present working directory, finds and displays all files named core and sends errors to /dev/null, the UNIX bit bucket.

8. Prints all files on the root partition that belong to user ellie.

9. Moves files that are older than 31 days into a directory, /old, and prints the files as it moves them.

finger—displays information about local and remote users

finger [ –bfhilmpqsw ] [ username... ] finger [–l] username@hostname...

By default, the finger command displays information about each logged-in user, including login name, full name, terminal name (prepended with a * if write permission is denied), idle time, login time, and location if known.

Recommended publications
  • TDS) EDEN PG MV® Pigment Ink for Textile

    TDS) EDEN PG MV® Pigment Ink for Textile

    Technical Data Sheet (TDS) EDEN PG MV® Pigment Ink for Textile Product Description: Applications: EDEN PG MV water based pigment Ink is ideal for high-speed direct digital printing on all standard fabrics available on the market. The ink prints on all types of fabrics with equal quality and coverage without color shifts or patches. EDEN PG MV Pigment Ink may be used on cotton, cotton blends, polyester, polyamide (nylon) and treated & untreated fabrics. EDEN PG MV is ideal for a wide range of applications including sportswear, footwear, fashion, home décor and home textile and is designed for indoor and outdoor applications. This ink is manufactured using cutting edge technology and • Home textile • Bags • Indoor furnishing high-performance pigments, for optimal ink fluidity and • Décor • Sport apparel • Fashion & apparel printability through piezo printheads without bleeding or • Footwear migration. Compatible Print Head Technology: Ricoh Gen 4 & Gen 5 Industrial print heads based printers. Pre-treatment: The use of Bordeaux EDEN PG™ OS® (pretreatment liquid) is Product Line: recommended in order to achieve higher chemical attributes (see table below for comparison). The pretreatment process for fabrics is done offline using a standard textile padder. Red Cleaning Print procedure: Cyan Magenta Yellow Black Magenta Liquid The printing process can be done either inline or in individual segments. Inline printing allows the printed fabric to be fed Material Compatibility: through a standard textile dryer directly. Alternatively, if the printing stage is separated from the drying stage, drying is • Cotton required before rolling the fabric. • Viscose • Polyester • Lycra Drying Working Parameters: • Silk Recommended drying time: 3 minutes • Polyamide Recommended temperature: 150°C (302°F) • Leather • Synthetic leather Post-treatment: No chemical post treatment is needed.
  • Unix Introduction

    Unix Introduction

    Unix introduction Mikhail Dozmorov Summer 2018 Mikhail Dozmorov Unix introduction Summer 2018 1 / 37 What is Unix Unix is a family of operating systems and environments that exploits the power of linguistic abstractions to perform tasks Unix is not an acronym; it is a pun on “Multics”. Multics was a large multi-user operating system that was being developed at Bell Labs shortly before Unix was created in the early ’70s. Brian Kernighan is credited with the name. All computational genomics is done in Unix http://www.read.seas.harvard.edu/~kohler/class/aosref/ritchie84evolution.pdfMikhail Dozmorov Unix introduction Summer 2018 2 / 37 History of Unix Initial file system, command interpreter (shell), and process management started by Ken Thompson File system and further development from Dennis Ritchie, as well as Doug McIlroy and Joe Ossanna Vast array of simple, dependable tools that each do one simple task Ken Thompson (sitting) and Dennis Ritchie working together at a PDP-11 Mikhail Dozmorov Unix introduction Summer 2018 3 / 37 Philosophy of Unix Vast array of simple, dependable tools Each do one simple task, and do it really well By combining these tools, one can conduct rather sophisticated analyses The Linux help philosophy: “RTFM” (Read the Fine Manual) Mikhail Dozmorov Unix introduction Summer 2018 4 / 37 Know your Unix Unix users spend a lot of time at the command line In Unix, a word is worth a thousand mouse clicks Mikhail Dozmorov Unix introduction Summer 2018 5 / 37 Unix systems Three common types of laptop/desktop operating systems: Windows, Mac, Linux. Mac and Linux are both Unix-like! What that means for us: Unix-like operating systems are equipped with “shells”" that provide a command line user interface.
  • Course Outline & Schedule

    Course Outline & Schedule

    Course Outline & Schedule Call US 408-759-5074 or UK +44 20 7620 0033 Linux Advanced Shell Programming Tools Curriculum The Linux Foundation Course Code LASP Duration 3 Day Course Price $1,830 Course Description The Linux Advanced Shell Programming Tools course is designed to give delegates practical experience using a range of Linux tools to manipulate text and incorporate them into Linux shell scripts. The delegate will practise: Using the shell command line editor Backing up and restoring files Scheduling background jobs using cron and at Using regular expressions Editing text files with sed Using file comparison utilities Using the head and tail utilities Using the cut and paste utilities Using split and csplit Identifying and translating characters Sorting files Translating characters in a file Selecting text from files with the grep family of commands Creating programs with awk Course Modules Review of Shell Fundamentals (1 topic) ◾ Review of UNIX Commands Using Unix Shells (6 topics) ◾ Command line history and editing ◾ The Korn and POSIX shells Perpetual Solutions - Page 1 of 5 Course Outline & Schedule Call US 408-759-5074 or UK +44 20 7620 0033 ◾ The Bash shell ◾ Command aliasing ◾ The shell startup file ◾ Shell environment variables Redirection, Pipes and Filters (7 topics) ◾ Standard I/O and redirection ◾ Pipes ◾ Command separation ◾ Conditional execution ◾ Grouping Commands ◾ UNIX filters ◾ The tee command Backup and Restore Utilities (6 topics) ◾ Archive devices ◾ The cpio command ◾ The tar command ◾ The dd command ◾ Exercise:
  • HEP Computing Part I Intro to UNIX/LINUX Adrian Bevan

    HEP Computing Part I Intro to UNIX/LINUX Adrian Bevan

    HEP Computing Part I Intro to UNIX/LINUX Adrian Bevan Lectures 1,2,3 [email protected] 1 Lecture 1 • Files and directories. • Introduce a number of simple UNIX commands for manipulation of files and directories. • communicating with remote machines [email protected] 2 What is LINUX • LINUX is the operating system (OS) kernel. • Sitting on top of the LINUX OS are a lot of utilities that help you do stuff. • You get a ‘LINUX distribution’ installed on your desktop/laptop. This is a sloppy way of saying you get the OS bundled with lots of useful utilities/applications. • Use LINUX to mean anything from the OS to the distribution we are using. • UNIX is an operating system that is very similar to LINUX (same command names, sometimes slightly different functionalities of commands etc). – There are usually enough subtle differences between LINUX and UNIX versions to keep you on your toes (e.g. Solaris and LINUX) when running applications on multiple platforms …be mindful of this if you use other UNIX flavours. – Mac OS X is based on a UNIX distribution. [email protected] 3 Accessing a machine • You need a user account you should all have one by now • can then log in at the terminal (i.e. sit in front of a machine and type in your user name and password to log in to it). • you can also log in remotely to a machine somewhere else RAL SLAC CERN London FNAL in2p3 [email protected] 4 The command line • A user interfaces with Linux by typing commands into a shell.
  • The Price of Safety: Evaluating IOMMU Performance Preliminary Results

    The Price of Safety: Evaluating IOMMU Performance Preliminary Results

    The Price of Safety: Evaluating IOMMU Performance Preliminary Results Muli Ben-Yehuda [email protected] IBM Haifa Research Lab The Price of Safety: Evaluating IOMMU Performance, 2007 Spring Xen Summit – p.1/14 Table of Contents IOMMUs Preliminary Performance Results Optimizations and Trade-offs Conclusions The Price of Safety: Evaluating IOMMU Performance, 2007 Spring Xen Summit – p.2/14 IOMMUs IOMMU - IO Memory Management Unit Think MMU for IO devices - separate address spaces! IOMMUs enable virtual machines direct hardware access for PV and FV guests ... while protecting the rest of the system from mis-behaving or malicious virtual machines But at what cost? The Price of Safety: Evaluating IOMMU Performance, 2007 Spring Xen Summit – p.3/14 Setting the Stage Full results to appear at OLS ’07 Calgary (PCI-X) and Calgary (PCI-e) IOMMUs Linux and Xen implementations ... with some corroborating evidence from the DART IOMMU on PPC JS21 blades and the IBM Research Hypervisor Joint work with Jimi Xenidis, Michal Ostrowski, Karl Rister, Alexis Bruemmer and Leendert Van Doorn Utilizing IOMMUs for Virtualization in Linux and Xen, by by M. Ben-Yehuda, J. Mason, O. Krieger, J. Xenidis, L. Van Doorn, A. Mallick, J. Nakajima, and E. Wahlig, OLS ’06 The Price of Safety: Evaluating IOMMU Performance, 2007 Spring Xen Summit – p.4/14 On Comparisons What are we comparing against? not emulation not virtual IO (frontend / backend drivers) Direct hardware access without an IOMMU compared with direct hardware access with an IOMMU on the IO path The Price of Safety:
  • Unix/Linux Command Reference

    Unix/Linux Command Reference

    Unix/Linux Command Reference .com File Commands System Info ls – directory listing date – show the current date and time ls -al – formatted listing with hidden files cal – show this month's calendar cd dir - change directory to dir uptime – show current uptime cd – change to home w – display who is online pwd – show current directory whoami – who you are logged in as mkdir dir – create a directory dir finger user – display information about user rm file – delete file uname -a – show kernel information rm -r dir – delete directory dir cat /proc/cpuinfo – cpu information rm -f file – force remove file cat /proc/meminfo – memory information rm -rf dir – force remove directory dir * man command – show the manual for command cp file1 file2 – copy file1 to file2 df – show disk usage cp -r dir1 dir2 – copy dir1 to dir2; create dir2 if it du – show directory space usage doesn't exist free – show memory and swap usage mv file1 file2 – rename or move file1 to file2 whereis app – show possible locations of app if file2 is an existing directory, moves file1 into which app – show which app will be run by default directory file2 ln -s file link – create symbolic link link to file Compression touch file – create or update file tar cf file.tar files – create a tar named cat > file – places standard input into file file.tar containing files more file – output the contents of file tar xf file.tar – extract the files from file.tar head file – output the first 10 lines of file tar czf file.tar.gz files – create a tar with tail file – output the last 10 lines
  • Unix Essentials (Pdf)

    Unix Essentials (Pdf)

    Unix Essentials Bingbing Yuan Next Hot Topics: Unix – Beyond Basics (Mon Oct 20th at 1pm) 1 Objectives • Unix Overview • Whitehead Resources • Unix Commands • BaRC Resources • LSF 2 Objectives: Hands-on • Parsing Human Body Index (HBI) array data Goal: Process a large data file to get important information such as genes of interest, sorting expression values, and subset the data for further investigation. 3 Advantages of Unix • Processing files with thousands, or millions, of lines How many reads are in my fastq file? Sort by gene name or expression values • Many programs run on Unix only Command-line tools • Automate repetitive tasks or commands Scripting • Other software, such as Excel, are not able to handle large files efficiently • Open Source 4 Scientific computing resources 5 Shared packages/programs https://tak.wi.mit.edu Request new packages/programs Installed packages/programs 6 Login • Requesting a tak account http://iona.wi.mit.edu/bio/software/unix/bioinfoaccount.php • Windows PuTTY or Cygwin Xming: setup X-windows for graphical display • Macs Access through Terminal 7 Connecting to tak for Windows Command Prompt user@tak ~$ 8 Log in to tak for Mac ssh –Y [email protected] 9 Unix Commands • General syntax Command Options or switches (zero or more) Arguments (zero or more) Example: uniq –c myFile.txt command options arguments Options can be combined ls –l –a or ls –la • Manual (man) page man uniq • One line description whatis ls 10 Unix Directory Structure root / home dev bin nfs lab . jdoe BaRC_Public solexa_public
  • ECOGEO Workshop 2: Introduction to Env 'Omics

    ECOGEO Workshop 2: Introduction to Env 'Omics

    ECOGEO Workshop 2: Introduction to Env ‘Omics Unix and Bioinformatics Ben Tully (USC); Ken Youens-Clark (UA) Unix Commands pwd rm grep tail install ls ‘>’ sed cut cd cat nano top mkdir ‘<’ history screen touch ‘|’ $PATH ssh cp sort less df mv uniq head rsync/scp Unix Command Line 1. Open Terminal window Unix Command Line 2. Open Chrome and navigate to Unix tutorial at Protocols.io 3. Group: ECOGEO 4. Protocol: ECOGEO Workshop 2: Unix Module ! This will allow you to copy, paste Unix scripts into terminal window ! ECOGEO Protocols.io for making copy, paste easier Unix Command Line $ ls ls - lists items in the current directory Many commands have additional options that can be set by a ‘-’ $ ls -a Unix Command Line $ ls -a lists all files/directories, including hidden files ‘.’ $ ls -l lists the long format File Permissions | # Link | User | Group | Size | Last modified $ ls -lt lists the long format, but ordered by date last modified Unix Command Line Unix Command Line $ cd ecogeo/ cd - change directory List the contents of the current directory Move into the directory called unix List contents $ pwd pwd - present working directory Unix Command Line /home/c-debi/ecogeo/unix When were we in the directory home? Or c-debi? Or ecogeo? $ cd / Navigates to root directory List contents of root directory This where everything is stored in the computer All the commands we are running live in /bin Unix Command Line / root bin sys home mnt usr c-debi BioinfPrograms cdebi Desktop Downloads ecogeo unix assembly annotation etc Typical Unix Layout Unix Command Line Change directory to home Change directory to c-debi Change directory to ecogeo Change directory to unix List contents Change directory to data Change directory to root Unix Command Line Change directory to unix/data in one step $ cd /home/c-debi/ecogeo/unix/data Tab can be used to auto complete names $ cd .
  • Unix/Linux Command Reference

    Unix/Linux Command Reference

    Unix/Linux Command Reference .com File Commands System Info ls – directory listing date – show the current date and time ls -al – formatted listing with hidden files cal – show this month's calendar cd dir - change directory to dir uptime – show current uptime cd – change to home w – display who is online pwd – show current directory whoami – who you are logged in as mkdir dir – create a directory dir finger user – display information about user rm file – delete file uname -a – show kernel information rm -r dir – delete directory dir cat /proc/cpuinfo – cpu information rm -f file – force remove file cat /proc/meminfo – memory information rm -rf dir – force remove directory dir * man command – show the manual for command cp file1 file2 – copy file1 to file2 df – show disk usage cp -r dir1 dir2 – copy dir1 to dir2; create dir2 if it du – show directory space usage doesn't exist free – show memory and swap usage mv file1 file2 – rename or move file1 to file2 whereis app – show possible locations of app if file2 is an existing directory, moves file1 into which app – show which app will be run by default directory file2 ln -s file link – create symbolic link link to file Compression touch file – create or update file tar cf file.tar files – create a tar named cat > file – places standard input into file file.tar containing files more file – output the contents of file tar xf file.tar – extract the files from file.tar head file – output the first 10 lines of file tar czf file.tar.gz files – create a tar with tail file – output the last 10 lines
  • Bash Crash Course + Bc + Sed + Awk∗

    Bash Crash Course + Bc + Sed + Awk∗

    Bash Crash course + bc + sed + awk∗ Andrey Lukyanenko, CSE, Aalto University Fall, 2011 There are many Unix shell programs: bash, sh, csh, tcsh, ksh, etc. The comparison of those can be found on-line 1. We will primary focus on the capabilities of bash v.4 shell2. 1. Each bash script can be considered as a text file which starts with #!/bin/bash. It informs the editor or interpretor which tries to open the file, what to do with the file and how should it be treated. The special character set in the beginning #! is a magic number; check man magic and /usr/share/file/magic on existing magic numbers if interested. 2. Each script (assume you created “scriptname.sh file) can be invoked by command <dir>/scriptname.sh in console, where <dir> is absolute or relative path to the script directory, e.g., ./scriptname.sh for current directory. If it has #! as the first line it will be invoked by this command, otherwise it can be called by command bash <dir>/scriptname.sh. Notice: to call script as ./scriptname.sh it has to be executable, i.e., call command chmod 555 scriptname.sh before- hand. 3. Variable in bash can be treated as integers or strings, depending on their value. There are set of operations and rules available for them. For example: #!/bin/bash var1=123 # Assigns value 123 to var1 echo var1 # Prints ’var1’ to output echo $var1 # Prints ’123’ to output var2 =321 # Error (var2: command not found) var2= 321 # Error (321: command not found) var2=321 # Correct var3=$var2 # Assigns value 321 from var2 to var3 echo $var3 # Prints ’321’ to output
  • Relational Constraint Driven Test Case Synthesis for Web Applications

    Relational Constraint Driven Test Case Synthesis for Web Applications

    Relational Constraint Driven Test Case Synthesis for Web Applications Xiang Fu Hofstra University Hempstead, NY 11549 [email protected] This paper proposes a relational constraint driven technique that synthesizes test cases automatically for web applications. Using a static analysis, servlets can be modeled as relational transducers, which manipulate backend databases. We present a synthesis algorithm that generates a sequence of HTTP requests for simulating a user session. The algorithm relies on backward symbolic image computation for reaching a certain database state, given a code coverage objective. With a slight adaptation, the technique can be used for discovering workflow attacks on web applications. 1 Introduction Modern web applications usually rely on backend database systems for storing important system infor- mation or supporting business decisions. The complexity of database queries, however, often complicates the task of thoroughly testing a web application. To manually design test cases involves labor intensive initialization of database systems, even with the help of unit testing tools such as SQLUnit [25] and DBUnit [8]. It is desirable to automatically synthesize test cases for web applications. There has been a strong interest recently in testing database driven applications and database man- agement systems (see e.g., [12, 4, 20]). Many of them are query aware, i.e., given a SQL query, an initial database (DB) instance is generated to make that query satisfiable. The DB instance is fed to the target web application as input, so that a certain code coverage goal is achieved. The problem we are trying to tackle is one step further – it is a synthesis problem: given a certain database state (or a relational constraint), a call sequence of web servlets is synthesized to reach the given DB state.
  • Praat Scripting Tutorial

    Praat Scripting Tutorial

    Praat Scripting Tutorial Eleanor Chodroff Newcastle University July 2019 Praat Acoustic analysis program Best known for its ability to: Visualize, label, and segment audio files Perform spectral and temporal analyses Synthesize and manipulate speech Praat Scripting Praat: not only a program, but also a language Why do I want to know Praat the language? AUTOMATE ALL THE THINGS Praat Scripting Why can’t I just modify others’ scripts? Honestly: power, flexibility, control Insert: all the gifs of ‘you can do it’ and ‘you got this’ and thumbs up Praat Scripting Goals ~*~Script first for yourself, then for others~*~ • Write Praat scripts quickly, effectively, and “from scratch” • Learn syntax and structure of the language • Handle various input/output combinations Tutorial Overview 1) Praat: Big Picture 2) Getting started 3) Basic syntax 4) Script types + Practice • Wav files • Measurements • TextGrids • Other? Praat: Big Picture 1) Similar to other languages you may (or may not) have used before • String and numeric variables • For-loops, if else statements, while loops • Regular expression matching • Interpreted language (not compiled) Praat: Big Picture 2) Almost everything is a mouse click! i.e., Praat is a GUI scripting language GUI = Graphical User Interface, i.e., the Objects window If you ever get lost while writing a Praat script, click through the steps using the GUI Getting Started Open a Praat script From the toolbar, select Praat à New Praat script Save immediately! Save frequently! Script Goals and Input/Output • Consider what