LAB – 3 Some More Linux Commands

Total Page:16

File Type:pdf, Size:1020Kb

LAB – 3 Some More Linux Commands LAB – 3 Some more linux commands Linux Commands Note: • Spend some time on man pages to know more about the commands, like their general syntax, how to modify their behavior, etc. • Before trying the following commands, ◦ create a folder lab_3, ◦ in this folder create some text files with random text (e.g. This is file1) and ◦ download one or two sample text files from http://www.textfiles.com/directory.html. ◦ Preferably, try to do the first two steps using terminal only. Accessing Files, Commands • more: allows file contents to be sent to the screen one page at a time. $ more <file_name> • less: is similar to more command, but has more features. A user can page up and down through the file. $ less <file_name> [Exercise: Try using more and less on a big sample text file and identify the difference] • head: is used to output first part of the files. $ head <file_name> • tail: is used to output last part of the files. $ tail <file_name> [Exercise: Use head/tail to output first/last k lines instead of default 10 lines] • whereis: is used to locate the binary, source and manual pages of any command. $ whereis <command> e.g. whereis ls will give you the following output: ls: /bin/ls /usr/share/man/man1/ls.1.gz • which: is used to locate executables in the system. $ which <command> e.g. which ls gives: /bin/ls [Exercise: Read about executables in linux and how is it different from windows] • whatis: is used to display one-line description from manual pages. $ whatis <command> e.g. whatis grep gives: grep (1) - print lines matching a pattern • find: searches for files in the given directory, hierarchically starting at the parent directory and moving to subdirectories. $ find -name <file_name> , here -name makes the search case sensitive. [Exercise: Find out how to search for all the files in a directory with a given (e.g. .txt) extension] • history: prints the history of long list of executed commands in the terminal. $ history Altering Files and Operations on Files • cp: is used to copy files and directories. It creates a new file if the destination file doesn't exist, and overwrites the destination file if it already exists. $ cp <file_name1> <file_name2> [Exercise: Find how to copy all the files, subdirectories in a directory to new a directory] • cat: is used to join two or more plain files and/or print contents of a file on standard output. $ cat <file_name1> <file_name2> [Exercise: Try cat <file_name1> <file_name2> >> <file_name3>. Find out the difference between using > and >>] • grep: searches the given input file for lines containing a match to the given strings or words. $ grep <pattern> <file_name> e.g. grep this one.txt searches for the word 'this' in file one.txt. [Exercise: Try different options (-i, -x, -c, -n) and identify the difference. Find out how to search for a word in all the files in a directory] • sort: is used to the sort the lines of text files. $ sort <file_name> [Exercise: Try sort, sort -r and sort -n and identify the differences] • cmp: compares two files of any type byte by byte and writes the results to the standard output. It outputs the byte where files differ. $ cmp <file_name1> <file_name2> [Exercise: Try using SKIP bytes (refer to man page) and identify the difference using '-b' option] • uniq: reports or omits repeated lines. $ uniq <file_name> [Exercise: Try options -c, -d, -i, -u] • touch: updates the access and modification times of given file to the current time. It also creates the file in case the given file doesn't exist. $ touch <file_name> [Exercise: Use ls -l before and after using touch command on a file, and check the difference between access times] • tar: stands for Tape ARchive, is a utility to archive and extract files in a number of file formats. $ tar <options> <archive_name> <file_name1> . Refer to these link: https://users.cs.duke.edu/~ola/courses/programming/tar.html and http://www.cyberciti.biz/faq/tar-extract-linux/ to know more about creating and extracting archives using tar. [Exercise: Try to create an archive using two or more files, display the table of contents in an archive and extract an archive] • chmod: is used to change file permissions of each given file, folder, script, etc. $ chmod <mode> <file_name> e.g. chmod 664 one.txt gives read and write permissions to user and group, and gives only read permission to others for one.txt file. [Exercise: Try to change permissions of all the files in a directory] Miscellaneous • lsblk: prints block devices by their assigned name. By default, ram is not listed (Try using option '-a'). $ lsblk • df: reports disk usages of file system. • kill: is used to send a signal to a process. To stop a process, a pid (process id) which can be obtained from ps or top and a signal number (optional) is required. Two signals in particular SIGTERM (9) and SIGKILL (15) are used to stop a process. The default signal is SIGTERM. $ kill <pid> [Exercise: Try kill -l, kill -9 pid, kill -15 pid. Also, identify the difference between kill and pkill.] • uname: prints detailed information about the machine name, OS and kernel. $ uname -a • echo: displays a line of text on the standard output. This command is commonly used in interactive shell scripting. Piping and Redirection '|' - Pipe operation is a mechanism for sending data/output from one program/command to another. The operator '|' feeds the output from the command on the left as input to the command on the right. e.g. In the below example output from ls is given to head which outputs first 3 lines of the input. We can pipe as many commands as we like: Some more examples: • ls -l | grep "Aug" outputs only list of files which were accessed or modified in August. • ls -l | sort -nk5 outputs the sorted list of files by their size. • cat one.txt | grep the searches for the word 'the' in the file one.txt. Redirection to files is performed using '>' - save output to a file, '<' - read input from a file, '>>' - append to a file. Using '>' outputs data to a file. If the file exists, it clears the file first and then saves the data. If the file doesn't exist, it creates a file first and then writes the data. e.g. ls > output1 writes the output of ls to file output1. Using '>>', we can append data to the existing file, instead of overwriting it. Below, we can see that '>>' appends the output of wc -l 221baker.txt at the end of the file output1. Similarly, '<' is used to read from a file. e.g. wc -l < output1 outputs the number of lines in the file output1. References: • Images on page 2 and 4 from tecmint.com..
Recommended publications
  • How to Find out the IP Address of an Omron
    Communications Middleware/Network Browser How to find an Omron Controller’s IP address Valin Corporation | www.valin.com Overview • Many Omron PLC’s have Ethernet ports or Ethernet port options • The IP address for a PLC is usually changed by the programmer • Most customers do not mark the controller with IP address (label etc.) • Very difficult to communicate to the PLC over Ethernet if the IP address is unknown. Valin Corporation | www.valin.com Simple Ethernet Network Basics IP address is up to 12 digits (4 octets) Ex:192.168.1.1 For MOST PLC programming applications, the first 3 octets are the network address and the last is the node address. In above example 192.168.1 is network address, 1 is node address. For devices to communicate on a simple network: • Every device IP Network address must be the same. • Every device node number must be different. Device Laptop EX: Omron PLC 192.168.1.1 192.168.1.1 Device Laptop EX: Omron PLC 127.27.250.5 192.168.1.1 Device Laptop EX: Omron PLC 192.168.1.3 192.168.1.1 Valin Corporation | www.valin.com Omron Default IP Address • Most Omron Ethernet devices use one of the following IP addresses by default. Omron PLC 192.168.250.1 OR 192.168.1.1 Valin Corporation | www.valin.com PING Command • PING is a way to check if the device is connected (both virtually and physically) to the network. • Windows Command Prompt command. • PC must use the same network number as device (See previous) • Example: “ping 172.21.90.5” will test to see if a device with that IP address is connected to the PC.
    [Show full text]
  • Disk Clone Industrial
    Disk Clone Industrial USER MANUAL Ver. 1.0.0 Updated: 9 June 2020 | Contents | ii Contents Legal Statement............................................................................... 4 Introduction......................................................................................4 Cloning Data.................................................................................................................................... 4 Erasing Confidential Data..................................................................................................................5 Disk Clone Overview.......................................................................6 System Requirements....................................................................................................................... 7 Software Licensing........................................................................................................................... 7 Software Updates............................................................................................................................. 8 Getting Started.................................................................................9 Disk Clone Installation and Distribution.......................................................................................... 12 Launching and initial Configuration..................................................................................................12 Navigating Disk Clone.....................................................................................................................14
    [Show full text]
  • Modern Programming Languages CS508 Virtual University of Pakistan
    Modern Programming Languages (CS508) VU Modern Programming Languages CS508 Virtual University of Pakistan Leaders in Education Technology 1 © Copyright Virtual University of Pakistan Modern Programming Languages (CS508) VU TABLE of CONTENTS Course Objectives...........................................................................................................................4 Introduction and Historical Background (Lecture 1-8)..............................................................5 Language Evaluation Criterion.....................................................................................................6 Language Evaluation Criterion...................................................................................................15 An Introduction to SNOBOL (Lecture 9-12).............................................................................32 Ada Programming Language: An Introduction (Lecture 13-17).............................................45 LISP Programming Language: An Introduction (Lecture 18-21)...........................................63 PROLOG - Programming in Logic (Lecture 22-26) .................................................................77 Java Programming Language (Lecture 27-30)..........................................................................92 C# Programming Language (Lecture 31-34) ...........................................................................111 PHP – Personal Home Page PHP: Hypertext Preprocessor (Lecture 35-37)........................129 Modern Programming Languages-JavaScript
    [Show full text]
  • Unix Command Line; Editors
    Unix command line; editors Karl Broman Biostatistics & Medical Informatics, UW–Madison kbroman.org github.com/kbroman @kwbroman Course web: kbroman.org/AdvData My goal in this lecture is to convince you that (a) command-line-based tools are the things to focus on, (b) you need to choose a powerful, universal text editor (you’ll use it a lot), (c) you want to be comfortable and skilled with each. For your work to be reproducible, it needs to be code-based; don’t touch that mouse! Windows vs. Mac OSX vs. Linux Remote vs. Not 2 The Windows operating system is not very programmer-friendly. Mac OSX isn’t either, but under the hood, it’s just unix. Don’t touch the mouse! Open a terminal window and start typing. I do most of my work directly on my desktop or laptop. You might prefer to work remotely on a server, instead. But I can’t stand having any lag in looking at graphics. If you use Windows... Consider Git Bash (or Cygwin) or turn on the Windows subsystem for linux 3 Cygwin is an effort to get Unix command-line tools in Windows. Git Bash combines git (for version control) and bash (the unix shell); it’s simpler to deal with than Cygwin. Linux is now accessible in Windows 10, but you have to enable it. If you use a Mac... Consider Homebrew and iTerm2 Also the XCode command line tools 4 Homebrew is a packaging system; iTerm2 is a Terminal replacement. The XCode command line tools are a must for most unixy things on a Mac.
    [Show full text]
  • 101 Useful Linux Commands - Haydenjames.Io
    101 Useful Linux Commands - haydenjames.io Some of these commands require elevated permissions (sudo) to run. Enjoy! 1. Execute the previous command used: !! 2. Execute a previous command starting with a specific letter. Example: !s 3. Short way to copy or backup a file before you edit it. For example, copy nginx.conf cp nginx.conf{,.bak} 4. Toggle between current directory and last directory cd - 5. Move to parent (higher level) directory. Note the space! cd .. 6. Go to home directory cd ~ 7. Go to home directory cd $HOME 8. Go to home directory (when used alone) cd 9. Set permissions to 755. Corresponds to these permissions: (-rwx-r-x-r-x), arranged in this sequence: (owner-group-other) chmod 755 <filename> 10. Add execute permission to all users. chmod a+x <filename> 11. Changes ownership of a file or directory to . chown <username> 12. Make a backup copy of a file (named file.backup) cp <file> <file>.backup 13. Copy file1, use it to create file2 cp <file1> <file2> 14. Copy directory1 and all its contents (recursively) into directory2 cp -r <directory1> <directory2>/ 15. Display date date 16. Zero the sdb drive. You may want to use GParted to format the drive afterward. You need elevated permissions to run this (sudo). dd if=/dev/zero of=/dev/sdb 17. Display disk space usage df -h 18. Take detailed messages from OS and input to text file dmesg>dmesg.txt 19. Display a LOT of system information. I usually pipe output to less. You need elevated permissions to run this (sudo).
    [Show full text]
  • “Linux at the Command Line” Don Johnson of BU IS&T  We’Ll Start with a Sign in Sheet
    “Linux at the Command Line” Don Johnson of BU IS&T We’ll start with a sign in sheet. We’ll end with a class evaluation. We’ll cover as much as we can in the time allowed; if we don’t cover everything, you’ll pick it up as you continue working with Linux. This is a hands-on, lab class; ask questions at any time. Commands for you to type are in BOLD The Most Common O/S Used By BU Researchers When Working on a Server or Computer Cluster Linux is a Unix clone begun in 1991 and written from scratch by Linus Torvalds with assistance from a loosely-knit team of hackers across the Net. 64% of the world’s servers run some variant of Unix or Linux. The Android phone and the Kindle run Linux. a set of small Linux is an O/S core programs written by written by Linus Richard Stallman and Torvalds and others others. They are the AND GNU utilities. http://www.gnu.org/ Network: ssh, scp Shells: BASH, TCSH, clear, history, chsh, echo, set, setenv, xargs System Information: w, whoami, man, info, which, free, echo, date, cal, df, free Command Information: man, info Symbols: |, >, >>, <, ;, ~, ., .. Filters: grep, egrep, more, less, head, tail Hotkeys: <ctrl><c>, <ctrl><d> File System: ls, mkdir, cd, pwd, mv, touch, file, find, diff, cmp, du, chmod, find File Editors: gedit, nedit You need a “xterm” emulation – software that emulates an “X” terminal and that connects using the “SSH” Secure Shell protocol. ◦ Windows Use StarNet “X-Win32:” http://www.bu.edu/tech/support/desktop/ distribution/xwindows/xwin32/ ◦ Mac OS X “Terminal” is already installed Why? Darwin, the system on which Apple's Mac OS X is built, is a derivative of 4.4BSD-Lite2 and FreeBSD.
    [Show full text]
  • Unix/Linux Command Reference
    Unix/Linux Command Reference .com File Commands System Info ls – directory listing date – show the current date and time ls -al – formatted listing with hidden files cal – show this month's calendar cd dir - change directory to dir uptime – show current uptime cd – change to home w – display who is online pwd – show current directory whoami – who you are logged in as mkdir dir – create a directory dir finger user – display information about user rm file – delete file uname -a – show kernel information rm -r dir – delete directory dir cat /proc/cpuinfo – cpu information rm -f file – force remove file cat /proc/meminfo – memory information rm -rf dir – force remove directory dir * man command – show the manual for command cp file1 file2 – copy file1 to file2 df – show disk usage cp -r dir1 dir2 – copy dir1 to dir2; create dir2 if it du – show directory space usage doesn't exist free – show memory and swap usage mv file1 file2 – rename or move file1 to file2 whereis app – show possible locations of app if file2 is an existing directory, moves file1 into which app – show which app will be run by default directory file2 ln -s file link – create symbolic link link to file Compression touch file – create or update file tar cf file.tar files – create a tar named cat > file – places standard input into file file.tar containing files more file – output the contents of file tar xf file.tar – extract the files from file.tar head file – output the first 10 lines of file tar czf file.tar.gz files – create a tar with tail file – output the last 10 lines
    [Show full text]
  • Problem Solving and Unix Tools
    Problem Solving and Unix Tools Command Shell versus Graphical User Interface • Ease of use • Interactive exploration • Scalability • Complexity • Repetition Example: Find all Tex files in a directory (and its subdirectories) that have not changed in the past 21 days. With an interactive file roller, it is easy to sort files by particular characteristics such as the file extension and the date. But this sorting does not apply to files within subdirectories of the current directory, and it is difficult to apply more than one sort criteria at a time. A command line interface allows us to construct a more complex search. In unix, we find the files we are after by executing the command, find /home/nolan/ -mtime +21 -name ’*.tex’ To find out more about a command you can read the online man pages man find or you can execute the command with the –help option. In this example, the standard output to the screen is piped into the more command which formats it to dispaly one screenful at a time. Hitting the space bar displays the next page of output, the return key displays the next line of output, and the ”q” key quits the display. find --help | more Construct Solution in Pieces • Solve a problem by breaking down into pieces and building back up • Typing vs automation • Error messages - experimentation 1 Example: Find all occurrences of a particular string in several files. The grep command searches the contents of files for a regular expression. In this case we search for the simple character string “/stat141/FINAL” in all files in the directory WebLog that begin with the filename “access”.
    [Show full text]
  • Exercise I: Basic Unix for Manipulating NGS Data
    Exercise I: Basic Unix for manipulating NGS data C. Hahn, July 2014 The purpose of this exercise is to practice manipulating NGS (Next Generation Sequencing) data using simple but incredibly powerful Unix commands. Try to solve the below tasks using your Unix skills. Do not hesitate to consult Google for help!! Hints for every task can be found on page 3 below. Possible solutions are suggested on page 4. 1. Download the testdata from https://www.dropbox.com/s/wcanmej6z03yfmt/testdata_1.tar?dl=0. Create a directory (exercise_1) in your home directory and copy the archive testdata.tar from ~/Downloads/ to this directory. 2. Decompress the archive. 3. Examine the content of the data files. For basic information on fastq format please visit: http://en.wikipedia.org/wiki/FASTQ_format. With this information try to interpret the content of your data files. Do you Know what all the lines represent? a) Which quality encoding (which offset) do these files use? b) What is the header of the third read in the file? Is this a single-end (forward) or a paired-end (reverse) read? c) What is the header of the last read in the file? Is this a forward or reverse read? 4. How many lines does the file have? 5. How many reads does the file contain? 6. How many single-end (“se” also referred to as forward reads) reads and how many paired-end (“pe” also reverse reads) reads does the file contain? 7. Extract all se/pe reads and write them to separate files testdata_1.fastq and testdata_2.fastq.
    [Show full text]
  • What Is UNIX? the Directory Structure Basic Commands Find
    What is UNIX? UNIX is an operating system like Windows on our computers. By operating system, we mean the suite of programs which make the computer work. It is a stable, multi-user, multi-tasking system for servers, desktops and laptops. The Directory Structure All the files are grouped together in the directory structure. The file-system is arranged in a hierarchical structure, like an inverted tree. The top of the hierarchy is traditionally called root (written as a slash / ) Basic commands When you first login, your current working directory is your home directory. In UNIX (.) means the current directory and (..) means the parent of the current directory. find command The find command is used to locate files on a Unix or Linux system. find will search any set of directories you specify for files that match the supplied search criteria. The syntax looks like this: find where-to-look criteria what-to-do All arguments to find are optional, and there are defaults for all parts. where-to-look defaults to . (that is, the current working directory), criteria defaults to none (that is, select all files), and what-to-do (known as the find action) defaults to ‑print (that is, display the names of found files to standard output). Examples: find . –name *.txt (finds all the files ending with txt in current directory and subdirectories) find . -mtime 1 (find all the files modified exact 1 day) find . -mtime -1 (find all the files modified less than 1 day) find . -mtime +1 (find all the files modified more than 1 day) find .
    [Show full text]
  • Useful Commands in Linux and Other Tools for Quality Control
    Useful commands in Linux and other tools for quality control Ignacio Aguilar INIA Uruguay 05-2018 Unix Basic Commands pwd show working directory ls list files in working directory ll as before but with more information mkdir d make a directory d cd d change to directory d Copy and moving commands To copy file cp /home/user/is . To copy file directory cp –r /home/folder . to move file aa into bb in folder test mv aa ./test/bb To delete rm yy delete the file yy rm –r xx delete the folder xx Redirections & pipe Redirection useful to read/write from file !! aa < bb program aa reads from file bb blupf90 < in aa > bb program aa write in file bb blupf90 < in > log Redirections & pipe “|” similar to redirection but instead to write to a file, passes content as input to other command tee copy standard input to standard output and save in a file echo copy stream to standard output Example: program blupf90 reads name of parameter file and writes output in terminal and in file log echo par.b90 | blupf90 | tee blup.log Other popular commands head file print first 10 lines list file page-by-page tail file print last 10 lines less file list file line-by-line or page-by-page wc –l file count lines grep text file find lines that contains text cat file1 fiel2 concatenate files sort sort file cut cuts specific columns join join lines of two files on specific columns paste paste lines of two file expand replace TAB with spaces uniq retain unique lines on a sorted file head / tail $ head pedigree.txt 1 0 0 2 0 0 3 0 0 4 0 0 5 0 0 6 0 0 7 0 0 8 0 0 9 0 0 10
    [Show full text]
  • Install and Run External Command Line Softwares
    job monitor and control top: similar to windows task manager (space to refresh, q to exit) w: who is there ps: all running processes, PID, status, type ps -ef | grep yyin bg: move current process to background fg: move current process to foreground jobs: list running and suspended processes kill: kill processes kill pid (could find out using top or ps) 1 sort, cut, uniq, join, paste, sed, grep, awk, wc, diff, comm, cat All types of bioinformatics sequence analyses are essentially text processing. Unix Shell has the above commands that are very useful for processing texts and also allows the output from one command to be passed to another command as input using pipe (“|”). less cosmicRaw.txt | cut -f2,3,4,5,8,13 | awk '$5==22' | cut -f1 | sort -u | wc This makes the processing of files using Shell very convenient and very powerful: you do not need to write output to intermediate files or load all data into the memory. For example, combining different Unix commands for text processing is like passing an item through a manufacturing pipeline when you only care about the final product 2 Hands on example 1: cosmic mutation data - Go to UCSC genome browser website: http://genome.ucsc.edu/ - On the left, find the Downloads link - Click on Human - Click on Annotation database - Ctrl+f and then search “cosmic” - On “cosmic.txt.gz” right-click -> copy link address - Go to the terminal and wget the above link (middle click or Shift+Insert to paste what you copied) - Similarly, download the “cosmicRaw.txt.gz” file - Under your home, create a folder
    [Show full text]