sign in sign up
<<
Home , Echo (command), Grep, Rmdir, Tree (command)

https://bit.ly/3bUvR1s presentation slides Introduction to Dr. Yongjun Choi 03/16/2021 Geting

• Wiki: https://wiki.hpcc.msu.edu/x/4ACE • Submit a ticket: https://contact.icer.msu.edu/contact External resources to learn Linux http://ss64.com/bash/ "An A-Z Index of the line for Linux." with links to 'man' manual page for each. It includes many commands that you will never use, or are specifc to certain versions of linux https://fosswire.com/post/2007/08/unixlinux-command-cheat-sheet/ useful cheatsheet organized by topic https://cvw.cac.cornell.edu/linux/ Full intro to Linux http://guide.bash.academy step by step intro with lots of explanatory text MSU HPC runs Linux • We will use the MSU High Performance Cluster for this course. • consistent experience (Windows, Mac laptop diversity) • It’s a system we know • Highly availability • Course is pre-requisite for other HPCC courses Get started • 1. /identify laptop software to connect • 2. Connect to HPC with secure • 3. Does your account work? Troubleshoot if necessary • 4. basic commands • Step 1: Install Software (if you haven’t) • Windows: install ‘MobaXterm’ free emulation app • When installed, program and then start New Session • Mac OS X: included a unix terminal, but install Xquartz for GUI • Open the application /Applications/Utilities/Terminal.app Get connected • From the terminal/MobaXterm • Type: ssh [email protected] • You can also use: ssh -Y [email protected] • -Y (or -X) is an option for GUI • Enter password - it is hidden and doesn’t look like you are typing, but you are. Linux commands • What is command? • $ • $ -l -a • ls: command (list fles), -l, -a: arguments (l: long , a: all fles including hidden fles) • $ - l myfle.txt • wc: command (word count), -l: argument (line count), myfle.txt: flename Basic commands • : change • ls: list • : current location • : a directory • : fles/dirs • : remove/rename fles/dirs ls • ls: list fles and directories • some options for ls command • -a list all fles and directories including hidden contents • -h: sizes in human readable format (e.g. 1k, 2.5M, 3.1G) • -l: list with a long listing format • -t: my modifcation Exploring files and folders with ls • Step one, explore your home directory with a command ‘ls’ • Note when giving command examples, $ the begging is the prompt (you SHOULD NOT it). Try these commands and check the diference. • $ ls show fles • $ ls -l shows fles in long view • $ ls -al shows all fles • $ ls /mnt/home lists folders of users • $ ls /mnt/home/your_net_id • Now try someone else’s folder - what happens? • $ ls /mnt/home/choiyj cd • cd directory_name: change to named directory • cd: change to home • cd ~: change to home dir • cd ..: change to one level upper dir • cd -: change to the previous dir cp, mkdir , rm, • cp : copy fles • cp -r : copy recursively: fles and directories • mkdir: make a named directory • mv : /rename fles/folders • rm flename: remove a named fle • rm -r : remove a nanmed dir (incluidng all sub directories and fles) • : Can you create a folder ‘test folder’ with mkdir? There IS a space between ‘test’ and ‘folder’. • Question? Use man. eg: man cp manipulation exercise • Create linux_workshop dir on your home. • Copy a folder and contents for this class from • /mnt/research/common-data/workshops/intro2Linux_2021_03_16 • to linux_workshop dir on your home • Go to linux_workshop • a hidden directory and rename it to not_hidden • Check the contents of not_hidden • Create a new directory called new_dir • Copy the fle youfoundit.txt into new_dir • remove garbage dir Files and folders • Linux has a single directory ‘’, separated by slash, th is the ‘root’. • All additional disk are connected on /mnt ‘mounted’. • No concept of driver letters • From your home, try $ tree -L 1, and $ tree -L 2 Exploring files and folders • On our system • /mnt/home/ are all user home directories folders • /mnt/scratch/ are working fles • /bin and /usr/bin and /opt/software software • User pwd to determin your current point in the tree • use ls -l to see all fles in your folder; note it starts with two entries single (.) short representing current folder, and double dot (..) short cut representing parent folder • cd: change directory. cd .. —> go up one level • ~ is shortcut for your home directory: Try $ cd ~ • Files are specifed by the ‘’ through directory tree • Path can be full (/mnt/home/choiyj/.bashrc) starting with / • or relative, from the current location ~/.bashrc permissions • $ ls /mnt/home/choiyj • $ ls /mnt/scratch/choiyj • Linux has a method to keep fles private and save. • permission for user, user groups, and all others File permissions • User, Group, other have read, , or execute permissions. • Permissions are set with ‘’ command, and ownwerhsip set with ‘’ • You can only change these for fles you are the owner of. • Dont grant other/owrld permission on yur home folder. Keep it private! chmod • excutable: 1, write: 2, read: 4, • chmod 777 fle1.txt: open ‘fle1.txt’ to the whole world. Never do that! • you can use • chmod g+ fle1.txt: opn to your group for writing access. • https://en.wikipedia.org/wiki/Chmod • >: write to, • >> : append to • 1: stdout • 2: stderr; • >&: redirection operation • eg: prog > outfle 2>&1 : send stdout and stderr to ‘outfle’ Wildcards • *: anything or nothing • ?: single character • [ ]: any character or range of characters • [! ]: inverse the match of [ ] • $ ls *.txt # list all txt fles • $ ls *-?.txt # list all fles with ‘-‘ and with one cha in front of ‘.txt’ • $ ls [0-9]*.txt # list all fles start with a number. • $ ls [A-Z]*.txt # list all fles start with a capital letter? • Try ls [[:lower:]].txt; ls [[:upper:]].txt; ls [[:lower:][:upper:]].txt • $ ls [!a-Z]*.txt # list txt fles that don’t begin with any letter. • https://wiki.hpcc.msu.edu/display/ITH/Regular+Expressions Learning commands with manual pages

• Linux includes a built in manual for nearly all commands. Type ‘man’ followed by the commands. e.g. • $man man • To navigate the man pages use the arrow keys to scroll up and down or use the enter key to advance al ine, space bar to advance a page, letter u to go back a page. Use the q key to quit out of the display. • The manual pages often include these sections: • Name: a one line desctiption of what it does • Synopsis: basic syntax for the command line. • Description: describes the program’s functionalities. • Options: lists commnand line options available for this program. • Example: examples of some of the options available. • See Also: list of related commands. • Note that options can be with single dash ‘-‘ or double dash ‘—’ Learning commands with manual pages • review tha man pages of common fle/directory commnds • ls, mkdir, cp, pwd, • How can you list all fles in a folder, including hidden fles that start with a dot (.)? • command would create a folder ‘class’ in your home directory? Can you do that? what are the options for this command? • What does pwd do? Shell environment variables • Shell maintains and you can set ‘variables’ that the shell uses for confguration and in your script. • Variables start with $, and can be seen with $VARNAME • Explore common variables with the echo command and list what they are • $HOME, $USER, $SHELL, $PATH • Combine variables in an echo command to make a greeting. • Set a variable of your own, then use in the sentence. • GREETING=Hello; echo $GREETING, $USER - welcom to $HOME • https://wiki.hpcc.msu.edu/display/ITH/1.+Variables+-+Part+I • https://wiki.hpcc.msu.edu/display/ITH/1.+Variables+-+Part+II $PATH • BASH looks for programs in each folder listed in $PATH • When you type a program name, how does the shell fnd it? Let’s try the wich command, it shows the location of programs. • which date • which • which python • each of these folders in the path variable. Command input/output • commands take text as input and output results as text • $ ls -l output text to the screen • $ ls -l > flelist.txt redirect text into a new fle ‘flelist.txt’ • $ ls -l >> flelist.txt redirect text into ‘flelist.txt’ (added at the end of the fle) • $ ls -l | txt pipe text into another command • $ wc -l < flelist.txt redirect fle as input into command • $ cat flelist.txt |wc -l outputfle, pipe into command • Some commands for data • wc: count words, lines or characters • | wc -l # number of users logged in • grep: fnd patterns in fles or data, returns just matching lines • who |grep $USER • sort: given a list of items, sort in various ways • who | sort • : list only top n lines of fle • who > who.txt; head who.txt • : list only last n lines of fle Some commands for data • zip: creat zip of multiple fles • unzip: unzip • : create (tar -) or extract (tar -x) ‘tape archive’ fle.

• Exercise : • Using the man pages, fnd out what the default number of lines that head and tail will display, and how to limit those to just one line • Can you tar all fles and folders in workshop folder? Question? Use man. Creating/editing text files • You can create/edit fles on your local machin and transfer them using ftp app. • Downside: very slow process. • need to run command ‘dos2unix’ if the fle is created under Windows system. • Much faster in the long term to learn how to edit fles with Linux editors. Editing a file with editors Editors • There are many editors in Linux • of an editor is really a religion. • emacs (popular, powerful) • /vim/gvim (popular, powerful) • nano (simple, easy to learn) • gedit (GUI) • Today, we will learn nano a little. To start, type • nano flename • Commands are in the bottom. (^ means control key) nano • EX: create fle ‘hello.sh’ with nano, and write echo ‘Hello!’, and save it. • nano hello.sh GUI • Windows: MobaXterm works out of the box • Mac: First launch Xquartz, then • ssh -Y [email protected] • Launch GUI editor on a dev-node • gedit hello.sh Shell scripting • https://wiki.hpcc.msu.edu/display/ITH/Learning+the+Shell