Unix Introduction
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
At—At, Batch—Execute Commands at a Later Time
at—at, batch—execute commands at a later time at [–csm] [–f script] [–qqueue] time [date] [+ increment] at –l [ job...] at –r job... batch at and batch read commands from standard input to be executed at a later time. at allows you to specify when the commands should be executed, while jobs queued with batch will execute when system load level permits. Executes commands read from stdin or a file at some later time. Unless redirected, the output is mailed to the user. Example A.1 1 at 6:30am Dec 12 < program 2 at noon tomorrow < program 3 at 1945 pm August 9 < program 4 at now + 3 hours < program 5 at 8:30am Jan 4 < program 6 at -r 83883555320.a EXPLANATION 1. At 6:30 in the morning on December 12th, start the job. 2. At noon tomorrow start the job. 3. At 7:45 in the evening on August 9th, start the job. 4. In three hours start the job. 5. At 8:30 in the morning of January 4th, start the job. 6. Removes previously scheduled job 83883555320.a. awk—pattern scanning and processing language awk [ –fprogram–file ] [ –Fc ] [ prog ] [ parameters ] [ filename...] awk scans each input filename for lines that match any of a set of patterns specified in prog. Example A.2 1 awk '{print $1, $2}' file 2 awk '/John/{print $3, $4}' file 3 awk -F: '{print $3}' /etc/passwd 4 date | awk '{print $6}' EXPLANATION 1. Prints the first two fields of file where fields are separated by whitespace. 2. Prints fields 3 and 4 if the pattern John is found. -
CS101 Lecture 9
How do you copy/move/rename/remove files? How do you create a directory ? What is redirection and piping? Readings: See CCSO’s Unix pages and 9-2 cp option file1 file2 First Version cp file1 file2 file3 … dirname Second Version This is one version of the cp command. file2 is created and the contents of file1 are copied into file2. If file2 already exits, it This version copies the files file1, file2, file3,… into the directory will be replaced with a new one. dirname. where option is -i Protects you from overwriting an existing file by asking you for a yes or no before it copies a file with an existing name. -r Can be used to copy directories and all their contents into a new directory 9-3 9-4 cs101 jsmith cs101 jsmith pwd data data mp1 pwd mp1 {FILES: mp1_data.m, mp1.m } {FILES: mp1_data.m, mp1.m } Copy the file named mp1_data.m from the cs101/data Copy the file named mp1_data.m from the cs101/data directory into the pwd. directory into the mp1 directory. > cp ~cs101/data/mp1_data.m . > cp ~cs101/data/mp1_data.m mp1 The (.) dot means “here”, that is, your pwd. 9-5 The (.) dot means “here”, that is, your pwd. 9-6 Example: To create a new directory named “temp” and to copy mv option file1 file2 First Version the contents of an existing directory named mp1 into temp, This is one version of the mv command. file1 is renamed file2. where option is -i Protects you from overwriting an existing file by asking you > cp -r mp1 temp for a yes or no before it copies a file with an existing name. -
Common Commands Cheat Sheet by Mmorykan Via Cheatography.Com/89673/Cs/20411
Common Commands Cheat Sheet by mmorykan via cheatography.com/89673/cs/20411/ Scripting Scripting (cont) GitHub bash filename - Runs script sleep value - Forces the script to wait value git clone <url > - Clones gitkeeper url Shebang - "# !bi n/b ash " - First line of bash seconds git add <fil ena me> - Adds the file to git script. Tells script what binary to use while [[ condition ]]; do stuff; done git commit - Commits all files to git ./file name - Also runs script if [[ condition ]]; do stuff; fi git push - Pushes all git files to host # - Creates a comment until [[ condition ]]; do stuff; done echo ${varia ble} - Prints variable words=" h ouse dogs telephone dog" - Package / Networking hello_int = 1 - Treats "1 " as a string Declares words array dnf upgrade - Updates system packages Use UPPERC ASE for constant variables for word in ${words} - traverses each dnf install - Installs package element in array Use lowerc ase _wi th_ und ers cores for dnf search - Searches for package for counter in {1..10} - Loops 10 times regular variables dnf remove - Removes package for ((;;)) - Is infinite for loop echo $(( ${hello _int} + 1 )) - Treats hello_int systemctl start - Starts systemd service as an integer and prints 2 break - exits loop body systemctl stop - Stops systemd service mktemp - Creates temporary random file for ((count er=1; counter -le 10; counter ++)) systemctl restart - Restarts systemd service test - Denoted by "[[ condition ]]" tests the - Loops 10 times systemctl reload - Reloads systemd service condition -
Unix Essentials (Pdf)
Unix Essentials Bingbing Yuan Next Hot Topics: Unix – Beyond Basics (Mon Oct 20th at 1pm) 1 Objectives • Unix Overview • Whitehead Resources • Unix Commands • BaRC Resources • LSF 2 Objectives: Hands-on • Parsing Human Body Index (HBI) array data Goal: Process a large data file to get important information such as genes of interest, sorting expression values, and subset the data for further investigation. 3 Advantages of Unix • Processing files with thousands, or millions, of lines How many reads are in my fastq file? Sort by gene name or expression values • Many programs run on Unix only Command-line tools • Automate repetitive tasks or commands Scripting • Other software, such as Excel, are not able to handle large files efficiently • Open Source 4 Scientific computing resources 5 Shared packages/programs https://tak.wi.mit.edu Request new packages/programs Installed packages/programs 6 Login • Requesting a tak account http://iona.wi.mit.edu/bio/software/unix/bioinfoaccount.php • Windows PuTTY or Cygwin Xming: setup X-windows for graphical display • Macs Access through Terminal 7 Connecting to tak for Windows Command Prompt user@tak ~$ 8 Log in to tak for Mac ssh –Y [email protected] 9 Unix Commands • General syntax Command Options or switches (zero or more) Arguments (zero or more) Example: uniq –c myFile.txt command options arguments Options can be combined ls –l –a or ls –la • Manual (man) page man uniq • One line description whatis ls 10 Unix Directory Structure root / home dev bin nfs lab . jdoe BaRC_Public solexa_public -
ECOGEO Workshop 2: Introduction to Env 'Omics
ECOGEO Workshop 2: Introduction to Env ‘Omics Unix and Bioinformatics Ben Tully (USC); Ken Youens-Clark (UA) Unix Commands pwd rm grep tail install ls ‘>’ sed cut cd cat nano top mkdir ‘<’ history screen touch ‘|’ $PATH ssh cp sort less df mv uniq head rsync/scp Unix Command Line 1. Open Terminal window Unix Command Line 2. Open Chrome and navigate to Unix tutorial at Protocols.io 3. Group: ECOGEO 4. Protocol: ECOGEO Workshop 2: Unix Module ! This will allow you to copy, paste Unix scripts into terminal window ! ECOGEO Protocols.io for making copy, paste easier Unix Command Line $ ls ls - lists items in the current directory Many commands have additional options that can be set by a ‘-’ $ ls -a Unix Command Line $ ls -a lists all files/directories, including hidden files ‘.’ $ ls -l lists the long format File Permissions | # Link | User | Group | Size | Last modified $ ls -lt lists the long format, but ordered by date last modified Unix Command Line Unix Command Line $ cd ecogeo/ cd - change directory List the contents of the current directory Move into the directory called unix List contents $ pwd pwd - present working directory Unix Command Line /home/c-debi/ecogeo/unix When were we in the directory home? Or c-debi? Or ecogeo? $ cd / Navigates to root directory List contents of root directory This where everything is stored in the computer All the commands we are running live in /bin Unix Command Line / root bin sys home mnt usr c-debi BioinfPrograms cdebi Desktop Downloads ecogeo unix assembly annotation etc Typical Unix Layout Unix Command Line Change directory to home Change directory to c-debi Change directory to ecogeo Change directory to unix List contents Change directory to data Change directory to root Unix Command Line Change directory to unix/data in one step $ cd /home/c-debi/ecogeo/unix/data Tab can be used to auto complete names $ cd . -
Introduction to Unix
Introduction to Unix Rob Funk <[email protected]> University Technology Services Workstation Support http://wks.uts.ohio-state.edu/ University Technology Services Course Objectives • basic background in Unix structure • knowledge of getting started • directory navigation and control • file maintenance and display commands • shells • Unix features • text processing University Technology Services Course Objectives Useful commands • working with files • system resources • printing • vi editor University Technology Services In the Introduction to UNIX document 3 • shell programming • Unix command summary tables • short Unix bibliography (also see web site) We will not, however, be covering these topics in the lecture. Numbers on slides indicate page number in book. University Technology Services History of Unix 7–8 1960s multics project (MIT, GE, AT&T) 1970s AT&T Bell Labs 1970s/80s UC Berkeley 1980s DOS imitated many Unix ideas Commercial Unix fragmentation GNU Project 1990s Linux now Unix is widespread and available from many sources, both free and commercial University Technology Services Unix Systems 7–8 SunOS/Solaris Sun Microsystems Digital Unix (Tru64) Digital/Compaq HP-UX Hewlett Packard Irix SGI UNICOS Cray NetBSD, FreeBSD UC Berkeley / the Net Linux Linus Torvalds / the Net University Technology Services Unix Philosophy • Multiuser / Multitasking • Toolbox approach • Flexibility / Freedom • Conciseness • Everything is a file • File system has places, processes have life • Designed by programmers for programmers University Technology Services -
Unix (And Linux)
AWK....................................................................................................................................4 BC .....................................................................................................................................11 CHGRP .............................................................................................................................16 CHMOD.............................................................................................................................19 CHOWN ............................................................................................................................26 CP .....................................................................................................................................29 CRON................................................................................................................................34 CSH...................................................................................................................................36 CUT...................................................................................................................................71 DATE ................................................................................................................................75 DF .....................................................................................................................................79 DIFF ..................................................................................................................................84 -
Useful Commands in Linux and Other Tools for Quality Control
Useful commands in Linux and other tools for quality control Ignacio Aguilar INIA Uruguay 05-2018 Unix Basic Commands pwd show working directory ls list files in working directory ll as before but with more information mkdir d make a directory d cd d change to directory d Copy and moving commands To copy file cp /home/user/is . To copy file directory cp –r /home/folder . to move file aa into bb in folder test mv aa ./test/bb To delete rm yy delete the file yy rm –r xx delete the folder xx Redirections & pipe Redirection useful to read/write from file !! aa < bb program aa reads from file bb blupf90 < in aa > bb program aa write in file bb blupf90 < in > log Redirections & pipe “|” similar to redirection but instead to write to a file, passes content as input to other command tee copy standard input to standard output and save in a file echo copy stream to standard output Example: program blupf90 reads name of parameter file and writes output in terminal and in file log echo par.b90 | blupf90 | tee blup.log Other popular commands head file print first 10 lines list file page-by-page tail file print last 10 lines less file list file line-by-line or page-by-page wc –l file count lines grep text file find lines that contains text cat file1 fiel2 concatenate files sort sort file cut cuts specific columns join join lines of two files on specific columns paste paste lines of two file expand replace TAB with spaces uniq retain unique lines on a sorted file head / tail $ head pedigree.txt 1 0 0 2 0 0 3 0 0 4 0 0 5 0 0 6 0 0 7 0 0 8 0 0 9 0 0 10 -
Install and Run External Command Line Softwares
job monitor and control top: similar to windows task manager (space to refresh, q to exit) w: who is there ps: all running processes, PID, status, type ps -ef | grep yyin bg: move current process to background fg: move current process to foreground jobs: list running and suspended processes kill: kill processes kill pid (could find out using top or ps) 1 sort, cut, uniq, join, paste, sed, grep, awk, wc, diff, comm, cat All types of bioinformatics sequence analyses are essentially text processing. Unix Shell has the above commands that are very useful for processing texts and also allows the output from one command to be passed to another command as input using pipe (“|”). less cosmicRaw.txt | cut -f2,3,4,5,8,13 | awk '$5==22' | cut -f1 | sort -u | wc This makes the processing of files using Shell very convenient and very powerful: you do not need to write output to intermediate files or load all data into the memory. For example, combining different Unix commands for text processing is like passing an item through a manufacturing pipeline when you only care about the final product 2 Hands on example 1: cosmic mutation data - Go to UCSC genome browser website: http://genome.ucsc.edu/ - On the left, find the Downloads link - Click on Human - Click on Annotation database - Ctrl+f and then search “cosmic” - On “cosmic.txt.gz” right-click -> copy link address - Go to the terminal and wget the above link (middle click or Shift+Insert to paste what you copied) - Similarly, download the “cosmicRaw.txt.gz” file - Under your home, create a folder -
DRAGEN V3.2 EULA and Readme
Software Copyright Notice Software: DRAGEN v3.2 © Illumina, Inc. 2018 This Software is licensed for use under an End User Software License Agreement: ILLUMINA END-USER SOFTWARE LICENSE AGREEMENT IMPORTANT-READ CAREFULLY. THIS IS A LICENSE AGREEMENT THAT YOU ARE REQUIRED TO ACCEPT BEFORE INSTALLING AND USING ILLUMINA, INC. SOFTWARE. CAREFULLY READ ALL THE TERMS AND CONDITIONS OF THIS LICENSE AGREEMENT BEFORE PROCEEDING WITH THE DOWNLOADING AND/OR INSTALLATION OF THIS SOFTWARE. YOU ARE NOT PERMITTED TO DOWNLOAD AND/OR INSTALL THIS SOFTWARE UNTIL YOU HAVE AGREED TO BE BOUND BY ALL OF THE TERMS AND CONDITIONS OF THIS LICENSE AGREEMENT. BY DOWNLOADING, INSTALLING, OR OTHERWISE ACCESING OR USING THE SOFTWARE, YOU ACKNOWLEDGE THAT YOU HAVE REVIEWED AND ACCEPTED THIS LICENSE AGREEMENT, AND AGREE TO BE BOUND BY THE TERMS OF THIS LICENSE AGREEMENT. YOU ALSO REPRESENT AND WARRANT THAT YOU ARE DULY AUTHORIZED TO ACCEPT THE TERMS AND CONDITIONS OF THIS LICENSE AGREEMENT ON BEHALF OF YOUR EMPLOYER. This End User License Agreement (“Agreement”) is made and entered into by and between Illumina, Inc., a Delaware corporation, having offices at 5200 Illumina Way, San Diego, CA 92122 (“Illumina”) and you as the end-user of the Software (hereinafter, “Licensee” or “you”). All computer programs identified above, software, firmware, and associated media, printed materials and online and electronic documentation, including any updates or upgrades thereof (collectively, “Software”) provided to Licensee are for use solely by Licensee and the provisions herein shall apply with respect to such Software. By using the Software, you indicate your acceptance of these terms and conditions, at which point this Agreement will become a legally binding agreement between you and Illumina. -
Laboratory 1: Getting Familiar with GLUE UNIX Programming Environment
Laboratory 1: Getting Familiar with GLUE UNIX Programming Environment Lecture notes: 1. Scope of the course Prerequisite for ENEE 150 (see the last page for more details), very basic skills in programming and UNIX. a. Principles of programming and software development. b. C will be used as the programming language to illustrate the concepts. c. Basic skills in UNIX operating systems. 2. How to program (or develop software package in the future) a. Document everything you do in each of the following steps. b. Understand the project/problem requirements c. Develop algorithm (the way or method to solve the problem) d. Plan for the implementation of your algorithm (data structure, etc.) e. Write the programming (C, C++, Java, Matlab, etc.) f. Compile the program (gcc or cc in GLUE UNIX for C codes. Compiler is the interpreter that translates the program written in the so-called high level programming languages like C by human, who call themselves programmers, and understandable by human to the low level language that the computer understands.) g. Execute, test, and debug your program on sample data. h. Go back to step d. (modify your code) if necessary (programming or syntax bugs). i. Go back to step c. or step b. if there are serious problems (algorithm or logic bugs). j. Confirm that all the project requirements are met. (output format, etc.) 3. What is UNIX? a. UNIX is an operating system, like windows, which is a complex set of computer codes that manages the activities and resources of the computer. It is very popular in universities and colleges. -
ANSWERS ΤΟ EVEN-Numbered
8 Answers to Even-numbered Exercises 2.1. WhatExplain the following unexpected are result: two ways you can execute a shell script when you do not have execute permission for the file containing the script? Can you execute a shell script if you do not have read permission for the file containing the script? You can give the name of the file containing the script as an argument to the shell (for example, bash scriptfile or tcsh scriptfile, where scriptfile is the name of the file containing the script). Under bash you can give the following command: $ . scriptfile Under both bash and tcsh you can use this command: $ source scriptfile Because the shell must read the commands from the file containing a shell script before it can execute the commands, you must have read permission for the file to execute a shell script. 4.3. AssumeWhat is the purpose ble? you have made the following assignment: $ person=zach Give the output of each of the following commands. a. echo $person zach b. echo '$person' $person c. echo "$person" zach 1 2 6.5. Assumengs. the /home/zach/grants/biblios and /home/zach/biblios directories exist. Specify Zach’s working directory after he executes each sequence of commands. Explain what happens in each case. a. $ pwd /home/zach/grants $ CDPATH=$(pwd) $ cd $ cd biblios After executing the preceding commands, Zach’s working directory is /home/zach/grants/biblios. When CDPATH is set and the working directory is not specified in CDPATH, cd searches the working directory only after it searches the directories specified by CDPATH.