DOCSLIB.ORG

Intermediate Linux

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Intermediate Linux Intermediate Linux th st !!***May subroutine 30 mpi_utils_step_parallel_edge– 31 , 2016 implicit none integer ele, ierr do ele = 1, nfaces Lecturerscall mpi_isend(commvec(ele)%field1(commvec(ele)%out_i1, (in alphabetical order): & & commvec(ele)%out_j1, & Jaakko& Leinonen commvec(ele)%out_k1), & Tomasz& 1, commvec(ele)%mpi_type_out1,Malkiewicz & & commvec(ele)%to_id, commvec(ele)%to_id, & Thomas& MPI_COMM_WORLD, Zwinger send_reqs(ele), ierr) if(ierr /= MPI_SUCCESS) then call pio_abort(ierr) end if call mpi_isend(commvec(ele)%field2(commvec(ele)%out_i2, & & commvec(ele)%out_j2, & & commvec(ele)%out_k2), & & 1, commvec(ele)%mpi_type_out2, & & commvec(ele)%to_id, commvec(ele)%to_id+tag_offset, & & MPI_COMM_WORLD, send_reqs(nfaces+ele), ierr) if(ierr /= MPI_SUCCESS) then call pio_abort(ierr) end if end do #ifdef NONBLOCK do ele = 1, nfaces All material (C) 2016 by the authors. call mpi_irecv(commvec(ele)%field1(commvec(ele)%in_i1, & This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported & commvec(ele)%in_j1, & 1 License, http://creativecommons.org/licenses/by-nc-sa/3.0/ 2 & commvec(ele)%in_k1), & & 1, commvec(ele)%mpi_type_in1, & Program, May 30th 09:30 – 10:00 Morning coffee & registration 10:00 – 10:15 Introduction to the course (whereabouts, etc.) 10:15 – 10:45 Recapping Introduction to Linux (to refresh memory) 10:45 – 11:30 Basic concepts of shells 11:30 – 12:30 Lunch 12:30 – 14:30 Working with the Bash shell 14:30 – 15:00 Coffee 15:00 – 15:45 Bash job management and signals About This Course 15:45 – 16:30 Hands-on exercises & Troubleshooter: Interactive session to deal with open questions and specific problems 3 4 Program, May 31st How We Teach 09:00 – 10:00 Pipes, named pipes and redirecting All topics are presented with interactive demonstrations. 10:00 – 10:30 Coffee – Please, indicate immediately, if pace is too fast. We want to have 10:30 – 12:00 Shell commands and utilities everyone with us all the time. 12:00 – 13:00 Lunch Additionally, exercises to each of the sections will be provided. 13:00 – 14:00 Pattern matching and regular expressions 14:00 – 14:30 Coffee The Troubleshooter section is meant for personal interaction and is 14:30 – 15:00 Software and revision control (SVN & Git) (with a time-limit to 16:30 or 16:00) kept in an open end style. 15:00 – 16:00 Hands-on exercises & Troubleshooter: Interactive session to deal with open questions and specific problems 5 6 Around CSC Practicalities B1 (102,103) Kamppi Keep the name tag visible. B2 (194/5,503/4/… Pasila,… Lunch is served in the same building. – Room will be locked during lunch but lobby is open, use the lockers. Toilets are in the lobby. B2 Restaurant You are Network: here – Wi-Fi: eduroam, HAKA authentication (Training – Ethernet cables on the tables CSC room) – CSC-Guest accounts upon request B1 Transportation: – Bus stops to Kamppi/Center are located at the other side of the street (102, 103). – Bus stops to Center/Pasila are on this side, towards the bridge (194, 195, 503-6). Restaurant – To arrive at CSC the stops are at the same positions, just on opposite sides on the street. – If you came by car: parking is being monitored – ask for a temporary parking permit from the reception (tell which workshop you’re participating). Visiting outside: the doors by the reception desks are open. Username and password for workstations: given on-site. (K4 Salad bar) (THINK restaurant) 7 8 What Is a Shell? "A shell in computing provides a user interface for access to an operating system’s kernel services." (Wikipedia) Remote login: – Normally no GUI (Graphical User Interface) – Text shell: Terminal with a set of commands Different flavours: – bash (default), tcsh (old default), zsh, corn-shell, … The basics (recapping earlier course) 9 10 What Is a Command? Listing of Directories A command is a small program provided by the shell Print contents of a directory or information on a file The over-all structure of a command is: Detailed list of directory: command -option [optional input] $ ls –lthr /etc/ Example:”$” is not part of the command, but depicts the command prompt – -l displays additional information (detailed list in Windows) $ ls –lsh /etc/init.d (we will see later) – -h displays size in human readable format Case sensitive? Try: – -t orders by date (use -r to reverse order, i.e., oldest first) $ Ls –lsh /etc/init.d – -d keeps from going into sub-directories How to find a command? Only print directory/filenames matching a wildcard expression: $ apropos list $ ls -d /etc/*.d How to find all options? Only print directory/filenames with a 4 char suffix: $ man ls $ ls –l /etc/*.???? 11 12 Moving Around in Directories Creating and (Re-)Moving Directories Change directory: $ cd /etc/ Make a new directory: $ mkdir test1 Print work directory: $ pwd – Recursively: $ mkdir -p test2/anotherone Go to subdirectory: $ cd ./init.d Moving a directory: $ mv test2 test3 Relative path: $ cd ../ Removing a directory: $ rmdir test2/anotherone Absolute path: $ cd /etc/init.d – Recursively: $ rmdir -p test3/anotherone Combination: $ cd ../../usr Where is home: $ cd or cd ~/ 13 14 Creating/Copying/(Re-)Moving Files File Permissions Copy a file: $ cp mytest.txt othertest2.txt UNIX distinguishes between users, groups and others – Recursively: $ cp -r test test2 – Check your groups: $ groups Move a file (renaming): $ mv mytest.txt othertest3.txt Each user belongs to at least one group Remove file(s): $ rm –f mytest.txt To displays the attributes of a file or directory: – Recursively: $ rm –r test2 $ ls -l name -rw-r--r-- 1 userid groupid 0 Jan 29 11:04 name type group user others Size of file (usually in bytes) Number of references from file system r = read, w = write, x = execute The above configuration means: user can read + write, group and all others only read 15 16 File Permissions (cont'd) Finding Stuff Changing permissions with chmod: The hard way: cd yourself through the tree and ls $ ls -l lala The elegant way: -rw-r--r-- 1 userid groupid 0 Jan 29 11:04 lala $ find /etc -name "*.conf" -print $ chmod o-r,g+w,u+x lala $ ls –l lala – Finds all config files in the /etc tree -rwxrw---- 1 userid groupid 0 Jan 29 11:04 lala The alternative: $ chmod u-xrw lala $ locate *.conf $ less lala 17 18 Finding Stuff (cont'd) Managing Space Finding expressions inside files: How much space is left on my filesystem? – For instance, we want to know all files in the directory /etc/init.d that $ df -h contain keyword "network": Filesystem Size Used Avail Use% Mounted on /dev/sda5 22G 20G 903M 96% / $ grep network /etc/init.d/* /dev/sda1 447M 27M 396M 7% /boot .host:/ 12G 8.0G 4.1G 66% /mnt/hgfs – Or recursively: $ grep -r network /etc – Getting rid of noise: What are the sub-directories that consume the most disk-space? $ grep -r network /etc 2 >/dev/null $ du -sh ./* 1.4M bin Piping of output: 6.3M core 44K Desktop – Instead of re-directing into files, output can be piped in a chain of 696M Documents 1.2G Downloads commands: … $ grep –r network /etc 2> /dev/null| grep start| less 19 20 Login Remote Copying Only secure connections (no telnet, rlogin) are recommended scp is like cp, but used for remote transfer Quotes are Secure Shell (SSH): $ scp lala [email protected]:'$HOME' important here $ ssh [email protected] -X rsync works local as well as remotely and helps to keep two – Option -X tunnels the graphical output or alternatively -Y, e.g. (remote) directories in sync: $ ssh [email protected] -Y $ mv lala test $ rsync –avt test/ test2 – This syncs everything in test with test2 – Important: Do not drop trailing / – Remotely: $ rsync –avt test [email protected]:'$HOME' 21 22 (De-)compressing files Archives of Files GNU zip: Most common: tar (tape archive) – To inflate: – Take whole sub-tree and make a single compressed file $ gunzip hello-2.7.tar.gz $ tar cvzf myfirsttarfile.tar.gz /etc/init.d – To compress: . c create new archive $ gzip hello-2.7.tar . v be verbose BZIP (Bzip2): bzip2, bunzip2 . z gzip simultaneously ZIP: . f target file – To compress: $ zip myvar.zip myvariables.txt – Check contents (and simultaneously gunzip) $ tar tvzf hello-2.7.tar.gz – Listing: $ unzip –l myvar.zip – Unpack (and simultaneously gunzip): – To inflate: $ unzip myvar.zip $ tar xvzf hello-2.7.tar.gz 23 24 What is shell? Command interpreter between machine and human, a basic user interface to a computer running Unix Different flavors – bash – tcsh – zsh – corn-shell, … Basic concepts of shells 25 26 Shell vs Terminal / Console Shell concept A shell is a program which provides the traditional, text- Shell is mandatory by POSIX specification only user interface for Linux (and other Unix like systems) Reads in a typed command terminated by Enter and then parses it Shell’s primary function is to read commands that are typed into a console or terminal window and then execute them Commands may be built-ins or executable applications Commands may contain options and/or arguments Terminal - a wrapper program which runs a shell Console – special sort of terminal, used for direct communication at a low level with the operating system 27 28 Shell commands Commands cont. Structure of a command: A command is an instruction given by a user telling a computer to do something, e.g.: command -option [optional input] – run a single program – run a group of linked programs Examples: – whoami Commands are generally issued by typing them in at the command line and then pressing the ENTER key, which – ls –l passes them to the shell – file skype 29 30 Programs Bash Command-line programs De-facto standard shell on most Unix-like operating systems Examples: Can generally be found everywhere – gnuplot – cal GNU project's Bourne Again SHell Open source – Bash implements essentially the same grammar, parameter and – gimp variable expansion, redirection, and quoting as the Bourne Shell – gnumeric Closed source – Main differences between Bash and Bourne: – skype .
Load more

Recommended publications
  • Beginning Portable Shell Scripting from Novice to Professional
    Beginning Portable Shell Scripting From Novice to Professional Peter Seebach 10436fmfinal 1 10/23/08 10:40:24 PM Beginning Portable Shell Scripting: From Novice to Professional Copyright © 2008 by Peter Seebach All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and the publisher. ISBN-13 (pbk): 978-1-4302-1043-6 ISBN-10 (pbk): 1-4302-1043-5 ISBN-13 (electronic): 978-1-4302-1044-3 ISBN-10 (electronic): 1-4302-1044-3 Printed and bound in the United States of America 9 8 7 6 5 4 3 2 1 Trademarked names may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, we use the names only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. Lead Editor: Frank Pohlmann Technical Reviewer: Gary V. Vaughan Editorial Board: Clay Andres, Steve Anglin, Ewan Buckingham, Tony Campbell, Gary Cornell, Jonathan Gennick, Michelle Lowman, Matthew Moodie, Jeffrey Pepper, Frank Pohlmann, Ben Renow-Clarke, Dominic Shakeshaft, Matt Wade, Tom Welsh Project Manager: Richard Dal Porto Copy Editor: Kim Benbow Associate Production Director: Kari Brooks-Copony Production Editor: Katie Stence Compositor: Linda Weidemann, Wolf Creek Press Proofreader: Dan Shaw Indexer: Broccoli Information Management Cover Designer: Kurt Krames Manufacturing Director: Tom Debolski Distributed to the book trade worldwide by Springer-Verlag New York, Inc., 233 Spring Street, 6th Floor, New York, NY 10013.
    [Show full text]
  • The Linux Command Line
    The Linux Command Line Fifth Internet Edition William Shotts A LinuxCommand.org Book Copyright ©2008-2019, William E. Shotts, Jr. This work is licensed under the Creative Commons Attribution-Noncommercial-No De- rivative Works 3.0 United States License. To view a copy of this license, visit the link above or send a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042. A version of this book is also available in printed form, published by No Starch Press. Copies may be purchased wherever fine books are sold. No Starch Press also offers elec- tronic formats for popular e-readers. They can be reached at: https://www.nostarch.com. Linux® is the registered trademark of Linus Torvalds. All other trademarks belong to their respective owners. This book is part of the LinuxCommand.org project, a site for Linux education and advo- cacy devoted to helping users of legacy operating systems migrate into the future. You may contact the LinuxCommand.org project at http://linuxcommand.org. Release History Version Date Description 19.01A January 28, 2019 Fifth Internet Edition (Corrected TOC) 19.01 January 17, 2019 Fifth Internet Edition. 17.10 October 19, 2017 Fourth Internet Edition. 16.07 July 28, 2016 Third Internet Edition. 13.07 July 6, 2013 Second Internet Edition. 09.12 December 14, 2009 First Internet Edition. Table of Contents Introduction....................................................................................................xvi Why Use the Command Line?......................................................................................xvi
    [Show full text]
  • Linux Hardening Techniques Vasudev Baldwa Ubnetdef, Spring 2021 Agenda
    Linux Hardening Techniques Vasudev Baldwa UBNetDef, Spring 2021 Agenda 1. What is Systems Hardening? 2. Basic Principles 3. Updates & Encryption 4. Monitoring 5. Services 6. Firewalls 7. Logging What is System Hardening? ⬡ A collection of tools, techniques, and best practices to reduce vulnerability in technology applications, systems, infrastructure, firmware, and other areas ⬡ 3 major areas: OS vs Software vs Network ⬠ When have we done hardening in this class before? ⬠ This lecture is focusing mostly on OS and software level Why Harden? ⬡ Firewalls can only get us so far, what happens when at attack is inside the network? ⬠ If you have nothing protecting your systems you are in trouble ⬡ We want some kind of secondary protection A Few Cybersecurity Principles ⬡ Zero Trust Security ⬠ Instead of assuming everything behind the firewall is safe, Zero Trust verifies each request as though it originates from an unsecure network ⬡ Principle of Least Privilege ⬠ Only privileges needed to complete a task should be allowed ⬠ Users should not have domain administrator/root privileges ⬡ Principle of Least Common Mechanism ⬠ Mechanisms used to access resources should not be shared in order to avoid the transmission of data. ⬠ Shared resources should not be used to access resources The Threat Model ⬡ A process by which potential threats can be identified and prioritized. ⬠ If you have a web server that feeds input to a mysql database, then protecting against mysql injections would be prioritized in your model. 2 considerations ⬡ *nix like is a very
    [Show full text]
  • Programming Project #2 Due: Thursday, May 13Th, 2004, 11:59 Pm
    CS155: Computer Security Spring 2004 Programming Project #2 Due: Thursday, May 13th, 2004, 11:59 pm 1 Introduction 1.1 Summary In this project, you will write Josh, the journaling operator shell. Josh is a setuid-root shell that allows users to undertake a subset of root’s capabilities. Your starting point is OSH, a minimal shell developed by Gunnar Ritter. You may work alone, or in pairs. You should not collaborate with others outside your group. You will use the Boxes system again. Remember to test your Josh in a closedbox. This as- signment specification is quite long, but much of it is intended to clarify points of confusion raised by previous years’ students. Please be sure to read it carefully before beginning to code. 1.2 The Problem As discussed in class, the classical Unix permissions model does not manage privilege delegation effectively. To perform routine administrative tasks (such as cleaning out the /tmp directory), one must possess the root password, and can therefore perform other, undesirable actions (such as reading users’ mail). This limitation manifests itself most acutely in large Unix installations, where there are typically many administrators tasked with various duties. Finer granularity of access is required for such systems, along with the ability to correctly log the important events for post-attack analysis. Various systems have been devised to overcome this limitation, and allow unprivileged users to un- dertake some subset of root’s capabilities. A commonly-used example is Sudo (http://www.sudo.ws/). We will take an approach more like that taken by OSH, the Operator Shell by Michael Neuman.
    [Show full text]
  • Processes and Job Control
    Processes and Job Control Hour 17 PObjectives < Definitions: process, orphan, and zombie < System processes < Process creation < Examining processes: the ps command < Job control: &, nohup, fg, bg, jobs, ( ), and kill < Exit status Copyright © 1998-2002 Delroy A. Brinkerhoff. All Rights Reserved. Hour 17 Unix Slide 1 of 12 Process Also called a job by C and Korn shells PWhen a program or executable file is loaded from disk and started running (i.e., when a command is run), it is called a process vi pid 641 < identified by a unique process ID (PID) number < has an owner vi < private data vi PA program can be loaded more than once pid 895 < creates multiple processes vi < each process has a different PID < each process may have a different owner PPIDs are unique, nonnegative integers < numbers recycle without collisions Hour 17 Unix Slide 2 of 12 System Processes Processes created during system boot P0System kernel < “hand crafted” at boot < called swap in older versions (swaps the CPU between processes) < called sched in newer versions (schedules processes) < creates process 1 P1 init (the parent of all processes except process 0) < general process spawner < begins building locale-related environment < sets or changes the system run-level P2 page daemon (pageout on most systems) P3 file system flusher (fsflush) Hour 17 Unix Slide 3 of 12 Process Life Cycle Overview of creating new processes fork init init pid 467 Pfork creates two identical pid 1 exec processes (parent and child) getty pid 467 Pexec < replaces the process’s instructions
    [Show full text]
  • Multiresolution Recurrent Neural Networks: an Application to Dialogue Response Generation
    Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generation Iulian Vlad Serban∗◦ Tim Klinger University of Montreal IBM Research 2920 chemin de la Tour, T. J. Watson Research Center, Montréal, QC, Canada Yorktown Heights, NY, USA Gerald Tesauro Kartik Talamadupula Bowen Zhou IBM Research IBM Research IBM Research T. J. Watson Research Center, T. J. Watson Research Center, T. J. Watson Research Center, Yorktown Heights, Yorktown Heights, Yorktown Heights, NY, USA NY, USA NY, USA Yoshua Bengioy◦ Aaron Courville◦ University of Montreal University of Montreal 2920 chemin de la Tour, 2920 chemin de la Tour, Montréal, QC, Canada Montréal, QC, Canada Abstract We introduce the multiresolution recurrent neural network, which extends the sequence-to-sequence framework to model natural language generation as two parallel discrete stochastic processes: a sequence of high-level coarse tokens, and a sequence of natural language tokens. There are many ways to estimate or learn the high-level coarse tokens, but we argue that a simple extraction procedure is sufficient to capture a wealth of high-level discourse semantics. Such procedure allows training the multiresolution recurrent neural network by maximizing the exact joint log-likelihood over both sequences. In contrast to the standard log- likelihood objective w.r.t. natural language tokens (word perplexity), optimizing the joint log-likelihood biases the model towards modeling high-level abstractions. We apply the proposed model to the task of dialogue response generation in arXiv:1606.00776v2 [cs.CL] 14 Jun 2016 two challenging domains: the Ubuntu technical support domain, and Twitter conversations. On Ubuntu, the model outperforms competing approaches by a substantial margin, achieving state-of-the-art results according to both automatic evaluation metrics and a human evaluation study.
    [Show full text]
  • 07 07 Unixintropart2 Lucio Week 3
    Unix Basics Command line tools Daniel Lucio Overview • Where to use it? • Command syntax • What are commands? • Where to get help? • Standard streams(stdin, stdout, stderr) • Pipelines (Power of combining commands) • Redirection • More Information Introduction to Unix Where to use it? • Login to a Unix system like ’kraken’ or any other NICS/ UT/XSEDE resource. • Download and boot from a Linux LiveCD either from a CD/DVD or USB drive. • http://www.puppylinux.com/ • http://www.knopper.net/knoppix/index-en.html • http://www.ubuntu.com/ Introduction to Unix Where to use it? • Install Cygwin: a collection of tools which provide a Linux look and feel environment for Windows. • http://cygwin.com/index.html • https://newton.utk.edu/bin/view/Main/Workshop0InstallingCygwin • Online terminal emulator • http://bellard.org/jslinux/ • http://cb.vu/ • http://simpleshell.com/ Introduction to Unix Command syntax $ command [<options>] [<file> | <argument> ...] Example: cp [-R [-H | -L | -P]] [-fi | -n] [-apvX] source_file target_file Introduction to Unix What are commands? • An executable program (date) • A command built into the shell itself (cd) • A shell program/function • An alias Introduction to Unix Bash commands (Linux) alias! crontab! false! if! mknod! ram! strace! unshar! apropos! csplit! fdformat! ifconfig! more! rcp! su! until! apt-get! cut! fdisk! ifdown! mount! read! sudo! uptime! aptitude! date! fg! ifup! mtools! readarray! sum! useradd! aspell! dc! fgrep! import! mtr! readonly! suspend! userdel! awk! dd! file! install! mv! reboot! symlink!
    [Show full text]
  • HP-UX Fundamentals
    Contact Us: (616) 875-4060 HP-UX Essentials Course Summary Length: 3 Days Prerequisite: Basic computer skills Recommendation Statement: Student should be able to use a computer monitor, keyboard, mouse. Student needs to be able to input commands and control characters from a keyboard. Course Description: For those new to HP-UX, this is the first course you will take. The course starts at the beginner level and works into intermediate and advanced HP-UX commands and utilities. The student is introduced to the HP-UX Korn shell, the command line and command syntax. Although you will be exposed to the various windowing platforms available in HP-UX, this course teaches UNIX at “the command line level” to develop a thorough understanding of the HP-UX operating environment. The student learns how to navigate the HP-UX file systems and how to work with files, directories and permissions. The student will learn how to use the HP-UX editors “ex” and “vi. “ Once the student is comfortable with basic HP-UX commands, they’ll move onto advanced power tools as well as how to customize the POSIX Korn shell. The classroom labs are run in a real-world UNIX environment, are structured to allow you to learn by doing, and are developed to simulate “real world” situations. The hands-on labs progressively build your HP-UX knowledge and command skills in a clear and concise manner. Working in a controlled UNIX classroom environment with an expert instructor, you will learn HP-UX concepts and commands. You will receive professional tips and techniques that will help you build your HP-UX confidence and skills.
    [Show full text]
  • Full Workload
    Command Line: Process Control C@ELOLJ<I GifZ\jjXe[afYZfekifc =LCCNFIBCF8; !Qn\`Ycld#g_fkfZXj\%Zfd What is happening on your Linux machine? Various shell commands All of these parameters can be com- bined – for example, pstree -apu. give you details about system processes and help you control them. C`jk`e^GifZ\jj\jn`k_gj BY HEIKE JURZIK The ps command gives you a list of the processes currently running on your sys- henever you or an application ing. All other processes share this com- tem. If you do not specify any command- launch a program, the operat- mon “ancestor” – init starts the operat- line parameters, the tool will restrict the Ning system starts one or multi- ing system’s basic programs. pstree list to the current shell. If you are inter- ple processes. These processes can enter shows the relationship between “parent” ested in all of your processes, call ps various states: They can be processed, and “child” processes. This tree struc- with the x option (Listing 1). stopped, restarted, and – under normal ture shows you at a glance who is de- The tabular output in the shell tells circumstances – stopped again. Linux scended from whom (Figure 1). you the following characteristics: has something similar to access controls ฀ PID: The process identifier, a unique for processes: Only the user that started 8[[`k`feXcFlkglk number that you can use to reference a process can stop, restart, or terminate The tool gives you more detailed output a process individually. the process. if you set the -a flag. This tells pstree to ฀ TTY: The terminal or console on The only exception to this rule is the show you, in addition, the parameters which the process was started.
    [Show full text]
  • The Linux Command Line
    The Linux Command Line Second Internet Edition William E. Shotts, Jr. A LinuxCommand.org Book Copyright ©2008-2013, William E. Shotts, Jr. This work is licensed under the Creative Commons Attribution-Noncommercial-No De- rivative Works 3.0 United States License. To view a copy of this license, visit the link above or send a letter to Creative Commons, 171 Second Street, Suite 300, San Fran- cisco, California, 94105, USA. Linux® is the registered trademark of Linus Torvalds. All other trademarks belong to their respective owners. This book is part of the LinuxCommand.org project, a site for Linux education and advo- cacy devoted to helping users of legacy operating systems migrate into the future. You may contact the LinuxCommand.org project at http://linuxcommand.org. This book is also available in printed form, published by No Starch Press and may be purchased wherever fine books are sold. No Starch Press also offers this book in elec- tronic formats for most popular e-readers: http://nostarch.com/tlcl.htm Release History Version Date Description 13.07 July 6, 2013 Second Internet Edition. 09.12 December 14, 2009 First Internet Edition. 09.11 November 19, 2009 Fourth draft with almost all reviewer feedback incorporated and edited through chapter 37. 09.10 October 3, 2009 Third draft with revised table formatting, partial application of reviewers feedback and edited through chapter 18. 09.08 August 12, 2009 Second draft incorporating the first editing pass. 09.07 July 18, 2009 Completed first draft. Table of Contents Introduction....................................................................................................xvi
    [Show full text]
  • Processes in Linux/Unix
    Processes in Linux/Unix A program/command when executed, a special instance is provided by the system to the process. This instance consists of all the services/resources that may be utilized by the process under execution. • Whenever a command is issued in unix/linux, it creates/starts a new process. For example, pwd when issued which is used to list the current directory location the user is in, a process starts. • Through a 5 digit ID number unix/linux keeps account of the processes, this number is call process id or pid. Each process in the system has a unique pid. • Used up pid’s can be used in again for a newer process since all the possible combinations are used. • At any point of time, no two processes with the same pid exist in the system because it is the pid that Unix uses to track each process. Initializing a process A process can be run in two ways: 1. Foreground Process : Every process when started runs in foreground by default, receives input from the keyboard and sends output to the screen. When issuing pwd command $ ls pwd Output: $ /home/geeksforgeeks/root When a command/process is running in the foreground and is taking a lot of time, no other processes can be run or started because the prompt would not be available until the program finishes processing and comes out. 2. Backround Process : It runs in the background without keyboard input and waits till keyboard input is required. Thus, other processes can be done in parallel with the process running in background since they do not have to wait for the previous process to be completed.
    [Show full text]
  • Trimming and Quality Control (2015-06-03)
    Trimming and quality control (2015-06-03) Alexander Jueterbock, Martin Jakt∗ PhD course: High throughput sequencing of non-model organisms Contents 1 Overview of sequence lengths2 2 Quality control3 3 Trimming low quality reads and adapters4 4 Fraction of duplicate reads7 5 BONUS Running programs in the background with nohup 7 5.1 Using the process-ID..................................8 5.2 Redirecting output................................... 10 After a general introduction to the UNIX command line, it is time for you to analyze your own fastq files. The first important step for any kind of sequencing data is to get rid of adapter contamination and bad quality reads. In this tutorial we will use the programs FastQC and TrimGalore! to check the quality of the sequenced libraries before and after trimming. We will also learn few more UNIX commands that extract important information from fastq files and that allow you to turn off your computer while the analysis continues to run on the remote server. IMPORTANT NOTE Before you get started: to compare characteristics of your libraries, please keep record of the resulting numbers, like the number of raw reads, reads after quality control, number of mapped reads etc. This helps to identify peculiarities/outliers in your libraries which may either be due to biological peculiarities of your species or unknown technical issues. Log on (with ssh) to the remote computer with the -X option to be able to use graphical interfaces. ∗University of Nordland, Norway 1 1 Overview of sequence lengths Next Generation Sequencing data is generally stored in fastq files. Most of the time the data are compressed, either in .zip or in .gz format.
    [Show full text]