DOCSLIB.ORG

Benchmark Recipe for GATK* Best Practices Pipeline Deployment

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Benchmark Recipe for GATK* Best Practices Pipeline Deployment BENCHMARK RECIPE Deploying GATK Best Practices Pipeline Below are the scripts files accompanying and as documented in the Infrastructure for Deploying GATK Best Practices Pipeline paper. ____________________________________________________________________________ GATK Best Practices Pipeline_README The following scripts are designed to take the same arguments to keep it consistent for running tests. For single threaded baseline analysis with no optimizations, run the data_colletion_gatk_best_practices_pl script with NumThreads as “1”. For both thread level and process level parallelism analysis, use the data_collection_gatk_best_practices_optimized.pl script with NumThreads as proposed in the paper. ____________________________________________________________________________________________ Data Collection Script for GATK Best Practices Pipeline (wgs_end2end_data_collection_gatk_best_practices.pl) #!/usr/bin/perl if (scalar(@ARGV) < 5) { die("Usage: SampleName NumThreads InputDataDirectory TempOutputDirectory profiling \n[if profiling is enabled, then the following is required]: collectstatspath interval stats \n"); } my $sample = $ARGV[0]; my $numThreads = $ARGV[1]; my $inDataDir = $ARGV[2]; my $tmpDir = "".$ARGV[3]; my $profiling = $ARGV[4]; #by default profiling is turned ON if invoked from the workflow profiler # arguments for collect_stats my $collectstatspath = $ARGV[5]; my $interval = $ARGV[6]; # by default sampling interval is 30s from the workflow profiler. my $stats = $ARGV[7]; #my $numLanes =$ARGV[1]; my $called = "$0 @ARGV"; my $numLanes = 0; my $sampleprefix = $sample.'_'.$numThreads.'T'; 1 BENCHMARK RECIPE Data Collection Script for GATK Best Practices Pipeline (cont) # INPUT FASTQ FILES #OTHER FORMATS FOR FQ: "_1.fastq.gz; #"_1.fastq"; my $fqFile1, $fqFile2; $fqFile1 = $inDataDir.$sample."_1.fq"; $fqFile2 = $inDataDir.$sample."_2.fq"; # Pipeline executables and its directories ### SPECIFY PATH IN THE VARIABLES BELOW ### my $toolsDir = '/PATH/TO/TOOLS_DIR'; my $homosapiensrefgenomeDir = '/PATH/TO/REF'; # TOOLS my $bwaDir = "$toolsDir/bwa"; my $bwa = "$bwaDir/bwa"; my $gatkDir = "$toolsDir/gatk-protected/target"; my $gatk = "$gatkDir/GenomeAnalysisTK.jar"; my $picardDir ="$toolsDir/picard/dist"; my $picard = "$picardDir/picard.jar"; # HOMOSAPIENSREFGENOME my $refgenomeFastaFile = "$homosapiensrefgenomeDir/Homo_sapiens_assembly19.fasta"; my $refgenomeBwtFile = "$homosapiensrefgenomeDir/Homo_sapiens_assembly19.fasta.bwt"; my $dbSNPvcf = "$homosapiensrefgenomeDir/Homo_sapiens_assembly19.dbsnp.vcf"; my $dbSNPindel = "$homosapiensrefgenomeDir/Homo_sapiens_assembly19.known_indels.vcf"; # EXOME TARGET INTERVALS my $exome_targets_intervals = "$homosapiensrefgenomeDir/nexterarapidcapture_exome_uniqueintervals.bed"; unless(-d $inDataDir) { die("Error: The InputDataDirectory $inDataDir doesn't exist\n"); } unless(-d $tmpDir) { die("Error: The TempOutputDirectory $tmpDir doesn't exist\n"); } # Output file names for each stage of the pipeline my $baseName = $sample; my $baseNameLane = $baseName.'_'.$numLanes.'L_'.$numThreads.'T'; my $bwamem_samFile = $tmpDir.$baseNameLane.".sam"; my $sort_bamFile = $tmpDir.$baseName."_sorted.bam"; my $duplicateMetricsFile = $tmpDir.$baseName."_dup.metrics"; my $bamDupRemFile = $tmpDir.$baseName."_dupRem.bam"; my $bamRealignFile = $tmpDir.$baseName."_realign.bam"; my $realnInterval = $tmpDir.$baseName."_realn.intervals"; my $finalBam = $tmpDir.$baseName."_final.bam"; my $HCvcf = $tmpDir.$baseName."_HaplotypeCaller.vcf"; my $genomeImportFile = $refgenomeFastaFile.".fai"; #ADD the relevant platform my $readGroupHeader = "\@RG\\tID:$baseNameLane\\tLB:$baseName\\tSM:$baseName\\tPL:PLATFORM"; my $recalOut = $tmpDir .$baseName."_recal.grp"; my $dryRun = 0; my $pwd = `pwd`; chomp $pwd; my $host = `hostname`; chomp $host; my $uname = `whoami`; chomp $uname; my $runningTime = time; my $commandsfile = $tmpDir.$uname."_".$sampleprefix."_processing.log"; my $outputfile = $tmpDir.$uname."_".$sampleprefix."_output.log"; open(LOG,">$commandsfile"); 2 BENCHMARK RECIPE Data Collection Script for GATK Best Practices Pipeline (cont) print LOG "#$called (version $version) in $pwd on $host.\n"; print LOG "#Started at ".`date +"%F %T"`."\n"; print LOG "#temporary files created in $tmpDir\n"; my $procTime = time; my $procFlag = 0; sub run_and_log { my $command = $_[0]; my $execute = !$dryRun; my $exitValue = 0; #a command we don't run is considered successful my $redirect; if (@_ >1){ $execute=!$_[1]; } #several of the programs like to output to STDERR, so we link that to log file $redirect = "1>>$outputfile 2>&1"; #that is because we redirect STDOUT in many cases, so let's not mess with it. $redirect = "2>>$outputfile" if $command =~ m/>/; # $redirect = "" if $command =~ m/>/; $command = $command." ".$redirect; if ($procFlag == 0){ $procFlag++; } else { $procTime = time - $procTime; printf LOG "#Processing Time %02d:%02d:%02d\n",int($procTime /3600),int(($procTime % 3600) /60),int($procTime %60); $procTime = time; } print LOG "#not run\n" if !$execute; print LOG "#".`date +"%F %T"`; print LOG $command."\n"; $exitValue = system($command) if $execute; #necessary if we use `` instead of system() #$exitValue = $? >>8; ##If the command failed, we want to stop it here. if ($exitValue != 0){ my $error = "Command failed with return value $exitValue : $command \n"; print LOG $error; close LOG; die $error; } } sub Start_profiling { my ($tag) = @_; if ($profiling) { system("$collectstatspath $stats -d $interval -td $tmpDir -n $sampleprefix -tag $tag -l 5 -u 1 -s 600 &"); } } sub Stop_Profiling { 3 BENCHMARK RECIPE Data Collection Script for GATK Best Practices Pipeline (cont) if ($profiling) { system("$collectstatspath --kill-all"); } } my $stage_tag=BwaMem; Start_profiling($stage_tag); print "$stage_tag\n"; run_and_log "$bwa mem -t $numThreads -Ma -R \'$readGroupHeader\' $refgenomeFastaFile $fqFile1 $fqFile2 > $bwamem_samFile"; Stop_Profiling(); sleep(60); my $stage_tag=SortSam; Start_profiling($stage_tag); print "$stage_tag\n"; run_and_log "java -Xmx8g -jar $picard SortSam I=$bwamem_samFile O=$sort_bamFile SO=coordinate CREATE_INDEX=true"; Stop_Profiling(); sleep(60); my $stage_tag=MarkDuplicates; Start_profiling($stage_tag); print "$stage_tag\n"; run_and_log "java -Xmx8g -jar $picard MarkDuplicates I=$sort_bamFile O=$bamDupRemFile M=$duplicateMetricsFile CREATE_INDEX=true TMP_DIR=$tmpDir"; Stop_Profiling(); sleep(60); my $stage_tag=RealignerTargetCreator; Start_profiling($stage_tag); print "$stage_tag\n"; run_and_log "java -Xmx8g -jar $gatk -T RealignerTargetCreator -nt $numThreads -R $refgenomeFastaFile -o $realnInterval -known:indels,vcf $dbSNPindel -I $bamDupRemFile"; Stop_Profiling(); sleep(60); my $stage_tag=IndelRealigner; Start_profiling($stage_tag); print "$stage_tag\n"; run_and_log "java -Xmx8g -Djava.io.tmpdir=$tmpDir -jar $gatk -T IndelRealigner -R $refgenomeFastaFile - targetIntervals $realnInterval -known:indels,vcf $dbSNPindel -I $bamDupRemFile -o $bamRealignFile -- filter_bases_not_stored"; Stop_Profiling(); sleep(60); my $stage_tag=BaseRecalibrator; Start_profiling($stage_tag); print "$stage_tag\n"; run_and_log "java -Xmx4g -jar $gatk -T BaseRecalibrator -I $bamRealignFile -R $refgenomeFastaFile - knownSites:mask,vcf $dbSNPvcf -o $recalOut"; Stop_Profiling(); sleep(60); 4 BENCHMARK RECIPE Stages of GATK Best Practices Pipeline (cont) my $stage_tag=PrintReads; Start_profiling($stage_tag); print "$stage_tag\n"; run_and_log "java -Xmx8g -jar $gatk -T PrintReads -R $refgenomeFastaFile -I $bamRealignFile -BQSR $recalOut -o $finalBam"; Stop_Profiling(); sleep(60); my $stage_tag=HaplotypeCaller; Start_profiling($stage_tag); print "$stage_tag\n"; run_and_log "java -Xmx8g -jar $gatk -T HaplotypeCaller -R $refgenomeFastaFile -I $finalBam -o $HCvcf -ERC GVCF --variant_index_type LINEAR --variant_index_parameter 128000"; Stop_Profiling(); sleep(60); $runningTime = time - $runningTime; printf LOG "#done in %02d:%02d:%02d\n",int($runningTime /3600),int(($runningTime % 3600) /60),int($runningTime %60); exit 0; Data Collection Script for GATK Best Practices Pipeline Optimized (wgs_end2end_data_collection_gatk_best_practices_optimized.pl) #!/usr/bin/perl if (scalar(@ARGV) < 5) { die("Usage: SampleName NumThreads InputDataDirectory TempOutputDirectory profiling \n[if profiling is enabled, then the following is required]: collectstatspath interval stats \n"); } my $sample = $ARGV[0]; my $numThreads = $ARGV[1]; my $inDataDir = $ARGV[2]; my $tmpDir = "".$ARGV[3]; my $profiling = $ARGV[4]; #by default profiling is turned ON if invoked from the workflow profiler # arguments for collect_stats my $collectstatspath = $ARGV[5]; my $interval = $ARGV[6]; # by default sampling interval is 30s from the workflow profiler. my $stats = $ARGV[7]; #my $numLanes =$ARGV[1]; my $called = "$0 @ARGV"; my $numLanes = 0; my $sampleprefix = $sample.'_'.$numThreads.'T'; 5 BENCHMARK RECIPE Data Collection Script for GATK Best Practices Pipeline Optimized (cont) # INPUT FASTQ FILES #OTHER FORMATS FOR FQ: "_1.fastq.gz; #"_1.fastq"; my $fqFile1, $fqFile2; $fqFile1 = $inDataDir.$sample."_1.fq"; $fqFile2 = $inDataDir.$sample."_2.fq"; # Pipeline executables and its directories ### SPECIFY PATH IN THE VARIABLES BELOW ### my $toolsDir = '/PATH/TO/TOOLS_DIR'; #SCALA_SCRIPTS: Scripts to be passed to Queue.jar my $QueueBroadBestPracticesDir = '/PATH/TO/SCALA_SCRIPTS'; my $homosapiensrefgenomeDir = '/PATH/TO/REF'; # TOOLS my $bwaDir = "$toolsDir/bwa"; my $bwa = "$bwaDir/bwa"; my $gatkDir = "$toolsDir/gatk-protected/target"; my $gatk = "$gatkDir/GenomeAnalysisTK.jar";
Load more

Recommended publications
  • At—At, Batch—Execute Commands at a Later Time
    at—at, batch—execute commands at a later time at [–csm] [–f script] [–qqueue] time [date] [+ increment] at –l [ job...] at –r job... batch at and batch read commands from standard input to be executed at a later time. at allows you to specify when the commands should be executed, while jobs queued with batch will execute when system load level permits. Executes commands read from stdin or a file at some later time. Unless redirected, the output is mailed to the user. Example A.1 1 at 6:30am Dec 12 < program 2 at noon tomorrow < program 3 at 1945 pm August 9 < program 4 at now + 3 hours < program 5 at 8:30am Jan 4 < program 6 at -r 83883555320.a EXPLANATION 1. At 6:30 in the morning on December 12th, start the job. 2. At noon tomorrow start the job. 3. At 7:45 in the evening on August 9th, start the job. 4. In three hours start the job. 5. At 8:30 in the morning of January 4th, start the job. 6. Removes previously scheduled job 83883555320.a. awk—pattern scanning and processing language awk [ –fprogram–file ] [ –Fc ] [ prog ] [ parameters ] [ filename...] awk scans each input filename for lines that match any of a set of patterns specified in prog. Example A.2 1 awk '{print $1, $2}' file 2 awk '/John/{print $3, $4}' file 3 awk -F: '{print $3}' /etc/passwd 4 date | awk '{print $6}' EXPLANATION 1. Prints the first two fields of file where fields are separated by whitespace. 2. Prints fields 3 and 4 if the pattern John is found.
    [Show full text]
  • Unix/Linux Command Reference
    Unix/Linux Command Reference .com File Commands System Info ls – directory listing date – show the current date and time ls -al – formatted listing with hidden files cal – show this month's calendar cd dir - change directory to dir uptime – show current uptime cd – change to home w – display who is online pwd – show current directory whoami – who you are logged in as mkdir dir – create a directory dir finger user – display information about user rm file – delete file uname -a – show kernel information rm -r dir – delete directory dir cat /proc/cpuinfo – cpu information rm -f file – force remove file cat /proc/meminfo – memory information rm -rf dir – force remove directory dir * man command – show the manual for command cp file1 file2 – copy file1 to file2 df – show disk usage cp -r dir1 dir2 – copy dir1 to dir2; create dir2 if it du – show directory space usage doesn't exist free – show memory and swap usage mv file1 file2 – rename or move file1 to file2 whereis app – show possible locations of app if file2 is an existing directory, moves file1 into which app – show which app will be run by default directory file2 ln -s file link – create symbolic link link to file Compression touch file – create or update file tar cf file.tar files – create a tar named cat > file – places standard input into file file.tar containing files more file – output the contents of file tar xf file.tar – extract the files from file.tar head file – output the first 10 lines of file tar czf file.tar.gz files – create a tar with tail file – output the last 10 lines
    [Show full text]
  • Unix/Linux Command Reference
    Unix/Linux Command Reference .com File Commands System Info ls – directory listing date – show the current date and time ls -al – formatted listing with hidden files cal – show this month's calendar cd dir - change directory to dir uptime – show current uptime cd – change to home w – display who is online pwd – show current directory whoami – who you are logged in as mkdir dir – create a directory dir finger user – display information about user rm file – delete file uname -a – show kernel information rm -r dir – delete directory dir cat /proc/cpuinfo – cpu information rm -f file – force remove file cat /proc/meminfo – memory information rm -rf dir – force remove directory dir * man command – show the manual for command cp file1 file2 – copy file1 to file2 df – show disk usage cp -r dir1 dir2 – copy dir1 to dir2; create dir2 if it du – show directory space usage doesn't exist free – show memory and swap usage mv file1 file2 – rename or move file1 to file2 whereis app – show possible locations of app if file2 is an existing directory, moves file1 into which app – show which app will be run by default directory file2 ln -s file link – create symbolic link link to file Compression touch file – create or update file tar cf file.tar files – create a tar named cat > file – places standard input into file file.tar containing files more file – output the contents of file tar xf file.tar – extract the files from file.tar head file – output the first 10 lines of file tar czf file.tar.gz files – create a tar with tail file – output the last 10 lines
    [Show full text]
  • Linux Shell Script to Find Kernel Version from Multiple Servers Linux Shell Script That Will Login to Server Via Ssh and Get Kernel Version Via Uname -R Command
    Linux Shell Script To Find Kernel Version From Multiple Servers Linux shell script that will login to server via ssh and get kernel version via uname -r command. List of all the servers are in server_list.txt file, script will take name by name and try to ssh with the root user and run uname -r command. Output will be written to variable and after into file with servername. This script helps system administrator who manage large linux servers , to take reports of what linux versions running. Here I have used uname -r but you can edit and use your own command as per your requirements. Source : http://linoxide.com/linux-shell-script/kernel-version-multiple/ Linux Shell Script #!/bin/bash #we user variable serverlist to keep there path to file with server names serverlist=’server_list.txt’ #we write in variable all server list servers=`cat $serverlist` #we use variable result to keep there path to file with result result=’result.txt’ #this print header to file with resilt using \t\t to add 2 tab symbols echo -e “Servername \t\t kernel version”> $result #this get each line of serverlist one by one and write to server variable for server in $servers do #this login to server by ssh and get uname -r kernel=`ssh root@${server} “uname -r”` #this write server name and kernel version separated by 2 tab to result file echo -e “$server \t\t $kernel” >> $result #end of for loop. done server_list.txt file # cat server_list.txt dev Linux Shell Script To Find Kernel Version From Multiple Servers 1 www.linoxide.com web1 svn Shell Script Output ./kernel_version.sh centos_node1@bobbin:~/Documents/Work/Bobbin$ cat result.txt Servername kernel version dev 3.3.8-gentoo web1 3.2.12-gentoo svn 3.2.12-gentoo Linux Shell Script To Find Kernel Version From Multiple Servers 2 www.linoxide.com.
    [Show full text]
  • Linux Networking Cookbook.Pdf
    Linux Networking Cookbook ™ Carla Schroder Beijing • Cambridge • Farnham • Köln • Paris • Sebastopol • Taipei • Tokyo Linux Networking Cookbook™ by Carla Schroder Copyright © 2008 O’Reilly Media, Inc. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (safari.oreilly.com). For more information, contact our corporate/institutional sales department: (800) 998-9938 or [email protected]. Editor: Mike Loukides Indexer: John Bickelhaupt Production Editor: Sumita Mukherji Cover Designer: Karen Montgomery Copyeditor: Derek Di Matteo Interior Designer: David Futato Proofreader: Sumita Mukherji Illustrator: Jessamyn Read Printing History: November 2007: First Edition. Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of O’Reilly Media, Inc. The Cookbook series designations, Linux Networking Cookbook, the image of a female blacksmith, and related trade dress are trademarks of O’Reilly Media, Inc. Java™ is a trademark of Sun Microsystems, Inc. .NET is a registered trademark of Microsoft Corporation. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and O’Reilly Media, Inc. was aware of a trademark claim, the designations have been printed in caps or initial caps. While every precaution has been taken in the preparation of this book, the publisher and author assume no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein.
    [Show full text]
  • Linux Networking 101
    The Gorilla ® Guide to… Linux Networking 101 Inside this Guide: • Discover how Linux continues its march toward world domination • Learn basic Linux administration tips • See how easy it can be to build your entire network on a Linux foundation • Find out how Cumulus Linux is your ticket to networking freedom David M. Davis ActualTech Media Helping You Navigate The Technology Jungle! In Partnership With www.actualtechmedia.com The Gorilla Guide To… Linux Networking 101 Author David M. Davis, ActualTech Media Editors Hilary Kirchner, Dream Write Creative, LLC Christina Guthrie, Guthrie Writing & Editorial, LLC Madison Emery, Cumulus Networks Layout and Design Scott D. Lowe, ActualTech Media Copyright © 2017 by ActualTech Media. All rights reserved. No portion of this book may be reproduced or used in any manner without the express written permission of the publisher except for the use of brief quotations. The information provided within this eBook is for general informational purposes only. While we try to keep the information up- to-date and correct, there are no representations or warranties, express or implied, about the completeness, accuracy, reliability, suitability or availability with respect to the information, products, services, or related graphics contained in this book for any purpose. Any use of this information is at your own risk. ActualTech Media Okatie Village Ste 103-157 Bluffton, SC 29909 www.actualtechmedia.com Entering the Jungle Introduction: Six Reasons You Need to Learn Linux ....................................................... 7 1. Linux is the future ........................................................................ 9 2. Linux is on everything .................................................................. 9 3. Linux is adaptable ....................................................................... 10 4. Linux has a strong community and ecosystem ........................... 10 5.
    [Show full text]
  • Linux Command Line Cheat Sheet by Davechild
    Linux Command Line Cheat Sheet by DaveChild Bash Commands ls Options Nano Shortcuts uname -a Show system and kernel -a Show all (including hidden) Files head -n1 /etc/issue Show distribution -R Recursive list Ctrl-R Read file mount Show mounted filesystems -r Reverse order Ctrl-O Save file date Show system date -t Sort by last modified Ctrl-X Close file uptime Show uptime -S Sort by file size Cut and Paste whoami Show your username -l Long listing format ALT-A Start marking text man command Show manual for command -1 One file per line CTRL-K Cut marked text or line -m Comma-separated output CTRL-U Paste text Bash Shortcuts -Q Quoted output Navigate File CTRL-c Stop current command ALT-/ End of file Search Files CTRL-z Sleep program CTRL-A Beginning of line CTRL-a Go to start of line grep pattern Search for pattern in files CTRL-E End of line files CTRL-e Go to end of line CTRL-C Show line number grep -i Case insensitive search CTRL-u Cut from start of line CTRL-_ Go to line number grep -r Recursive search CTRL-k Cut to end of line Search File grep -v Inverted search CTRL-r Search history CTRL-W Find find /dir/ - Find files starting with name in dir !! Repeat last command ALT-W Find next name name* !abc Run last command starting with abc CTRL-\ Search and replace find /dir/ -user Find files owned by name in dir !abc:p Print last command starting with abc name More nano info at: !$ Last argument of previous command find /dir/ - Find files modifed less than num http://www.nano-editor.org/docs.php !* All arguments of previous command mmin num minutes ago in dir Screen Shortcuts ^abc^123 Run previous command, replacing abc whereis Find binary / source / manual for with 123 command command screen Start a screen session.
    [Show full text]
  • Cyber502x Computer Forensics
    CYBER502x Computer Forensics Unit 2: Linux/Unix Forensics Acquisition CYBER 502x Computer Forensics | Yin Pan Investigating Linux/Unix systems • Four basic forensics steps • Collect • Preserve • Analyze • Present (report) CYBER 502x Computer Forensics | Yin Pan Preparation • A tool box (CD or USB) containing trusted forensic tools • A powerful machine with forensic tools installed and clean-wiped hard drive to store acquired evidence. CYBER 502x Computer Forensics | Yin Pan Remember! • Always have your OWN tool sets ready !! • You are dealing with a compromised system • Run tools from your own USB or device • Save the output outside of the compromised system CYBER 502x Computer Forensics | Yin Pan Forensics tools in common • Ensure forensically-sound operations • Process data structure from the image bypassing kernel’s support • Work on both images and live systems CYBER 502x Computer Forensics | Yin Pan Basic imaging steps • Obtain volatile data (including RAM) –According to policy and the case nature • Acquire non-volatile data (image drives and removable media) CYBER 502x Computer Forensics | Yin Pan Acquire volatile information • System information • Memory usage • Running processes • Logged in users • Network connections • Network interface configuration (promiscuous mode?) • …. CYBER 502x Computer Forensics | Yin Pan Volatile Evidence • Most volatile Least volatile Nonvolatile • Memory • Swap space or gagefile • Network status and connections • Processes running • File opening • Hard drive media • Removable media (CD, Zip, USB,
    [Show full text]
  • Unix Commands (09/04/2014)
    Unix Commands (09/04/2014) • Access control – login <login_name> – exit – passwd <login_name> – yppassswd <loginname> – su – • Login as Super user – su <login> • Login as user <login> • Root Prompt – [root@localhost ~] # • User Prompt – [bms@raxama ~] $ On Line Documentation – man <command/topic> – info <command/topic> • Working with directories – mkdir –p <subdir> ... {-p create all directories in path if not present} mkdir –p /2015/Jan/21/14 will create /2015, Jan, 21 & 14 in case any of these is absent – cd <dir> – rm -r <subdir> ... Man Pages • 1 Executable programs or shell commands • 2 System calls (functions provided by the kernel) • 3 Library calls (functions within program libraries) • 4 Special files (usually found in /dev) • 5 File formats and conventions eg /etc/passwd • 6 Games • 7 Miscellaneous (including macro packages and conventions), e.g. man(7), groff(7) • 8 System administration commands (usually only for root) • 9 Kernel routines [Non standard] – man grep, {awk,sed,find,cut,sort} – man –k mysql, man –k dhcp – man crontab ,man 5 crontab – man printf, man 3 printf – man read, man 2 read – man info Runlevels used by Fedora/RHS Refer /etc/inittab • 0 - halt (Do NOT set initdefault to this) • 1 - Single user mode • 2 - Multiuser, – without NFS (The same as 3, if you do not have networking) • 3 - Full multi user mode w/o X • 4 - unused • 5 - X11 • 6 - reboot (Do NOT set init default to this) – init 6 {Reboot System} – init 0 {Halt the System} – reboot {Requires Super User} – <ctrl> <alt> <del> • in tty[2-7] mode – tty switching • <ctrl> <alt> <F1-7> • In Fedora 10 tty1 is X.
    [Show full text]
  • In This Report I Will Mention the Steps to Write Simple Kernel Module in Linux Environment
    In this report I will mention the steps to write simple kernel module in Linux environment Structure of the module :­ init_module() { // code that initializes the kernel module data structures //and may call some helping functions //entry function of the module //This functions is called when the module is loaded into kernel using “insmod” //insmod stands for install kernel module } cleanup_module() { //code that frees all the module data structures etc //exit function of the module //This functions is called when the module is unloaded from kernel using “rmmod” // rmmod stands for remove module i.e unload the loadable module } some other helping functions() Structure of makefile :­ Say the module is written in file named as hello.c obj­m+=hello.o all: make ­c /lib/modules/$(shell uname ­r)/build M=$(pwd) modules clean: make ­c /lib/module/$(shell uname ­r)/build M=$(pwd) clean Note:­ please make a note of tab before make command Command to build an external module:­ make ­c <path­to­kernel> M=’pwd’ Command to build for running kernel use:­ make ­c /lib/modules/’uname ­r’/build M=’pwd’ uname command gives the underlined machine kernel version when passed with ­r option M is used to tell kbulid that an external module is being built To install modules that were just built:­ make ­c <path­to­kernel> M=’pwd’ modules­install Steps:­ 1) write the module with exit and entry functions of the module 2) create a makefile as shown in the above example 3) execute makefile 4)use insmod to load the module into kernel sudo insmod ./hello.ko 5) we can see the loaded module in /proc/modules file 6)we can view the logs created by module in /var/log/messages a) If syslod is not installed we can't view the messages use command sudo apt­get install inetuils­syslogd to install it 7) we can remove the module by using command sudo rmmod ./hello.ko The entry and exit functions is supposed to have standard name as shown in the above example before kernel 2.6 version.
    [Show full text]
  • Gnu Coreutils Core GNU Utilities for Version 5.93, 2 November 2005
    gnu Coreutils Core GNU utilities for version 5.93, 2 November 2005 David MacKenzie et al. This manual documents version 5.93 of the gnu core utilities, including the standard pro- grams for text and file manipulation. Copyright c 1994, 1995, 1996, 2000, 2001, 2002, 2003, 2004, 2005 Free Software Foundation, Inc. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the section entitled “GNU Free Documentation License”. Chapter 1: Introduction 1 1 Introduction This manual is a work in progress: many sections make no attempt to explain basic concepts in a way suitable for novices. Thus, if you are interested, please get involved in improving this manual. The entire gnu community will benefit. The gnu utilities documented here are mostly compatible with the POSIX standard. Please report bugs to [email protected]. Remember to include the version number, machine architecture, input files, and any other information needed to reproduce the bug: your input, what you expected, what you got, and why it is wrong. Diffs are welcome, but please include a description of the problem as well, since this is sometimes difficult to infer. See section “Bugs” in Using and Porting GNU CC. This manual was originally derived from the Unix man pages in the distributions, which were written by David MacKenzie and updated by Jim Meyering.
    [Show full text]
  • Oracle Ksplice for Oracle Linux
    Oracle Ksplice for Oracle Linux Oracle Corporation Oracle Corporation Oracle Ksplice Oracle Ksplice is an exciting new addition to the Oracle Linux Premier Support subscription. The Oracle Ksplice technology allows customers to update their systems with new kernel security errata (CVEs) without the need to reboot the system. This allows customers to remain current with their OS vulnerability patches while at the same time minimizing downtime. Oracle Ksplice actually actively applies these updates to the running kernel image, not just an on-disk change that would only take effect after a subsequent reboot. As a general rule, the Linux kernel receives security updates approximately once a month. For any general purpose Operating System on the market today, applying these updates requires the system administrator to schedule downtime and reboot the server into the new OS kernel with the security updates. As system setups become more and more complex with multi-system interdependencies, the ability to schedule reboots is becoming more and more difficult and costly. Oracle Ksplice is available as part of the Oracle Linux Premier and Premier Limited support subscriptions. It is also part of the Oracle Premier Support for Systems and Operating Systems subscriptions offering. Oracle Linux Basic, Basic Limited and Network Support subscribers can contact their sales representatives to discuss potential upgrade of their subscription to a Premier Support plan. Another requirement for the Oracle Ksplice updates, is the use of the Oracle Unbreakable Enterprise Kernel(UEK). The lowest Linux kernel version at time of writing is 2.6.32-100.28.9. This kernel (and newer) can be installed on both Oracle Linux(OL) 5 and 6 distribution versions.
    [Show full text]