Linux Guide Contents

LINUX GUIDE CONTENTS

1. Introduction-of-Linux 2. Is Linux hard to learn ? 3. Diference between Unix and Linux ? 4. Features of Linux

5. Command line interface

6. Advantage or Disadvantage

7. Connect to Linux

8. Hardware information

9. Terminal shortcut

10. Basic command

11. Vim Editor

12. How to edit text documents using vi editor

13. User and group

14. File system

15. Disk Partitions

16. Formatting

17. Mounting a partition

18. Directory structure

19. File Type

20. File permissions

21. Files processing

22. Cut

23. Paste

24. Copy

25. Jion

26. Grep

28. Awk

29. Find

30. Locate

31. Archives

32. RegEx & Wildcards

33. Network confguration

34. Troubleshooting

35. Services

36. Package management

37. Yum & Rpm

38. Remote connection

39. FTP,NFS,SAMBA

Contributors Abhinay Kumar, Gulam Rabbany(MrGr33n)

The architecture of Linux can be divided into four levels of functionality as shown in below fgure.

Hardware - It is a combination of all peripherals associated with the system. For example, RAM, CPU,Hard disk etc...

Kernel - It is the core part of the operating system and manages the CPU, memory, and peripheral devices. The kernel is the "lowest" level of the OS. Linux kernel is written in the version of the C programming language. It is responsible for all major activities of this operating system.It manages the communication between devices and software, manages the system resources (like CPU time, memory, network...) and shields of the complexity of device programming from the developer as it provides an interface for the programmer to manipulate hardware.

Shell - Shell acts as a user interface, interpreting user commands and starting an application. Users typically interact with a Linux shell using a terminal emulator, however, direct operation via serial hardware connections, or networking sessions, are common for server systems.

System utilities - The system tools are built using the system libraries and enable administrators to administer the system, manage processes, navigate on the fle system, execute other applications,confgure the network etc..

The Linux OS is a collection of a number of components

The Boot-loader - It is a program that loads the Linux kernel into the computer's main memory, by being executed by the computer when it is turned on and after the frmware initialization is performed.

The kernel - It is the core part of the operating system and manages the CPU, memory, and peripheral devices.

Daemons - These are background services that either startup during boot, or after you log into the desktop.

The Shell - Shell is a command processor that allows you to control the computer via commands typed into a text interface.

System Libraries - System libraries are special programs that help in accessing the kernel's features. Programmers have developed a standard library of procedures to communicate with the kernel. Each operating system supports these standards and then these are transferred to system calls for that operating system.

Graphical Server - This is the sub-system that displays the graphics on your monitor. It is commonly referred to as the X server or just "X".

© CEHPOINT E-LEARNING 2020 ( 3 ) System Tools- Linux OS has a set of utility tools which are usually simple commands. It is a software which GNU projects has written and published under their open source license so that software is freely available to everyone.

What is Linux ?

Like Windows and Mac OS, Linux is the best known and most used open source operating system. As an operating system, Linux manages the communication between your software and your hardware. Without the operating system, the software wouldn't function.Linux is a multi-tasking and multi-user operating system. Linux was originally developed for personal computers but due to its some fantastic features and robustness, it is used in with various hardware. Currently Linux is a very popular operating system for servers. Linux is the leading operating system on servers and mainframe computers, and is used on 99.6% of the TOP 500 super computers. Some mobile operating systems like Android are also built on Linux kernel.As Linux is an opensource version of Unix, Linux kernel code is freely available. You can download the Linux kernel code and modify it as per your business requirement and create your own operating system. Because of the dominance of Android on smartphones, Linux has the largest installed base of all general purpose operating systems.

Diference between Unix and Linux ?

Unix is an operating system developed in the 1970s at Bell Labs by Ken Thompson, Dennis Ritchie, and others. Unix and Linux are similar in many ways, and in fact, Linux was originally created to be similar to Unix. Both have similar tools for interfacing with the systems, programming tools, flesystem layouts, and other key components. However, Unix is not free. Over the years, a number of diferent operating systems have been created that attempted to be unix-like or unix-compatible, but Linux has been the most successful, far surpassing its predecessors in popularity.

Is Linux hard to learn ?

If you will take Linux as general operating system software then Linux is even easier than other operating systems. Linux is not only a powerful OS but it is easy to use and has lots of fexibility for user.Current there are lots of GUI (Graphics user interface) available in Linux. But still the command line interface of Linux is great. You can solve your all problems using a simple command line.Once you will start using Linux you will realize the biggest power of Linux as easy to use.

Diference between Unix and Linux ?

Difference between Unix and Linux ?

Unix is an operating system developed in the 1970s at Bell Labs by Ken Thompson, Dennis Ritchie, and others. Unix and Linux are similar in many ways, and in fact, Linux was originally created to be similar to Unix. Both have similar tools for interfacing with the systems, programming tools, flesystem layouts, and other key components. However, Unix is not free. Over the years, a number of different operating systems have been created that attempted to be unix-like or unix-compatible, but Linux has been the most successful, far surpassing its predecessors in popularity.

Features of Linux

Multi-User :Multiple users can access system resources at the same time.

Multi tasking :Multiple jobs can run at the same time.

Multiprogramming :Linux is a multiprogramming system meaning multiple applications can run at same time.

Open Source :Linux is an open source operating system, a freely distributed, cross-platform operating system based on Unix.

Portable :All programs can run on different systems in the same functionality.

Communication :Linux provides Electronic mail. The communication may be within the network of a single computer, or between two or more such computer networks.

Hierarchical File System :Linux has the hierarchical fle system. start from / (root) directory.

Shell :It provides an interface to the users to run their programs.

Security :Linux provides high security. users and passwords are strictly managed. In Linux no need for other anti-malware applications because itself provides high security.

Help Facility :Linux provides manual pages for LINUX commands.

Widely used Linux distributionsDebian : It is a non-commercial distribution and one of the earliest, maintained by a volunteer developer community with a strong commitment to free software principles and democratic project management.

Knoppix: It is the frst Live CD distribution derived from Debian to run completely from removable media without installation to a hard disk.

Linux Mint Debian Edition (LMDE): It uses Debian packages directly.

Ubuntu: It is a desktop and server distribution derived from Debian, maintained by British company Canonical Ltd.Kubuntu, the KDE version of Ubuntu.Linux Mint, a distribution based on and compatible with Ubuntu. Supports multiple desktop environments.Trisquel, an Ubuntu-based distribution based on Linux-libre kernel composed entirely of free software.Elementary OS, an Ubuntu-based distribution with a strong focus on the visual experience without sacrifcing performance.

Fedora :It is a commercial distribution sponsored by American company Red Hat. It aims to be a technology testbed for Red Hat's commercial Linux offering, where new open source software is prototyped, developed, and tested in a communal setting before maturing into Red Hat Enterprise Linux.Red Hat Enterprise Linux (RHEL): It is a derivative of Fedora, maintained and commercially

© CEHPOINT E-LEARNING 2020 ( 7 ) supported by Red Hat. It seeks to provide tested, secure, and stable Linux server and workstation support to businesses.CentOS, a distribution derived from the same sources used by Red Hat, maintained by a dedicated volunteer community of developers with both 100% Red Hat-compatible versions and an upgraded version that is not always 100% upstream compatible.Oracle Linux, which is a derivative of Red Hat Enterprise Linux, maintained and commercially supported by Oracle.Scientifc Linux, a distribution derived from the same sources used by Red Hat, maintained by Fermilab.

Mandriva Linux :It was a Red Hat derivative popular in several European countries and Brazil, backed by the French company of the same name. After the company went bankrupt, it was superseded by OpenMandriva Lx, although a number of derivatives now have a larger user base.Mageia, a community fork of Mandriva Linux created in 2010.PCLinuxOS, a derivative of Mandriva, which grew from a group of packages into a community-spawned desktop distribution.ROSA Linux, another former derivative of Mandriva, now developed independently.

OpenSUSE :It is a community distribution mainly sponsored by German company SUSE.SUSE Linux Enterprise, derived from openSUSE, maintained and commercially supported by SUSE.

Arch Linux :It is a rolling release distribution targeted at experienced Linux users and maintained by a volunteer community, offers offcial binary packages and a wide range of unoffcial user-submitted source packages. Packages are usually defned by a single PKGBUILD text fle. Manjaro Linux, a derivative of Arch Linux that includes a graphical installer and other ease-of-use features for less experienced Linux users. Rolling release packages from Arch repositories are held for further testing to achieve increased stability, and packages identifed as addressing security issues of critical or high severity are “fast-tracked” to the stable branch.

Gentoo :A distribution targeted at power users, known for its FreeBSD Ports-like automated system for compiling applications from source code.Chrome OS, Google's commercial operating system (using Gentoo and its Portage) that primarily runs web applications.

Slackware :Created in 1993, one of the frst Linux distributions and among the earliest still maintained, committed to remaining highly Unix-like and easily modifable by end users.

Command line interface

A command-line user interface (CLI), also known as a console user interface and character user interface (CUI), is a means of interacting with a computer program where the user (or client) issues commands to the program in the form of successive lines of text (command lines). A program which handles the interface is called a command language interpreter or shell.Command-line interfaces to computer operating systems are less widely used by casual computer users, who favor graphical user interfaces or menu-driven interaction.Command-line interfaces are often preferred by more advanced computer users, as they often provide a more concise and powerful means to control a program or operating system.Programs with command-line interfaces are generally easier to automate via scripting.Command line interfaces for software other than operating systems include a number of programming languages such as Tcl/Tk, PHP and others, as well as utilities such as the compression utilities WinZip and UltimateZip, and some FTP and ssh/telnet clients.

Advantage or Disadvantage

Advantages-Requires fewer resources Concise access to options Expert-friendly Easier to automate via scripting or batch fles Commands can be logged to review or repeat Easy to add special sub-options Shorter to show multi-step actions.

Disadvantages-Requires help guide for commands Commands can foster complex options Not visually rich, results might scroll off-screen Beginner-unfriendly.

Connect to Linux

To connect Linux server, you need server IP address, username and password and

These will be provided by the System Administrator.

If you install Linux server in your local machine or in Virtual machines you can get the

IP address with ifconfg command.

Step 1 : Login to your Local server

Step 2 : Right click and select open in terminal

Step 3 : use ifconfg command and get IP address.

Once you have required details, Download PuTTY application from the PuTTY website.

Be sure to comply with the license requirements.

Website : http://www.putty.org/

PuTTY is an SSH and telnet client, developed originally by Simon Tatham for the Windows platform. PuTTY is open source software that is available with source code and is developed and supported by a group of volunteers.

Step 1 : Launch PuTTY client and for Host Name enter the IP address of your server and the connection type is set to SSH.

Step 2 : Click on OpenIf you have used PuTTY for the frst time to log in to your server with SSH, you may see a warning popup.ClickYes. Subsequent connections will not show the above warning because the host key is now cached in the registry of your local computer.You can expect to see that warning, if you connect to your server from a different computer.

Step 3 : Once you accept the warning, the terminal prompts you for your username and password.Once successfully authenticated, you can work on Linux server.You can change background colour in colours

© CEHPOINT E-LEARNING 2020 ( 11 ) options. You can increase the lines of the scrollbar in Windows. You can save all sessions into logs. You can name the logs with hostname, date and time with & H, &D etc..You don't need to set these every time. Save all the changes as Session with name and next time click on the saved session and click on the load button to load and click open.Save username and password - You can't save username and password in PuTTY. But you can pass username and password as arguments. Create a shortcut to PuTTY and right click on it and select properties and add below line in Target location.username@IP address -pw password To use saved session properties add below line in Target location. -load MySession [email protected] -pw password Then click on shortcut to login to your server. You don't need to enter user name and password.There are some other applications to work with your server like PuTTY Manager, mRemote and Secure CRT(commercial).

Hardware information

•Display CPU information. -$ cat /proc/cpuinfo

•Display memory information -$ cat /proc/meminfo

•Display version details -$ cat /proc/version

•Display partitions details -$ cat /proc/partitions

•Display SCSI/Sata devices details-$ cat /proc/scsi/scsi

•Display information about Sata devices -$ hdparam /dev/sda1

•List Hardware components -# lshw

•Display hardware information -# hwinfo --short

•List scsi devices -# lsscsi

•List all PCI devices -$ lspci

•List USB devices -$ lsusb

•List block devices -$ lsblk

•Partition table manipulator for Linux -# fdisk -l

•Locate the binary, source, and manual page fles for a command -# whereis bin

# whereis etc

•Show the full path of (shell) commands -# which cat

# which ls

•Search the whatis database for complete words -# whatis ls

# whatis cat

•Upstart process management daemon -

# init 0 - halt (Do NOT set init default to this)

# init 1 - Single user mode

# init 2 - Multiuser, without NFS

# init 3 - Full multiuser mode

# init 6 - reboot

•Extract hardware information by reading data from the DMI tables -

# dmidecode -t memory

# dmidecode -t system

# dmidecode -t bios

# dmidecode -t processor

Shortcut key Description

•Ctrl + C Kill the current command

•Ctrl + Z Suspend the current command, resume with fg in the foreground or bg in the background

•Ctrl + D Logout the current session, similar to exit

•Ctrl + W Erases or cut one word in the current line

•Ctrl + U Erases or cut the whole line

•Ctrl + Y Paste the previous cut text

•Alt + Y Paste the second latest cut text

•Ctrl + K Clear the line after the cursor

•Ctrl + S Search

•Ctrl + R Reverse search, Type to bring up a recent command

•Ctrl + J It end the Ctrl + R search

•Ctrl + G It abort the search by Ctrl + R and restore original line

•Alt + R Restores all changes made to line

!! Repeats the last command

•Ctrl + A Go to the beginning of the line

•Ctrl + E Go to the end of the line

•Ctrl + L Clears the Screen, similar to the clear command

•Ctrl + H Same as backspace

•Tab Auto-complete fles and folder names

•End Button Go to the end of the line

•Up/Down Arrows To run last used command, you can also edit the command before run

•Ctrl+Left and Ctrl+Right Ctrl and the left or right arrow keys jumps between arguments in your command. So you can easily modify your commands in the middle.

•Ctrl + P It shows previous history,same like UP arrow key

•Ctrl + N It shows forward history,same like DOWN arrow key

•Ctrl + B Move cursor one character to the left side

•Ctrl + F Move cursor one character to the right side

•Ctrl + Shift + C To copy selected text

•Ctrl + Shift + V To paste you last copied by Ctrl + Shift + C

•Shift + Insert Paste from the clipboard

•Ctrl + T Swap the last two characters before the cursor

•Esc + T Swap the last two words before the cursor

•Alt + F Move cursor forward one word on the current line

•Alt + B Move cursor backward one word on the current line •Ctrl + M It is like a Enter key (If ENTER key do not work, you can try this)

*ls - list directory contents

List information about the fles (the current directory by default).

Options :

*-a, --all : do not ignore entries starting with . (show hidden fles)

*-c with -lt : sort by, and show, last modifcation of fle status information.

*-l use a long listing format ( equivalent to ll command)

*-r, --reverse : reverse order while sorting

Use cases:

*$ ls -lhtr

*$ ls -a

Use case: pwd - print name of current/working directory

*$ pwd

*cd - Change directory

Use cases: Goto home directory

*$ cd

Goto mydir directory

*$ cd mydir

move up one directory

*$ cd ..

move up two directories

*$ cd ../../

*cp - copy fles and directories

*$ cp fle.txt fle_copy.txt

*$ cp fle.txt dir/fle_copy.txt

*$ cp fle.txt ../../fle_copy1.txt

*mv - move (rename) fles

*$ mv fle.txt notes.txt

Move fle.txt into another directory with notes.txt name

*$ mv fle.txt dir/subdir/notes1.txt

Use cases: touch - change fle timestamps

*$ touch fle.txt

If a fle doesn't exist, it will create a zero byte fle.

Use cases: Display contents of fles.txt

*$ cat fles.txt

Override content to fles.txt

*$ cat > fles.txt

Append content to fles.txt

*$ cat >> fles.txt

Use cases: Create directory with name mydir

*$ mkdir mydir

*$ mkdir -p mydir/notes

*rm - remove fles or directories

Use cases: Remove fles

*$ rm fles.txt fles2.txt

*$ rm -f fles.txt fles.txt

Remove directories and their contents recursively (-r)

*$ rm -r mydir

*rmdir - remove empty directories

Use case: Remove an empty directory

*$ rmdir dir_name

Use cases: Create symbolic link (like shortcut fles in windows)

*$ ln -s fles.txt mydir/fles.txt

Create a hard link, like another copy of the fle.

*$ ln fles.txt mydir/notes/fles.txt

Use cases: Display frst 10 lines

*$ head fles.txt

Display frst n lines

tail - output the last part of fles

Use cases: Display last 10 lines

*$ tail fles.txt

*$ tail -n fles.txt

head-tail combination to get middle n lines

*$ tail -30 fles.txt | head

wc - print newline, word, and byte counts for each fle

Use cases: Print bytes counts

Print characters counts

*$ wc -m fles.txt

Print lines counts

*$ wc -l fles.txt

Print the word counts

*$ wc -w fles.txt

Print the length of the longest line

*$ wc -L fles.txt

*$ vi fles.txt

You see more info in the next Chapter.

*view - view fles as read-only

$ view fles.txt

The more command is a "pager" utility used to view text in the terminal window one page or screen at a time.

Use case:

$ more fles.txt

Use case

*$ less fles.txt

date - print or set the system date and time

*$ date

cal - displays a calendar

Use case:

*$ cal

Use case:

*$ w

locate - fnd fles by name

Use case:

*$ locate fles.txt

*$ locate -i fles.txt

ps - report a snapshot of the current processes.

Use case:To see every process on the system using standard syntax

*$ ps -e

To see every process on the system using BSD syntax

*$ ps axu

Use cases:

*$ top

press q to quit. kill - terminate a process

Use case:

Use cases:

*df -h

du - estimate fle space usage

Use cases:Display only a total for each argument

*$ du -s fle

Display only a total for each fle in human readable

free - Display amount of free and used memory in the system

Use case:

*$ free

ssh - OpenSSH SSH client

Use case:

*$ ssh username@remotehost scp - secure copy

(remote fle copy program)

Use case:

*$ scp fles.txt user@remotehost:path man - format and display the on-line manual pages

Use cases:

*$ man ls

*$ man scp

*$ man du

Use case:

*$ wget https://example.com/downloads/test.tar

su - substitute user

Use case:

Substitute other user

*$ su - username

Substitute root user

*$ su - exit - exit or close terminal

Use case:

*$ exit (CTRL -D shortcut)

Use case:

*$ clear (CTRL +L shortcut)

The default editor that comes with the Linux operating system is called vi (visual) editor.There are many ways to edit the fle but the best way is using vi editor.The vi editor is a full screen editor and it has two modes.

1.Command Mode

This mode enables you to perform tasks such as saving fles, moving the cursor, cutting, copy and pasting lines or words, and fnding and replacing. In this mode, whatever you type is executed as a command.

2.Insert Mode

This mode enables you to insert text into the fle. Everything that's typed in this mode is inserted in the fle .By default it is in command mode at the time editing a fle with vi editor. To insert any record press 'i' to insert mode. After that, come back to command mode by pressing the ESC button.

How to edit sample.txt with vi editor

$ vi sample.txt

"sample.txt" [New File]

How to quit from vi editor

Press ESC + :q whenever no changes in the fle. If any modifcations are done, press

ESC + :q!. To quit from vi by saving changes in the fle, press ESC + :wq

This is a sample text page.

:wq

Editing Commands

Command Description

•i Inserts text before current cursor location.

•I Inserts text at beginning of current line.

•a Inserts text after current cursor location.

•A Inserts text at end of current line.

•o Creates a new line for text entry below cursor location.

•O Creates a new line for text entry above cursor location.

Deleting Commands

•x Deletes the character under the cursor location.

•X Deletes the character before the cursor location.

•dw Deletes from the current cursor location to the next word.

•d^ Deletes from current cursor position to the beginning of the line.

•d$ Deletes from current cursor position to the end of the line.

•D Deletes from the cursor position to the end of the current line.

•dd Deletes the line the cursor is on.

•cc Removes contents of the line, leaving you in insert mode.

•cw Changes the word the cursor is on from the cursor to the lowercase w end of the word.

•r Replaces the character under the cursor. vi returns to command mode after the replacement is entered.

•R Overwrites multiple characters beginning with the character currently under the cursor. You must use Esc to stop the overwriting.

•s Replaces the current character with the character you type. Afterward, you are left in insert mode.

•S Deletes the line the cursor is on and replaced with new text. After the new text is entered, vi remains in insert mode.

Copy and Paste Commands

•yy Copies the current line.

•yw Copies the current word from the character the lowercase w cursor is on until the end of the word.

•p Puts the copied text after the cursor.

•P Puts the yanked text Before the cursor.

Moving commands

•k Moves the cursor up one line.

•j Moves the cursor down one line.

•h Moves the cursor to the left one character position.

•l Moves the cursor to the right one character position.

•0 or |Positions cursor at beginning of line.

•$ Positions cursor at end of line.

•w Positions cursor to the next word.

•b Positions cursor to previous word.

•( Positions cursor to beginning of current sentence.

•) Positions the cursor to the beginning of next sentence.

•E Move to the end of Blank delimited word

•{ Move a paragraph back

•} Move a paragraph forward insert mode.

•[[ Move a section back

•n| Moves to the column n in the current line

•1G Move to the frst line of the fle

•G Move to the last line of the fle

•nG Move to nth line of the fle

•:n Move to nth line of the fle

•fc Move forward to c

•Fc Move back to c

•H Move to top of screen

•nH Moves to nth line from the top of the screen

•M Move to middle of screen

•L Move to bottom of screen

•nL Moves to nth line from the bottom of the screen

•: x Colon followed by a number would position the cursor on line number represented by x

Control Commands

CTRL+d Move forward 1/2 screen

CTRL+f Move forward one full screen

CTRL+u Move backward 1/2 screen

CTRL+b Move backward one full screen

CTRL+e Moves screen up one line

CTRL+y Moves screen down one line

CTRL+u Moves screen up 1/2 page

CTRL+d Moves screen down 1/2 page

CTRL+b Moves screen up one page

CTRL+f Moves screen down one page

CTRL+I Redraws screen

Searching commands

•^ Search at the beginning of the line.

•. Matches a single character.

•$ Search at End of the line

•[ Starts a set of matching, or non-matching expressions.

•< Put in an expression escaped with the backslash to fnd the ending or beginning of a word.

•> Put in an expression escaped with the backslash to fnd the ending or beginning of a word

Replacing Text

•:s/ replacing command

Running Commands

•:! to run the command within the vi editor.

•:! ls to see the output of the running command.

Set Commands

:set ic Ignores case when searching

:set ai Sets autoindent

:set noai To unset autoindent.

:set nu Displays lines with line numbers on the left side.

:set sw Sets the width of a software tabstop. For example you would set a shift width of 4 with this command: :set sw=4

:set ws If wrapscan is set, if the word is not found at the bottom of the fle, it will try to search for it at the beginning.

:set wm If this option has a value greater than zero, the editor will automatically "word wrap". For example, to set the wrap margin to two characters, you would type this: :set wm=2

:set ro Changes fle type to "read only"

:set term Prints terminal type

:set bf Discards control characters from input

Other Commands

•J Join the current line with the next one. A count joins that many lines.

•<< Shifts the current line to the left by one shift width.

•>> Shifts the current line to the right by one shift width.

•~ Switch the case of the character under the cursor.

•^G Press CTRL and G keys at the same time to show the current flename and the status.

•u Undo the last change to the fle. Typing 'u' again will re-do the change.

•:f Displays current position in the fle in % and fle name, total number of fle.

•:f flename Renames current fle to flename.

•:w flename Write to fle flename.

•:e flename Opens another fle with flename.

•:cd dirname Changes current working directory to dirname.

•:e # Use to toggle between two opened fles.

•:n In case you open multiple fles using vi, use :n to go to the next fle in the series.

•:p In case you open multiple fles using vi, use :p to go to the previous fle in the series.

•:N In case you open multiple fles using vi, use :N to go to the previous fle in the series.

•:r fle Reads fle and inserts it after current line

•:nr fle Reads fle and inserts it after line n.

There are three types of user accounts in the Linux operating system. They are

1. Root user

The root user is also called a super user. A super user can run any command without any restrictions. This account gives you the ability to carry out all facets of system administration, including adding accounts, changing user passwords, examining log fles, installing software, etc.When you are signed in as root the shell prompt displays '#' as the last character (if you are using terminal).

2. System user

The System accounts are needed for the operation of system-specifc components like mail accounts and the sshd accounts. Also if we install Apache, it will create a user apache. These kinds of users are called system users.

3. Normal user

These users are created by root users. Normal user accounts provide access to the system for users and groups of users. These users have limited access to critical system fles and directories.

Attribute

User Type : Super user

User Name : root

User ID : 0

Group ID : 0

Home directory : /root

Shell : /bin/bash

User Type : System user

User Name : ftp,apache etc..

User ID : 1 to 499

Group ID : 1 to 499

Home directory : /var/ftp, /var/etc etc..

Shell : /sbin/nologin

User Name : mark, scott etc..

User ID : 500 to 60000

Group ID : 500 to 60000

Home directory : /home/username

Shell : /bin/bash

Important Files

/etc/passwd: - Keeps user account and password information. This fle holds the majority of information about accounts on the Linux system.

/etc/shadow: - Holds the encrypted password of the corresponding account.

Create User and assign password to user # useradd scott

Create a user with account expiry date

# useradd -e 2017-04-01 mark

# passwd mark

Changing password for user mark.

New password:

BAD PASSWORD: it is too short

BAD PASSWORD: is too simple

Retype new password: passwd: all authentication tokens

Assign password to user

# passwd scott

Changing password for user scott.

New password:

BAD PASSWORD: it is too short

BAD PASSWORD: is too simple

Retype new password: passwd: all authentication tokens updated successfully.

# useradd -u 504 jithacker

Create a user with Specifc user id and group id

# useradd -u 600 -g 504 cehpoint

Add User with comment

# useradd -c"Developer" sujan

# tail -1 /etc/passwd sujan:x:603:603:Developer:/home/sujan:/bin/bash

Add user with home directory

# useradd -d /home/mydir jithacker

# tail -1 /etc/passwd jithacker:x:604:604::/home/mydir:/bin/bash

Add a user to multiple groups

# useradd -G dev,qa sujan

# id sujan uid=601(sujan) gid=601(sujan) groups=601(sujan),3460(dev),3461(qa)

# useradd -G 3460,3461 cehpoint

# id cehpoint uid=602(cehpoint) gid=602(cehpoint)groups=602(cehpoint),3460(dev),3461(qa)

Useradd options :

-u user id

-g primary group id

-G secondary group id

-d home directory

-s shell

Delete users

Deleting a existing user

# userdel allen

Delete user even if he is still logged in

# userdel -f jones

Delete user along with home directory

# userdel -r mark

Changing the home directory of user

# grep sujan /etc/passwd sujan:x:504:504::/home/sujan:/bin/bash

# usermod -d /home/ceh sujan

# grep sujan /etc/passwd user8:x:604:604::/home/ceh:/bin/bash

Changing the primary group of a user

# usermod -g dev cehpoint

Changing login name and password

# usermod -l sujan banerjee

# usermod -p P@ssWord nAhI hai

View account aging information

# chage -l sujan

Changing password parameters

# chage sujan

Changing the aging information for sujan

Enter the new value, or press ENTER for the default

Create a new group

# groupadd grp01

# tail -1 /etc/group grp01:x:3461:

# Creating group force with success status.

# groupadd -f grp02

# tail -1 /etc/group grp02:x:3462:

# Creating group with a custom GID.

# groupadd -g 5000 grp03

# tail -1 /etc/group grp03:x:5000:

Creating system group

# groupadd -r sysgrp

# tail -1 /etc/group sysgrp:x:403:

Modifying groups

Changing name of the group

# groupmod -n newgrp grp01

Adding and removing user to a group

# tail /etc/group | grep grp03 grp03:x:5000:user8,user7

# gpasswd -d user6 grp03

# tail /etc/group | grep grp03 grp03:x:5000:user8,user7

Delete group Deleting a group

# groupdel newgrp

Options:

-M to add multiple users

-A to add group administrator

-a to add single user to group

-d to removing user from a group

Linux groups are a mechanism to manage a collection of users. All

Linux users have a user ID (UID) and a group ID (GID). Groups can be assigned users together for a common security, privilege and access purpose. It is the foundation of Linux security and

ID or group ID.

Important fles

/etc/group:- This fle contains the group information for each account.

/etc/gshadow:- This fle contains secure group account information.

ext2: Second Extended fle system

It was introduced in 1993. It doesn't have a feature. Maximum fle size can be 16 GB to 2

TB.Maximum ext2 fle system can be from 2TB to 32 TB. Can not convert ext fle system to ext2. ext3: Third extended fle system

It was introduced in 2001. It supports journaling features. Maximum fle size can be 16

GB to 2 TB and maximum ext3 fle system size can be from 2 TB to 32 TB. You can convert an ext2 fle system to ext3 fle system directly. ext4: Fourth extended fle system

It was introduced in 2008. It supports journaling features. Maximum fle size can be from 16 GB to 16 TB and maximum ext4 fle system size is 1 EB (Exabyte). All previous ext fle systems can easily be converted into ext4 fle systems.

Every disk can have only 3 Primary partitions.

Primary partition is a partition which usually holds the operating system. Only one amongst the three partitions can be active which will be booted by MBR to load the operating system Extended Partition is a special type of primary partition which can be subdivided into multiple logical partitions. As there can be only 3 primary per disk, and if the user is required to make further partitions then all the space remaining on the disk should be allocated to extended partitions, which can be used to create the logical partitions later. There can be only one extended partition per disk.

Logical partitions are the partitions which are created under extended partitions, all the space in the extended partition can be used to create any number of logical partitions.

Disk Identifcation

IDE drive will be shown as /dev/hda

SCSI dive will be shown as /dev/sda

Virtual drive will be shown as /dev/vda

To view existing partitions :

The following command lists all existing disk partitions on your system.

# fdisk -l or

# parted -l

Partition administration using fdisk.

To enter into disk utility, the syntax is fdisk disk name

# fdisk /dev/sda

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u').

Command (m for help):

Use m to list out various options that can be used in fdisk.

Creating a new partition

If you have free space left on your disk /dev/sda and would like to create a new partition under it. Type the following command to enter into command mode of specifc

# fdisk /dev/sda

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u').

Command (m for help):

After entering in command mode, now press n command to create a new partition under /dev/sda with specifc size.

# fdisk /dev/sda

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u').

Command (m for help): n

Command action

e extended

p primary partition (1-4)

You can enter the size of the cylinder by adding +500M in the last cylinder. Here, + means addition and

500M means the size of the new partition.

After creating a new partition, you should run w command to alter and save new changes to partition table and fnally reboot your system to verify newly created partition.

Command (m for help): w

The partition table has been altered!

Calling ioctl() to re-read the partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.

The kernel still uses the old table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8)Syncing disks.

Deleting a partition If you would like to delete a specifc partition from disk /dev/sda, use the option to delete a partition and specify the device name.

# fdisk /dev/sda

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to

Command (m for help): d

Partition number (1-4):

Formatting a partition.

After creating a partition we need to assign some fle system to it so that we can start storing the data into it.

Syntax : # mkfs.ext4 partition_name

# mkfs.ext4 /dev/sda4

Likewise you can format the different partitions with different fle systems like below.

# mkfs.ext3 /dev/sda5

# mkfs.vfat /dev/sda6

Check Size of a Partition.

Use s (displays size in blocks) with fdisk command.

# fdisk -s /dev/sda2

4096000

Creating a Swap partition

Create a normal partition with fdisk command by using n option.

Next, Change hex code to make it swap partitions by using the option.

The hex code for swap is 82.

Update the partition table table using partx -a or kpartx command.

Format the partition with swap fle system

# mkswap /dev/sda6

Turn on newly created swap space and verify it.

# swapon /dev/sda6

# swapon -s

# free -m

Making the newly created swap partition to mount after reboot In order to make the swap partition mount automatic after reboot, we need to make an entry in /etc/fstab fle.

/dev/sda6 swap swap defaults 0 0

# vi /etc/fstab

Removing the swap partition Deactivate the swap partition using # swapoff /dev/sda6

Remove the entry from /etc/fstab

Delete the partition through fdisk.

Even after formatting the partition, we can not add the data into the partition.

In order to add the data into the partition it is required to be mounted.

Mounting a partition Mounting is the attaching of an additional fle system to the currently accessible fle system of a computer. There are two types of mounting which will be used in Linux.

Temporary Mounting

Permanent Mounting

Temporary Mounting

In a temporary mount point we will create a directory and mount it, but this mount point will last only till the system is up, once it is rebooted the mounting will be lost.

Syntax : # mount device directory

# mkdir /data1

# mount /dev/sda4 /data1

To view all the mounted partitions, use below command.

# mount

Once we successfully mount the partition we can access it and can store the data.

# cd /data1

Unmounting a partition Removing the connection between the mounted device and the rest of the fle system is referred to as unmounting.

# umount /dev/sda4 /data1

Once unmounted, verify with mount command.

Permanent Mounting.

Permanent mounting procedure is exactly same like temp mounting, but here we will update the /etc/fstab fle with the mounting details. So that it will be mounted even after system is reboot.

Steps to make a permanent mount point.

Make a directory or use an existing directory Add entry in /etc/fstab fle

Use mount -a command to check it is mounting. ( mount -a will mount all the entries placed in /etc/fstab)

Entry : /dev/sda4 /data1 ext4 defaults 0 0

Device Name - Mount point - Type of File system - Mount options - Dumping - Check Sequence

# vi /etc/fstb

After adding the above entry, use mount -a command to mount and use mount command to verify it.

Now you can access the directory and add,delete or modify the contents and also unmount the fle system at any time,

Some times a directory refects error while unmounting You are the same directory and trying to unmount it.

Check with the pwd command.

Some users are present in the directory and using the contents in it.

Check with fuser -cu /dev/sda4

Check for the fles which are open with lsof /dev/sda4

Kill open connections using fuser -ck /data1/export where export is the fle which is open

Now you can use umount command to unmount the fle system.

To view usage information of mounted partition use the command df -h To view the size od the fle or directory use the command du -h fle or directory Assigning label to the partition

Assigning the label is giving some name to the partition. To assign label to the partition use e2label command.

# e2label /dev/sda4 data

To list the all mounted partitions along with their labels, use mount -l command

# mount -l

Mounting a partition using its label

Mounting a /dev/sda4 partition with its label data

# mount LABEL=data /data1

Verify it with mount command

# mount

Making a permanent mount point using a label.

As we know that to make a permanent mount point, an entry has to be made in /etc/fstab.

# vi /etc/fstab

Now use mount -a command and verify it with mount command whether it is mounted or not.

Mounting a partition permanently with its block id (UUID)

To check the uuid of a partition use blkid /dev/sda4

Copy the uuid

Make an entry in /etc/fstab using UUID

Verify it with mount -a option.

# blkid /dev/sda4

/dev/sda4: LABEL=data UUID="cb87e43f-f72c-405e-a282-f6b7d29e3d81" TYPE="ext4"

# vi /etc/fstab

Entry :

UUID=cb87e43f-f72c-405e-a282-f6b7d29e3d81 /data1 ext4 defaults 0 0

Now mount it with mount -a command and verify it with mount command.

# mount -a

Directory structure

The Linux fle system is hierarchically structured. In the Filesystem Hierarchy Standard (FHS), all fles and directories appear under the root directory /, even if they are stored on different physical or virtual devices.

Most of these directories exist in all UNIX like operating systems and are generally used in much the same way; however, the descriptions here are those used specifcally for the FHS, and are not considered authoritative for platforms other than Linux.

/ root directory. It is the top level directory.

/bin

It is the place where all executable fles are located and available for all users.

/dev

It Contains references to all the CPU peripheral hardware, which are represented as fles with special properties.

/etc

It contains most important System confguration fles like disk confguration fles,valid user lists, groups,

Ethernet etc.. just like control panel in windows.

/lib

It contains library fles ,includes fles for all kinds of programs needed by the system and the users .

/home

It is home directory of common user.

/root

It is home directory for root users.

/boot

It contains the fles for booting the system.

It Contains all processes considerable as a fle by process number or other information that is dynamic to the system.

/tmp

It contains temporary fles used by the system ,cleaning upon reboot.

/usr

It contains Programs, libraries, documentation etc.. for all user-related programs.

/var

It is the Storage for all variable fles and temporary fles created by users.

/sbin

It contains binary executable fles. Programs for use by the system and the system administrator.

In Linux there are three basic types of fles

1.Ordinary fles

These are the text fles or contain some data and program instructions.

2.Directory fles

These are just like folders in windows. Directory fles having Ordinary and Special fles also.

3.Special fles

These special fles provide access to hardware such as hard drives, CD-ROM drives, modems, and Ethernet adapters. Other special fles are similar shortcuts and enable you to access a single fle using different names.

Hidden fles :

A hidden folder (directory) or hidden fle which flesystem utilities do not display by default when showing a directory listing. Hidden fles begin with "." and are commonly used for storing user preferences or preserving the state of a utility, and are frequently created implicitly by using various utilities.

To display hidden fles use ls -a command.

File systems have methods to assign permissions to specifc users and groups.

File system use permissions to regulate the level of interaction that system processes can have with fles and directories. Permission Types

Each fle or directory has three basic permission types: read (r) - user able to read the contents of the fle. write (w) - user able to write or modify a fle or directory. execute (x) - user able to execute a fle or view the contents of a directory.

Permission Groups

Each fle and directory has three user based permission groups: owner (u)- A single person who owns the fle. But ownership may be granted to some one else by certain users group (g) - The group that the fle belongs to. others (o) - The other users, i.e. everyone else.

Use the ls command's -l option to view the permissions for the contents of a directory.

Changing permissions chmod is a command in Linux that allows to change the permissions of a fle or directory.

Add single permission

$ chmod g+r systemDataFile.txt

Add multiple permissions

$ chmod g+x,o+r User.txt

$ chmod g+wx,o+rw Note.txt

Remove permission

$ chmod o-wx systemDataFile.txt

Clone permission from one fle to another by reference.

$ chmod --reference=User.txt Note.txt

Using numbers, allows you to edit the permissions for all three owners, group, and others at the same time. r = 4 w = 2 x = 1

When you want to set permissions, we just add up the number. For example, to set the permissions to read and write, we will use '6' (4 + 2) for the permission. For read, write and execute, we will use '7' (4 + 2 + 1) for the permission.

Here's the different permutation.

0 no permission

1 execute

2 write

3 write and execute

4 read

5 read and execute

6 read and write

7 read , write and execute

Depending on the permissions you want to grant to the fle, you just set the number accordingly.

$ chmod 764 systemDataFile.txt

$ chmod 744 User.txt

$ chmod 700 Note.txt

10 Types of fle processing

1 Sort

2 Uniq

3 Cut

4 Paste

5 Jion

6 Grep

7 Sed

8 Awk

9 fnd

10 locate

Sort is a Linux program used for sort lines of text fles and concatenation of all fles in sorted order.

Before we start with 'sort' let's have a look at the contents of the fle

$ cat Distros.txt

Debian

Mint

Ubuntu

Fedora

RedHat

CENTOS

ARCH

Mint

RedHat

Now sort the content of the fle using following command.

$ sort Distros.txt

ARCH

Debian

Fedora

Mint

RedHat

Ubuntu

Sort the contents of text fle in reverse order by using '-r' option.

$ sort -r Distros.txt

Ubuntu

RedHat

Mint

Fedora

Debian

CENTOS

ARCH

Also, we can remove duplicates in sorted text with -u option

$ sort -u Distros.txt

ARCH

CENTOS

Debian

Fedora

Mint

Ubuntu

If the fle is comma, tab or any other delimited fle, then we can sort by columns as well.

Below fle is the comma delimited fle with three columns.

$ cat versions.txt

JDK,Beta,1994

JDK,1.0,1996

JDK,1.1,1997

J2SE,1.2,1998

J2SE,1.3,2000

J2SE,1.4,2002

J2SE,5.0,2004

Java,SE 6,2006

Java,SE 7,2011

Java,SE 8,2014

Now, we can sort on the basis of 2nd column by using following command.

$ sort -t"," -k2 versions.txt

JDK,1.0,1996

JDK,1.1,1997

J2SE,1.2,1998

J2SE,1.3,2000

J2SE,1.4,2002

J2SE,5.0,2004

JDK,Beta,1994

Java,SE 6,2006

Java,SE 7,2011

Java,SE 8,2014

Here, -t is used to mention feld-separator (delimiter) and -k is used to mention column number.

We can sort numeric columns by using -n option.

JDK,Beta,1994

JDK,1.0,1996

JDK,1.1,1997

J2SE,1.2,1998

J2SE,1.3,2000

J2SE,1.4,2002

J2SE,5.0,2004

Java,SE 6,2006

Java,SE 7,2011

Java,SE 8,2014

Note : Option '-n' must be used only when we want to sort a fle on the basis of a column which contains numerical values.

Following are the examples to sort on basic of multiple columns.

$ sort -t"," -k1,3n versions.txt

J2SE,1.2,1998

J2SE,1.3,2000

J2SE,1.4,2002

J2SE,5.0,2004

Java,SE 6,2006

Java,SE 7,2011

Java,SE 8,2014

JDK,1.0,1996

JDK,1.1,1997

JDK,Beta,1994

Now we can see how to sort, merge and remove duplicates from these two fles.

$ sort versions.txt Distros.txt

ARCH

CENTOS

Debian

J2SE,1.2,1998

J2SE,1.3,2000

J2SE,1.4,2002

J2SE,5.0,2004

Java,SE 6,2006

Java,SE 7,2011

Java,SE 8,2014

JDK,1.0,1996

JDK,1.1,1997

JDK,Beta,1994

Mint

RedHat

Ubuntu

$ sort -u versions.txt Distros.txt

ARCH

CENTOS

Debian

Fedora

J2SE,1.2,1998

J2SE,1.3,2000

J2SE,1.4,2002

J2SE,5.0,2004

Java,SE 6,2006

Java,SE 7,2011

Java,SE 8,2014

JDK,1.0,1996

JDK,Beta,1994

Mint

RedHat

Ubuntu

We don't need to sort directly on fles. We can use sort by pipe symbol as well.

$ cat Distros.txt |sort -u

ARCH

CENTOS

Debian

Fedora

Mint

RedHat

Ubuntu

Uniq

Uniq command is helpful to remove or detect duplicate entries in a fle.

When uniq command is run without any option, it removes duplicate lines and displays unique lines as shown below

$ cat Distros.txt

Debian

Ubuntu

RedHat

ARCH

RedHat

$ uniq Distros.txt

Debian

Ubuntu

RedHat

Count Number of Occurrences using -c option

$ uniq -c Distros.txt

1 Debian

2 Mint

1 Ubuntu

1 Fedora

2 RedHat

1 CENTOS

1 ARCH

We can print only duplicate Lines using -d option

$ uniq -d Distros.txt

Mint

RedHat

The above example displayed all the duplicate lines, but only once. But, this -D option will print all duplicate lines in fle.

$ uniq -D Distros.txt

Mint

RedHat

Mint

RedHat

We can print only Unique Lines using -u option

$ uniq -u Distros.txt

Debian

Ubuntu

Fedora

CENTOS

ARCH

Linux cut command is used for text processing. You can use this command to extract portion of text from a fle by selecting columns.

To extract only a desired column from a fle use -c option. The following example displays 3rd character from each line of the fle.

$ cat features.txt

Linux is a multiprogramming system.

Linux is a open source operating system.

Programs can run on different system in same functionality

Multiple jobs can run at the same time.

$ cut -c3 features.txt n n o l

We can also extract range of characters from a fle by specifying start and end position.

$ cut -c1-7 features.txt

Linux i

Program

Multipl

If we give the optio like -c3-, it will extract the lines from 3rd character to end character. If we give the optio like -c-7, it will extract the lines from 1st character to 7th character.

We can use options -f and -d to extract specifc felds from delimited fles.

$ cat versions.txt

JDK,Beta,1994

JDK,1.0,1996

JDK,1.1,1997

J2SE,1.2,1998

J2SE,1.3,2000

J2SE,1.4,2002

J2SE,5.0,2004

Java,SE 6,2006

Java,SE 7,2011

Java,SE 8,2014

$ cut -d',' -f2 versions.txt

Beta

1.0

1.1

1.3

1.4

5.0

SE 6

SE 7

SE 8

Here, -d option is used to specify delimiter and -f used to specify the feld numbers.

You can also extract more than one feld from a fle.

$ cut -d',' -f2,3 versions.txt

Beta,1994

1.0,1996

1.1,1997

1.2,1998

1.3,2000

1.4,2002

5.0,2004

SE 6,2006

SE 7,2011

SE 8,2014

To display the range of felds specify start feld and end feld as shown below. In this example, we are selecting feld 1 through 4, 6 and 7.

$ grep "/bin/bash" /etc/passwd | cut -d':' -f1-4,6,7 root:x:0:0:/root:/bin/bash bala:x:1000:1000:/home/bala:/bin/bash

Here delimiter is ":".

We can also select all felds except the specifed felds.

$ cut -d',' --complement -f2 versions.txt

JDK,1994

JDK,1997

J2SE,1998

J2SE,2000

J2SE,2002

J2SE,2004

Java,2006

Java,2011

Java,2014

By default the output delimiter is same as input delimiter that we specify in the cut -d option.

To change the output delimiter use the option -output-delimiter as shown below.

$ cut -d',' -f1,3 --output-delimiter='|' versions.txt

JDK|1994

JDK|1996

JDK|1997

J2SE|1998

J2SE|2000

J2SE|2002

J2SE|2004

Java|2006

Java|2011

Java|2014

$ cut -d',' -f1,3 --output-delimiter='-' versions.txt

JDK-1994

JDK-1996

JDK-1997

J2SE-1998

J2SE-2000

J2SE-2002

Java-2006

Java-2011

Java-2014

To output a delimiter to newline, use following command.

$ cut -d',' -f1,3 --output-delimiter=$'\n' versions.txt |head

JDK

1994

JDK

1996

JDK

1997

J2SE

1998

J2SE

2000

We must use $ for \n, \t etc..

We can combine cut commands with other commands with pipes.

Paste command is one of the useful commands in unix or linux operating system. The paste command merges the lines from multiple fles.

The paste command sequentially writes the corresponding lines from each fle separated by a TAB delimiter on the terminal.

$ cat fle1

Linux

Unix

Windows

$ cat fle2

Open Source

Commercial

By default, the paste command merges the fles in parallel.

$ paste fle1 fle2

Linux Open Source

Unix Commercial

Windows

$ paste fle2 fle1

Open Source Linux

Commercial Unix

Windows

We can change the delimiter to any other character by using the -d option.

$ paste -d'-' fle1 fle2

Linux-Open Source

Unix-Commercial

Windows-

Linux|Open Source

Unix|Commercial

Windows|

We can merge the fles in sequentially using the -s option. The paste command reads each fle in sequentially.

It reads all the lines from a single fle and merges all these lines into a single line.

$ paste -s fle1 fle2

Linux Unix Windows

Open Source Commercial

$ paste -s -d',' fle1 fle2

Linux,Unix,Windows

Open Source,Commercial

Specifying multiple delimiters.

$ paste -d'-,' fle1 fle2 fle3

Linux-Open Source,CommandLine

Unix-Commercial,User Interface

Windows-,

The paste command can also be used to merge N consecutive lines from a fle into a single line.

The following example merges 2 consecutive lines into a single line.

$ cat fle1 | paste - -

Linux Unix

Windows

Copy command is one of the useful commands in unix or linux operating system. The copy command copy multiple fles.

The copy command sequentially writes the corresponding lines from each fle separated by a TAB delimiter on the terminal.

$ cat fle1

Linux

Unix

Windows

$ cat fle2

Open Source

Commercial

By default, the paste command merges the fles in parallel.

$ paste fle1 fle2

Linux Open Source

Unix Commercial

Windows

$ paste fle2 fle1

Open Source Linux

Commercial Unix

Windows

We can change the delimiter to any other character by using the -d option.

$ paste -d'-' fle1 fle2

Linux-Open Source

Unix-Commercial

$ paste -d'|' fle1 fle2

Linux|Open Source

Unix|Commercial

Windows|

We can merge the fles in sequentially using the -s option. The paste command reads each fle in sequentially.

It reads all the lines from a single fle and merges all these lines into a single line.

$ paste -s fle1 fle2

Linux Unix Windows

Open Source Commercial

$ paste -s -d',' fle1 fle2

Linux,Unix,Windows

Open Source,Commercial

Specifying multiple delimiters.

$ paste -d'-,' fle1 fle2 fle3

Linux-Open Source,CommandLine

Unix-Commercial,User Interface

Windows-,

The paste command can also be used to merge N consecutive lines from a fle into a single line.

The following example merges 2 consecutive lines into a single line.

$ cat fle1 | paste - -

Linux Unix

Windows

Join command is used to combine two fles based on matching felds in the fles. If you know SQL, the join command is similar to joining two tables in a database.

By default, the join command treats the feld delimiter as space or tab.

$ cat emp.txt

1101 scott

1102 joshua

1103 Ross

1104 Sami

1106 Rose

1113 Lucky

$ cat dept.txt

1101 Support

1102 QA

1103 Dev

1104 Support

1106 Admin

1113 Admin

$ join emp.txt dept.txt

1101 scott Support

1102 joshua QA

1103 Ross Dev

1104 Sami Support

1106 Rose Admin

1113 Lucky Admin

VNote: Before joining the fles, make sure to sort the felds on the joining felds.

Otherwise you will get incorrect results.

$ cat wines.txt

Red Beaunes France

White Riesling Germany

Red Riocha Spain

$ cat comments.txt

Beaunes Great!

Reisling Terrible!

Riocha Meh

These fles can be joined by specifying the felds that should be used to join the fles. Common to both fles is the name of the wine. In wine.txt

This is the second feld. In comments.txt this is the frst feld.

The fles can be joined using -1 and -2 by specifying these felds.

$ join -1 2 -2 1 wines.txt comments.txt

Beaunes Red France Great!

Reisling White Germany Terrible!

Riocha Red Spain Meh

The sort command can sort the fles before passing to join.

$ join -1 2 -2 1 <(sort -k 2 wine.txt) <(sort reviews.txt)

Beaunes Red France Great!

Reisling White Germany Terrible!

Riocha Red Spain Meh

To specify a feld separator for joining using the join command use the -t option.

In the following example there are two fles names.csv and deposits.csv.

$ cat names.csv

1,John Smith,London

2,Arthur Dent, Newcastle

3,Sophie Smith,London

$ cat transactions.csv

4534,Withdrawal,Arthur Dent

4675,Deposit,Sophie Smith

$ join -1 2 -2 3 -t , names.csv transactions.csv

John Smith,1,London,1234,Deposit

Arthur Dent,2, Newcastle,4534,Withdrawal

Sophie Smith,3,London,4675,Deposit

To specify the order the list of felds are passed to -o.

$ join -1 2 -2 3 -t , -o

1.2,1.3,2.1,2.2 names.csv transactions.csv

John Smith,London,1234,Deposit

Arthur Dent,

Newcastle,4534,Withdrawal

Sophie

Smith,London,4675,Deposit

Note : To ignore case, we can use -i option.

Grep is the frequently used command in Unix (or Linux). Most of us use grep just for fnding the words in a fle.

The power of grep comes with using its options and regular expressions.

You can analyze large sets of log fles with the help of grep command.

$ cat ora_dba.txt

LEARN LINUX, SQL AND ORACLE 11G ADMIN STEP BY STEP WITH EXAMPLES. learn linux, sql and oracle 10G administration step by step with examples.

Learn Linux, Sql and Oracle 11g Administration Step by Step with examples.

Here You can learn Oracle core DBA.

Finally, linux, sql and Oracle DBA Interview Question and Answers.

Search for a string in a fle

$ grep "linux" ora_dba.txt learn linux, sql and oracle 10G administration step by step with examples.

Finally, linux, sql and Oracle DBA Interview Question and Answers.

Search for a string in multiple fles.

$ cp ora_dba.txt ora_dba2.txt

$ grep "linux" ora_dba* ora_dba2.txt:learn linux, sql and oracle 10G administration step by step with examples. ora_dba2.txt:Finally, linux, sql and Oracle DBA Interview Question and Answers. ora_dba.txt:learn linux, sql and oracle 10G administration step by step with examples. ora_dba.txt:Finally, linux, sql and Oracle DBA Interview Question and Answers.

Case insensitive search

$ grep -i "linux" ora_dba.txt

LEARN LINUX, SQL AND ORACLE 11G ADMIN STEP BY STEP WITH EXAMPLES. learn linux, sql and oracle 10G administration step by step with examples.

Learn Linux, Sql and Oracle 11g Administration Step by Step with examples.

Checking for full words, not for sub-strings using grep -w

$ grep -i "admin" ora_dba.txt

LEARN LINUX, SQL AND ORACLE 11G ADMIN STEP BY STEP WITH EXAMPLES. learn linux, sql and oracle 10G administration step by step with examples.

Learn Linux, Sql and Oracle 11g Administration Step by Step with examples.

$ grep -iw "admin" ora_dba.txt

LEARN LINUX, SQL AND ORACLE 11G ADMIN STEP BY STEP WITH EXAMPLES.

Displaying the lines after the match.

$ grep -A 2 -i "10" ora_dba.txt learn linux, sql and oracle 10G administration step by step with examples.

Learn Linux, Sql and Oracle 11g Administration Step by Step with examples.

Displaying the lines before the match.

$ grep -B 2 -i "10" ora_dba.txt

LEARN LINUX, SQL AND ORACLE 11G ADMIN STEP BY STEP WITH EXAMPLES. learn linux, sql and oracle 10G administration step by step with examples.

Displaying the lines around the match

$ grep -C 2 -i "10" ora_dba.txt

LEARN LINUX, SQL AND ORACLE 11G ADMIN STEP BY STEP WITH EXAMPLES. learn linux, sql and oracle 10G administration step by step with examples.

Learn Linux, Sql and Oracle 11g Administration Step by Step with examples.

Specifying the search string as a regular expression pattern.

$ grep "1*G" ora_dba.txt

LEARN LINUX, SQL AND ORACLE 11G ADMIN STEP BY STEP WITH EXAMPLES.

Searching for a sting in all fles recursively

$ grep -r "Linux" *

Linux:Linux is an operating system.

Linux:Linux is very similar to other operating systems, such as Windows and OS X.

LinuxIntro:Linux is an operating system.

LinuxIntro:Linux is very similar to other operating systems, such as Windows and OS X. ora_dba2.txt:Learn Linux, Sql and Oracle 11g Administration Step by Step with examples. ora_dba.txt:Learn Linux, Sql and Oracle 11g Administration Step by Step with examples.

Inverting the pattern match

$ grep -iv "sql" ora_dba.txt

Here You can learn Oracle core DBA.

Displaying the non-empty lines

$ grep -v "^$" ora_dba.txt

LEARN LINUX, SQL AND ORACLE 11G ADMIN STEP BY STEP WITH EXAMPLES. learn linux, sql and oracle 10G administration step by step with examples.

Learn Linux, Sql and Oracle 11g Administration Step by Step with examples.

Here You can learn Oracle core DBA.

Finally, linux, sql and Oracle DBA Interview Question and Answers.

Displaying the count of the number of matches.

$ grep -ic "oracle" ora_dba.txt

Display the fle names that match the pattern.

$ grep -l "sql" * ora_dba2.txt ora_dba.txt

Display the fle names that do not contain the pattern.

Desktop

Documents

Downloads

Linux

LinuxIntro

Music operating system

Pictures

Public sample.txt

Templates test2.txt.gz test.txt.gz

Videos

Displaying only the matched pattern.

$ grep -o "Linux" ora_dba.txt

Linux

Displaying the line numbers.

$ grep -n "sql" ora_dba.txt

2:learn linux, sql and oracle 10G administration step by step with examples.

6:Finally, linux, sql and Oracle DBA Interview Question and Answers.

$ grep -o -b -i "sql" ora_dba.txt

13:SQL

79:sql

267:sql

Matching the lines that start with a string.

$ grep "^Finally" ora_dba.txt

Finally, linux, sql and Oracle DBA Interview Question and Answers.

Matching the lines that end with a string

$ grep "examples.$" ora_dba.txt learn linux, sql and oracle 10G administration step by step with examples.

Learn Linux, Sql and Oracle 11g Administration Step by Step with examples.

egrep egrep is used to search for a pattern using extended regular expressions. egrep is essentially the same as running grep with the -E option.

Search with multiple words.

$ egrep "CLERK|ANALYST|SALESMAN" emplist.txt

SMITH CLERK 20

ALLEN SALESMAN 30

WARD SALESMAN 30

MARTIN SALESMAN 30

SCOTT ANALYST 20

TURNER SALESMAN 30

ADAMS CLERK 20

JAMES CLERK 30

FORD ANALYST 20

MILLER CLERK 10

Search with multiple words by ignoring case sensitive.

$ egrep -i "manager|president" emplist.txt

BLAKE MANAGER 30

CLARK MANAGER 10

KING PRESIDENT10

Reverse search by using egrep

-v

$ egrep -v

"MANAGER|ANALYST" emplist.txt

ENAME JOB DEPTNO

------

SMITH CLERK 20

ALLEN SALESMAN 30

WARD SALESMAN 30

MARTIN SALESMAN 30

KING PRESIDENT 10

TURNER SALESMAN 30

ADAMS CLERK 20

JAMES CLERK 30

MILLER CLERK 10

Sed is a Stream Editor used for modifying the fles in linux/unix.

Sed command is mostly used to replace the text in a fle. The below simple sed command replaces the word "unix" with "linux" in the fle.

$ cat sample.txt.

$ sed 's/unix/linux/' sample.txt

Here, "s" specifes the substitution operation.

The "/" are delimiters. The "unix" is the search pattern and the "linux" is the replacement string.

We can replace the string at any particular occurrence of the word.

The following example replaces the second occurrence of the word "unix" with "linux" in a line.

$ sed 's/unix/linux/2' sample.txt

The substitute fag /g (global replacement) specifes the sed command to replace all the occurrences of the string in the line.

$ sed 's/unix/linux/g' sample.txt

We can replace the string from nth occurrence to all occurrences in a line.

$ sed 's/unix/linux/2g' sample.txt

Note : To replace /, You can use any delimiter other than the slash.

If you want to change the web url to another url:

$ sed 's/http:\/\//www/' test.txt sed 's_http://_www_' test.txt sed 's|http://|www|' test.txt

There might be some cases where you want to search for the pattern and replace that pattern by adding some extra characters to it. In such cases & comes in handy. The & represents the matched string.

$ sed 's/unix/--&--/' sample.txt

$ sed 's/unix/--&&--/' sample.txt

The /p print fag prints the replaced line twice on the terminal.

$ sed 's/unix/linux/p' sample.txt

Use the -n option along with the /p print fag to display only the replaced lines.

$ sed -n 's/unix/linux/p' sample.txt

Here the -n option suppresses the duplicate rows generated by the /p fag and prints the replaced lines only one time.

You can run multiple sed commands by piping the output of one sed command as input to another sed command.

$ sed 's/unix/linux/' sample.txt | sed 's/os/operating system/'

Sed provides -e option to run multiple sed commands in a single sed command. The above output can be achieved in a single sed command as shown below.

$ sed -e 's/unix/linux/' -e 's/os/operating system/' sample.txt

We can restrict the sed command to replace the string on a specifc line number.

$ sed '2 s/unix/linux/' sample.txt

We can specify a range of line numbers to the sed command for replacing a string.

$ sed '1,3 s/unix/linux/' sample.txt

$ sed '2,$ s/unix/linux/' sample.txt.

Here $ indicates the last line in the fle. So the sed command replaces the text from second line to last line in the fle.

We can add a new line after a pattern match is found. The "a" command to sed tells it to add a new line after a match is found.

$ sed '/unix/ a "New Line" ' sample.txt

"New Line"

We can add a new line before a pattern match is found. The "i" command to sed tells it to add a new line before a match is found.

"New Line" unix linux which one you choose.

The sed command can be used to replace an entire line with a new line. The "c" command to sed tells it to change the line.

$ sed '/unix/ c "Change Line" ' sample.txt

"Change Line"

The sed command can be used to convert the lower case letters to upper case letters by using the transform "y" option.

$ sed 'y/ul/UL/' sample.txt

$ sed 'y/UL/ul/' sample.txt

Awk stands for the names of its authors “Aho, Weinberger, and Kernighan”. It allows easy manipulation of structured data and the generation of formatted reports.

It searches one or more fles to see if they contain lines that match with the specifed patterns and then perform associated actions.

By default Awk prints every line from the fle.

$ awk '{print;}' emp.txt

101 Scott Manager Production $5,000

102 Jason Developer Technology $5,500

103 Sanjay Sysadmin Technology $7,000

104 Nisha Manager Marketing $9,500

105 Randy DBA Technology $6,000

To print the lines which matches with the pattern:

$ awk '/Technology/' emp.txt

102 Jason Developer Technology $5,500

103 Sanjay Sysadmin Technology $7,000

105 Randy DBA Technology $6,000

Awk has a number of built in variables.

For each record i.e line, it splits the record delimited by whitespace character by default and stores it in the $n variables.

To print 2nd and 3rd columns:

$ awk '{print $2,$3;}' emp.txt

Scott Manager

Jason Developer

Sanjay Sysadmin

Nisha Manager

$ awk '{print $2,$NF;}' emp.txt

Scott $5,000

Jason $5,500

Sanjay $7,000

Nisha $9,500

Randy $6,000

NF is a built in variable which represents the total number of felds in a record. $0 represents whole line.

Awk has two important patterns which are specifed by the keyword called BEGIN and END.

Actions specifed in the BEGIN section will be executed before start reading the lines from the input. END actions will be performed after completing the reading and processing the lines from the input.

$ awk 'BEGIN {print "Name\tDesignation\tDepartment\tSalary";}

{print $2,"\t",$3,"\t",$4,"\t",$NF;}

END{print "Report Generated\n------";

}' emp.txt

Name Designation Department Salary

Scott Manager Production $5,000

Jit Developer Technology $5,500

Sanjay Sysadmin Technology $7,000

Bhanu Manager Marketing $9,500

Adittya DBA Technology $6,000

Report Generated

Conditions in awk

To fnd the employees who have employee id greater than 103.

$ awk '$1 >103' emp.txt

104 Nisha Manager Marketing $9,500

105 Randy DBA Technology $6,000

Check for patterns in particular felds and print matched lines.

$ awk '$4 ~/Technology/' emp.txt

102 Jason Developer Technology $5,500

105 Randy DBA Technology $6,000

Operator ~ is for comparing with the regular expressions. If it matches the default action i.e print whole line will be performed.

$ awk 'BEGIN { count=0;}

> $4 ~ /Technology/ { count++; }

> END { print "Number of employees in Technology Dept =",count;}' emp.txt

Number of employees in Technology Dept =

The Linux fnd command is very useful to search for fles from the command line. It can be used to fnd fles based on a variety of flters and parameters like type, name, modifcation date/time, size etc..

Finding by Name

To fnd a fle by name

$ fnd -name install.log

./install.log

To fnd a fle by name, but ignore the case of the query

$ fnd -iname install.log

./Install.log

./install.log

You can invert the search with "-not" or "!". If you use "!", you must escape the character so that bash does not try to interpret it before fnd can act.

$ fnd -not -name install.log

./Install.log

./anaconda-ks.cfg

./.xauthntfvB2

./.cshrc

./.bash_profle

./.xauthmlFVbC

- or -

$ fnd \! -name install.log

./Install.log

./anaconda-ks.cfg

./.xauthntfvB2

./.bash_profle

./.xauthmlFVbC

List all fles in current and sub directories

$ fnd

./Install.log

./anaconda-ks.cfg

./.xauthntfvB2

./.cshrc

./.bash_profle

./.xauthmlFVbC

./install.log

./install.log.syslog

./.bash_history

The command is same as the following

$ fnd . fnd . -print

Search specifc directory or path

$ fnd /home/myftp/

/home/myftp/

/home/myftp/.bash_profle

/home/myftp/.mozilla

/home/myftp/.mozilla/extensions

/home/myftp/.mozilla/plugins

/home/myftp/.bash_logout

/home/myftp/ftp.txt

/home/myftp/.bashrc

/home/myftp/.gnome2

You can specify the type of fles you want to fnd with the "-type" parameter. fnd -type type_descriptor query

Some of the most common descriptors that you can use to specify the type of fle are here f: regular fle d: directory l: symbolic link c: character devices b: block devices

extension.

If we wanted to fnd all the directories under /home/myftp, issue below command.

$ fnd /home/myftp -type d

/home/myftp

/home/myftp/.mozilla

/home/myftp/.mozilla/extensions

/home/myftp/.mozilla/plugins

/home/myftp/.gnome2

We can also use wildcards, search for all fles that end in ".log"

$ fnd / -type f -name "*.log"

Hidden fles on linux begin with a period. So it's easy to mention that in the name criteria and list all hidden fles.

$ fnd ~ -type f -name ".*"

Limit depth of directory traversal

The fnd command by default travels down the entire directory tree recursively, which is time and resource consuming. However the depth of directory traversal can be specifed. For example we don't want to go more than 2 or 3 levels down in the sub directories. This is done using the maxdepth option.

$ fnd ./test -maxdepth 2 -name "*.php"

./test/subdir/how.php

./test/cool.php fnd ./test -maxdepth 1 -name *.php

The second example uses maxdepth of 1, which means it will not go lower than 1 level deep, either only in the current directory.

This is very useful when we want to do a limited search only in the current directory or max 1 level deep sub directories and not the entire directory tree which would take more time.

Just like maxdepth there is an option called mindepth which does what the name suggests, that is, it will go at least N level deep before searching for the fles.

$ fnd -mindepth 4 -name fle

You can combine the min and max depth parameters to focus in on a narrow range:

$ fnd -mindepth 2 -maxdepth 3 -name fle

Combine multiple search criteria

It is possible to use multiple criterias when specifying name and inverting. For example

$ fnd ./test -name 'abc*' ! -name '*.php'

./test/abc.txt

./test/abc

The above fnd command looks for fles that begin with abc in their names and do not have a php extension.

This is an example of how powerful search expressions can be build with the fnd command.

OR operator

When using multiple name criterias, the fnd command would combine them with AND operator, which means that only those fles which satisfy all criterias will be matched. However if we need to perform an

OR based matching then the fnd command has the "o" switch.

$ fnd -name '*.php' -o -name '*.txt'

./abc.txt

./subdir/how.php

./abc.php

./cool.php

The above command search for fles ending in either the php extension or the txt

Filtering by Time

$ fnd / -mtime 50

Find all fles that were accessed in the last 50 days.

$ fnd / -atime 50

Find all fles that were modifed between 50 to 100 days ago.

$ fnd / -mtime +50 –mtime -100

Find fles modifed within the last 1 hour.

$ fnd /home/oracle -cmin -60

To fnd all the fles which are modifed in last 1 hour.

$ fnd / -mmin -60

To fnd all the fles which are accessed in last 1 hour.

$ fnd / -amin -60

Filtering by Size

To fnd all 50MB fles

$ fnd / -size 50M

To fnd all the fles which are greater than 50MB and less than 100MB.

$ fnd / -size +50M -size -100M

The following command uses the "empty" option of the fnd command, which fnds all fles that are empty. fnd /tmp -type f -empty

To fle all empty directories use the type "d".

$ fnd ~/ -type d -empty

Finding by Owner

To fnd fle under /root directory of owner root.

$ fnd /root -user root

We could also specify the name of the fle or any name related criteria along with user criteria

$ fnd /root -user root -name "*cfg"

Its very easy to see, how we can build up criteria after criteria to narrow down our search for matching fles.

$ fnd /var/www -group developer

Did you know you could search your home directory by using the ~ symbol

$ fnd ~ -name "test.xml"

Finding by Permissions

The following command searches for fles with the permission 0664

$ fnd -type f -perm 0664

Inversion can also be applied to permission checking.

$ fnd -type f ! -perm 0777

Find fles with sgid/suid bits set

The following command fnds all fles with permission 644 and sgid bit set.

$ fnd / -perm 2644

Similarly use 1664 for sticky bit. The perm option also supports using an alternative syntax instead of octal numbers.

$ fnd / -maxdepth 2 -perm /u=s 2>/dev/null

Note that the "2>/dev/null" removes those entries that have an error of "Permission Denied"

Find all Read Only fles.

$ fnd /etc -maxdepth 1 -perm /u=r

The following command will fnd executable fles

$ fnd /bin -maxdepth 2 -perm /a=x

Executing and Combining Find Commands

We can execute an arbitrary helper command on everything that fnd matches by using the "-exec" parameter.

$ fnd fnd_parameters -exec command_and_params {} \;

The "{}" is used as a placeholder for the fles that fnd matches. The "\;" is used so that fnd knows where the command ends.

We could fnd the fles in the previous section that had "644" permissions and modify them to have "664" permissions.

$ fnd . -type f -perm 644 -exec chmod 664 {} \;

$ fnd . -type d -perm 755 -exec chmod 700 {} \;

If you want to chain different results together, you can use the "-and" or

"-or" commands. The "-and" is assumed if omitted.

$ fnd . -name fle1 -or -name fle9

Delete all matching fles or directories

$ fnd /home/myftp/-type f -name *.txt -size +10M -exec rm -f {} \;

Linux locate command

An alternative to using fnd is the locate command. This command is often quicker and can search the entire fle system with ease.

To fnd fles with locate, simply use this syntax:

$ locate install.log

You can use the "-b" for only searching the "basename".

$ locate -b install.log

To have locate only return results that still exist in the flesystem (that were not remove between the last "updatedb" call and the current "locate" call), use the "-e" fag.

$ locate -e install.log

To see statistics about the information that locate has cataloged, use the "-S" option:

$ locate -S

Database /var/lib/mlocate/mlocate.db:

13,269 directories

112,119 fles

6,233,493 bytes in fle names

2,700,931 bytes used to store database

Find and locate are powerful commands that can be strengthened by combining them with other utilities through pipelines. Experiment with fltering by using commands like wc, sort and grep.

Using tar

Create tar Archive File

Below command will create a tar archive fle myarchives.tar for notes.txt and imp.txt fles in current working directory.

$ tar -cvf myarchives.tar notes.txt imp.txt

Let's discuss the each option we have used in the above command for creating tar archive fle. c - Creates a new .tar archive fle. v - Verbosely show the .tar fle progress. f- Filename type of the archive fle.

Create tar.gz Archive File

To create a compressed gzip archive fle we use the option as z. Below command will create a compressed documents.tar.gz fle for the directory /home/docs.

$ tar cvzf documents.tar.gz /home/docs/

Create tar.bz2 Archive File

The bz2 feature compresses and creates archive fle less than the size of the gzip. The bz2 compression takes more time to compress and decompress fles as compared to gzip which takes less time. To create highly compressed tar fle we use option as j. The following example of command will create a logfles.tar.bz2 fle for a directory

/home/logs/.

$ tar cvfj logfles.tar.bz2 /home/logs/

Untar tar Archive File

To untar or extract a tar fle, just issue following command using option x (extract). For example the below command will untar the fle myarchives.tar in present working directory. If you want untar in a different directory

$ tar -xvf myarchives.tar

$ tar -xvf myarchives.tar -C /home/learning/docs/

Uncompress tar.gz Archive File

To Uncompress tar.gz archive fle, just run following command. If would like to untar in different directory just use option -C and the path of the directory, like we show in the above example.

$ tar -xvf documents.tar.gz

Uncompress tar.bz2 Archive File

To Uncompress highly compressed tar.bz2 fle, just use the following command. The below example command will untar all the .fv fles from the archive fle.

$ tar -xvf logfles.tar.bz2

List Content of tar Archive File

To list the content of tar archive fle, just run the following command with option t (list content). The below command will list the content of uploadprogress.tar fle.

$ tar -tvf myarchives.tar

List Content tar.gz Archive File

Use the following command to list the content of tar.gz fle.

$ tar -tvf documents.tar.gz

List Content tar.bz2 Archive File

To list the content of tar.bz2 fle, issue the following command.

$ tar -tvf logfles.tar.bz2

Untar Single fle from tar File

To extract a single fle called cleanfles.sh from cleanfles.sh.tar use the following command.

$ tar -xvf myarchives.tar notes.txt

Untar Single fle from tar.gz File

To extract a single fle tecmintbackup.xml from tecmintbackup.tar.gz archive fle, use the command as follows.

$ tar -zxvf documents.tar.gz "/home/docs/contents.txt"

To extract a single fle called index.php from the fle Phpfles-org.tar.bz2 use the following option.

$ tar -jxvf logfles.tar.bz2 "home/logs/error.log.2015-06-06"

Untar Multiple fles from tar, tar.gz and tar.bz2 File

To extract or untar multiple fles from the tar, tar.gz and tar.bz2 archive fle. For example the below command will extract “fle 1” “fle 2” from the archive fles.

$ tar -xvf myarchives.tar notes.txt imp.txt

$ tar -zxvf documents.tar.gz /home/docs/*

$ tar -jxvf logfles.tar.bz2 /home/logs/*.log

Add Files or Directories to tar Archive File

To add fles or directories to existing tar archived fle we use the option r (append). For example we add fle xyz.txt and directory php to existing myarchives.tar archive fle.

$ tar -rvf myarchives.tar xyz.txt

$ tar -rvf myarchives.tar php

The tar command don't have a option to add fles or directories to a existing compressed tar.gz and tar.bz2 archive fle. If we do, we will get errors.

How To Verify tar, tar.gz and tar.bz2 Archive File

To verify any tar or compressed archive fle we use option as W (verify). To do, just use the following examples of command.

Note : You cannot do verifcation on a compressed ( *.tar.gz, *.tar.bz2 ) archive fle.

$ tar tvfW myarchives.tar

Check the Size of the tar, tar.gz and tar.bz2 Archive File

To check the size of any tar, tar.gz and tar.bz2 archive fle, use the following command. For example the below command will display the size of archive fle in Kilobytes (KB).

$ tar -czf - myarchives.tar | wc -c

12820480

$ tar -czf - documents.tar.gz | wc -c

$ tar -czf - logfles.tar.bz2 | wc -c

20480

Tar Usage and Options c - create an archive fle. x - extract an archive fle. v - show the progress of archive fle. f - flename of archive fle. t - viewing content of archive fle. j - flter archive through bzip2. z - flter archive through gzip. r - append or update fles or directories to existing archive fle.

W - Verify an archive fle.

Using zip

Compress fles using zip command

$ zip fles.zip fle1 fle2 fle3 or to zip a directory

$ zip -r dir1.zip dir1

Decompress fles using unzip command

$ unzip fles.zip

Using gzip

Compress fles using gzip

Syntax : gzip flename

$ gzip mydocument.txt

Decompress fle using gzip command

$ gzip -d mydocument.txt.gz

Force a fle to be Compressed

$ gzip -f mydocument.txt

How to keep the uncompressed fle By default when you compress a fle using the gzip command you end up with

If you want to compress the fle and keep the original fle you have to run the following command:

$ gzip -k mydocument.txt

To get the list of statistics run the following command:

$ gzip -l mydocument.txt.gz

The information returned by the above command is as follows:

Compressed size

Uncompressed size

Ratio as a percentage

Uncompressed flename

Compress Every File In A Folder And Sub directories

You can compress every fle in a folder and its sub folders by using the following command:

$ gzip -r documents

How to test the validity of a compressed fle

If you want to check that a fle is valid, you can run the following command:

$ gzip -t mydocuments.txt

If the fle is valid there will be no output.

How to change the compression level

You can compress a fle in different ways. For instance, you can go for a smaller compression which will work faster or you can go for maximum compression which has the trade off of taking longer to run.

To get minimum compression at the fastest speed run the following command:

$ gzip -1 notes.txt

To get maximum compression at the slowest speed run the following command:

$ gzip -9 docs.txt

You can vary the speed and compression level by picking different numbers between 1 and 9.

Regular Expressions (RegEx or RegExp)

A Regular expression (sometimes called a rational expression) is a string that describes a search pattern used when working with text. Regular expressions can be used with grep (the global regular expression parser) and sed (stream editor) and also used in shell scripting , Perl, Java, python and so on.

A Regular Expression contains one or more of the following:

(.) dot

It will match any single character and equivalent to ? (question mark) in standard wildcard expressions.

\ (backslash) backslash is the escape character i.e. to protect a subsequent special character. Thus, "\\" searches for a backslash. Note you may need to use quotation marks and backslash(es).

(.*) (dot and asterisk)

Used to match any string and equivalent to * in standard wildcards.

* (asterisk)

The preceding item will be matched zero or more times.

{N}

The preceding item is matched exactly N times.

{N,}

The preceding item is matched N or more times.

{N,M}

The preceding item is matched at least N times, but not more than M times.

^ (caret)

It means "the beginning of the line". So "^a" means fnd a line starting with an "a".

$ (dollar sign)

It means the end of the line". So "a$" means fnd a line ending with an "a".

- (hyphen)

Used to represent range.

This makes a logical OR relationship between wildcards.

Wildcards

A wildcard is a character that can be used as a substitute for any class of characters. Wildcards are useful to perform actions on more than one fle at a time.

Three types of wildcards are used with Linux commands. Those are

Star Wildcard

Question Mark Wildcard

Square Brackets Wildcard

Star Wildcard ( * )

The character * matches zero or more characters in a fle or directory name. Here are the examples.

$ ls a* : It displays all fles starting letter 'a'

$ ls b*t : It displays all fles starting letter is 'b' and ending letter 't'

$ ls *g : It displays all fles ending letter 'g'

$ ls *.txt: It displays all text fles (.txt)

$ ls *.html : It displays all html fles (.html)

$ ls */* : It displays all fles in all directories which exist in current directory.

Question Mark Wildcard ( ? )

The character "?" can represent any single character. Below are the examples.

$ ls a?c : It displays all three character length fles but starting letter is 'a' and ending letter is 'c'.

$ ls b??k : It displays all four character fles or directories but starting letter is 'b' and ending letter is 'k'.

Square Brackets Wildcard []

It is used to specify range. It allows you to limit to a subset of characters. Below are the examples.

$ ls [aeiou]* : It displays all fles but frst character of the flename to listed must be any of the letters given within the square bracket and remaining can be anything. ls [!aeiou]* It displays all fles whose frst character is anything others than letters given in the square bracket.

$ ls [k-v]* : It displays all fles whose starting letter is between k an v.

$ ls *[0-9]* : It displays all fles whose name contains numbers from 0 to 9.

Before confgurations, we should know about some important fles and directories.

/etc/resolv.conf is a fle which keeps the address of DNS server to which the clients will be accessing to resolve IP to hostname and hostname to IP.

# cat /etc/resolv.conf

# Generated by NetworkManager domain localdomain search localdomain nameserver 192.168.32.128

/etc/hosts is fle which is responsible for resolving hostname into IP locally, in other word it acts as local

DNS if DNS server is not accessible.

# cat /etc/hosts

127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4

::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.32.128 learndba learndba.com

For Red Hat/Fedora/CentOS

/etc/sysconfg/network is fle which keeps the information about the hostname assigned to the system. If you want to change the hostname permanently, you need to change the hostname in this fle.

# cat /etc/sysconfg/network

NETWORKING=yes

HOSTNAME=learndba

/etc/sysconfg/network-scripts/ is the directory which keeps the confguration of network devices connected to the system.

# cd /etc/sysconfg/network-scripts/

For debian/Ubuntu

/etc/network/interfaces is a fle which keeps the information about network confguration and devices. ex.

Static IP and info, DHCP, etc.

# cat /etc/network/interfaces

# interfaces(5) fle used by ifup(8) and ifdown(8) auto lo iface lo inet loopback auto eth0 iface eth0 inet dhcp

Static IP address assignment

Command Line

# ifconfg eth0 192.168.32.10 netmask 255.255.255.0 broadcast 192.168.32.255

Network IP aliasing

Assign more than one IP address to one ethernet card.

# ifconfg eth0 192.168.32.10 netmask 255.255.255.0 broadcast 192.168.32.255

# ifconfg eth0:0 192.168.32.20 netmask 255.255.255.0 broadcast 192.168.32.255

# ifconfg eth0:1 192.168.32.30 netmask 255.255.255.0 broadcast 192.168.32.255

Check with ifconfg command

# inconfg

Changing the hostname

# hostname learndba

# cat /etc/sysconfg/network

NETWORKING=yes

HOSTNAME=learndba

# vi /etc/sysconfg/network

Change HOSTNAME value

NETWORKING=yes

HOSTNAME=linuxlearning

Restart the system with init 6 command and check hostname.

# init 6

# hostname linuxlearning

Network confguration utility setup or $ system-confg- network commands will open a text base utility.

# setup

Move the cursor to Network confguration and press Enter

Move the cursor to Device confguration and press Enter

Note : If system-confg-network command is used, it will directly take you to above position.

Now select the NIC adapter i.e and press Enter

Assign IP address and other details per your requirement , move cursor to OK and press Enter.

Move the cursor to Save and press Enter to save the changes in device confguration.

Once again move cursor to Save & Quit and press Enter.

Finally move the cursor to Quit and press enter to quit the utility.

Now restart the network service and check IP address.

# service network restart

If the changes are not refected with above service restart, restart network manager.

# service NetworkManager restart

Check IP address with ifconfg command.

If the server is in the remote location, use mii-tool to check whether the cable is connected or not.

# mii-tool eth0 eth0: negotiated 100baseTx-FD, link ok

To know more about the NIC card/adapter, use below command.

# ethtool eth0

Chapter 34 Troubleshooting

Computers are connected in a network to exchange data and resources with each other using a data link. Maintaining the network up and running is a task of Network Administrator's job. Below are the commands to troubleshoot networks on linux servers. ping

You can test network connection. ping also can be used for basic DNS diagnostics, we can ping hosts by IP address or by its hostname and then decide if DNS works at all. ifconfg

Ifconfg is used to confgure the kernel-resident network interfaces. It is used at boot time to set up interfaces as necessary. After that, it is usually only needed when debugging or when system tuning is needed. If no arguments are given, ifconfg displays the status of the currently active interfaces. If a single interface argument is given, it displays the status of the given interface only; if a single -a argument is given, it displays the status of all interfaces, even those that are down. Otherwise, it confgures an interface. traceroute traceroute tracks the route packets taken from an IP network on their way to a given host. It utilizes the IP protocol’s time to live (TTL) feld and attempts to elicit an ICMP TIME_EXCEEDED response from each gateway along the path to the host. tracepath

It traces a path to destination discovering MTU along this path. It uses UDP port port or some random port. It is similar to traceroute, only does not not require superuser privileges and has no fancy options. tracepath6 is a good replacement for traceroute6 and classic example of application of Linux error queues. The situation with tracepath is worse, because commercial IP routers do not return enough information in icmp error messages. Probably, it will change, when they will be updated. For now it uses Van Jacobson’s trick, sweeping a range of UDP ports to maintain trace history. dig - DNS lookup utility dig (domain information groper) is a fexible tool for interrogating DNS name servers. It performs DNS lookups and displays the answers that are returned from the name server(s) that were queried. Most DNS administrators use dig to troubleshoot DNS problems because of its fexibility, ease of use and clarity of output. Other lookup tools tend to have less functionality than dig. nslookup

© CEHPOINT E-LEARNING 2020 ( 119 ) Nslookup is a program to query Internet domain name servers. Nslookup has two modes: interactive and non-interactive. Interactive mode allows the user to query name servers for information about various hosts and domains or to print a list of hosts in a domain. Non-interactive mode is used to print just the name and requested information for a host or domain. dmesg dmesg is used to examine or control the kernel ring buffer. The program helps users to print out their bootup messages. Instead of copying the messages by hand, the user needs only: dmesg > boot.messages and mail the boot.messages fle to whoever can debug their problem. dmesg | less or dmesg | tail or dmesg | grep -i error - for understanding what the Linux kernel thinks about some trouble. netstat

Print network connections, routing tables, interface statistics, masquerade connections, and multicast memberships Netstat prints information about the Linux networking subsystem. The type of information printed is controlled by the frst argument, as follows:

(none)

By default, netstat displays a list of open sockets. If you don't specify any address families, then the active sockets of all confgured address families will be printed.

--route , -r

Display the kernel routing tables.

--groups , -g

Display multicast group membership information for IPv4 and IPv6.

--interfaces=iface , -I=iface , -i

Display a table of all network interfaces, or the specifed iface.

--masquerade , -M

Display a list of masqueraded connections.

--statistics , -s

Display summary statistics for each protocol. ss

Another utility to investigate sockets. ss is used to dump socket statistics. It allows showing information similar to netstat. It can display more TCP and state information than other tools. iptables

Administration tool for IPv4 packet fltering and NAT Iptables is used to set up, maintain, and inspect the tables of IPv4 packet flter rules in the Linux kernel. Several different tables may be defned. Each table contains a number of built-in chains and may also contain user-defned chains. Each chain is a list of rules which can match a set of packets. Each rule specifes what to do with a packet that matches. This is called a 'target', which may be a jump to a user-defned chain in the same table.

© CEHPOINT E-LEARNING 2020 ( 120 ) To investigate frewall issues on Linux: iptables -nvL shows how many packets are matched by each rule (iptables -Z to zero the counters). The LOG target inserted in the frewall chains is useful to see which packets reach them and how they have already been transformed when they get there. To get further NFLOG (associated with ulogd) will log the full packet. iptables-save iptables-save is used to dump the contents of an IP Table in easily parsable format to STDOUT. Use I/O-redirection provided by your shell to write to a fle. ethtool ethtool is used to query and control network device driver and hardware settings, particularly for wired Ethernet devices. devname is the name of the network device on which ethtool should operate. openssl

OpenSSL is a cryptography toolkit implementing the Secure Sockets Layer (SSL v2/v3) and Transport Layer Security (TLS v1) network protocols and related cryptography standards required by them. The openssl program is a command line tool for using the various cryptography functions of OpenSSL’s crypto library from the shell. It can be used for Creation and management of private keys, public keys and parameters Public key cryptographic operations

Creation of X.509 certifcates, CSRs and CRLs

Calculation of Message Digests

Encryption and Decryption with Ciphers

SSL/TLS Client and Server Tests

Handling of S/MIME signed or encrypted mail

Time Stamp requests, generation and verifcation

Arp

Arp manipulates the kernel’s ARP cache in various ways. The primary options are clearing an address mapping entry and manually setting up one. For debugging purposes, the arp program also allows a complete dump of the ARP cache. route

Route manipulates the kernel’s IP routing tables. Its primary use is to set up static routes to specifc hosts or networks via an interface after it has been confgured with the ifconfg(8) program. When the add or del options are used, route modifes the routing tables. Without these options, route displays the current contents of the routing tables. strace strace is a useful diagnostic, instructional, and debugging tool. System administrators, diagnosticians and trouble-shooters will fnd it invaluable for solving problems with programs for which the source is not readily available since they do not need to be recompiled in order to trace them. Students, hackers and the overly-curious will fnd that a great deal can be learned about a system and its system calls by tracing even ordinary programs. And programmers will fnd that since system calls and signals are events that

© CEHPOINT E-LEARNING 2020 ( 121 ) happen at the user/kernel interface, a close examination of this boundary is very useful for bug isolation, sanity checking and attempting to capture race conditions. tcpdump

Tcpdump prints out a description of the contents of packets on a network interface that match the boolean expression. It can also be run with the -w fag, which causes it to save the packet data to a fle for later analysis, and/or with the -r fag, which causes it to read from a saved packet fle rather than to read packets from a network interface. In all cases, only packets that match expression will be processed by tcpdump.

Linux is a multitasking operating system. Linux creates a process whenever a program is launched, either by user or by OS itself. If a process runs and terminates properly, then everything is fne. If not, You need to manage the processes by commands. If we would like to get a snapshot of what is currently happening on the system we may use a program called top

Type of processes:

Foreground Processes: They run on the screen and need input from the user.

Background Processes: They run in the background and usually do not need user input.

Let's start with a process management commands. ps To display the currently working processes

$ ps

PID TTY TIME CMD

2896 pts/0 00:00:00 bash

2911 pts/0 00:00:00 ps

Display all processes

$ ps ax

PID TTY STAT TIME COMMAND

1 ? Ss 0:04 /sbin/init

2 ? S 0:00 [kthreadd]

3 ? S 0:00 [migration/0]

4 ? S 0:00 [ksoftirqd/0]

5 ? S 0:00 [migration/0]

6 ? S 0:00 [watchdog/0]

7 ? S 0:00 [migration/1]

$ ps -ef

Use the "u" option or "-f" option to display detailed information about the processes

$ ps -ef -f

Display process by user

$ ps -f -u ops

Show process by name

$ ps -C metacity

Show process by process id

$ ps -f -p 2609,2704,2585

Show process by using grep (searching)

$ ps -ef | grep "metacity"

Show process by sorting on cpu usage percentage

$ ps aux --sort=-pcpu,+pmem |head

Customizing the ps output format

$ ps -eo uname,pid,start_time,time,args |head top Display all running process

$ top kill pid terminate a process

$ kill -9 2895 killall proc Kill all the process named proc

$ killall -9 frefox pkill pattern Will kill all processes matching the pattern

$ pkill -USR1 sample bg List stopped or background jobs,resume a stopped job in the background

$ bg fg Brings the most recent job to foreground

$ fg

A Linux service is the program that runs in the background, waiting to be used or carrying out essential tasks. For example the Apache server has a demon called httpd that listens on port 80 on your computer and when it receives a request for a page it sends the appropriate data back to the client machine. Many services are required to rull all the time however many can be safely turned off for both security reasons as running unnecessary services opens more doors into your computer, but also for performance reasons. It may not make much difference but your computer should boot slightly faster with less services it has to start on boot. One of the techniques in every Linux administrator's toolbox to improve box security is to turn off unneeded services. Below are two commands used to control services. service - It is used for starting and stopping of services during a session. If you start Apache with this command it will continue to run on next reboot and it will not start automatically. chkconfg - It controls which services are set to start on boot, by their name these settings are saved and are applied at next boot. Changing these settings will not start the service immediately. It will just fag them to start from the next boot.

Commands used for maintaining a service.

To check the status of a service.

# service (name of the service) status

TO start a service.

# service (name of the service) start

To stop a service

# service (name of the service) stop

To reload a service

# service (name of the service) reload

To restart a service

# service (name of the service) restart

Commands used for service availability.

To check availability of service

# chkconfg --list

To make the service available after reboot

# chkconfg service on

To make service unavailable after reboot

# chkconfg service off

Check status of SSH service (sshd).

To Stop SSH service

# service sshd stop

Stopping sshd: [ OK ]

To start SSH Service

# service sshd start

Starting sshd: [ OK ]

Reload service may be required after doing some changes in confg fle.

# service sshd reload

Reloading sshd: [ OK ]

To restart any service required when reload doesn't work.

# service sshd restart

Stopping sshd: [ OK ]

Starting sshd: [ OK ]

Check status of the all service availability.

# chkconfg --list abrt-ccpp 0:off 1:off 2:off 3:on 4:off 5:on 6:off abrtd 0:off 1:off 2:off 3:on 4:off 5:on 6:off acpid 0:off 1:off 2:on 3:on 4:on 5:on 6:off atd 0:off 1:off 2:off 3:on 4:on 5:on 6:off auditd 0:off 1:off 2:on 3:on 4:on 5:on 6:off

To check particular service

# chkconfg --list httpd httpd 0:off 1:off 2:off 3:off 4:off 5:off 6:off

To make service availability on for httpd

# chkconfg httpd on

# chkconfg --list httpd httpd 0:off 1:off 2:on 3:on 4:on 5:on 6:off

# chkconfg httpd off

# chkconfg --list httpd httpd 0:off 1:off 2:off 3:off 4:off 5:off 6:off

To make service httpd availability on only run level 5

# chkconfg --level 5 httpd on

# chkconfg --list httpd httpd 0:off 1:off 2:off 3:off 4:off 5:on 6:off

The same can be done for making service unavailable in a particular run level.

YUM (Yellowdog Updater Modifed) is an open source command-line as well as graphical based package management tool for RPM based Linux systems.

Yum is the standard method of managing the installation and removal of software. YUM uses numerous third party repositories to install packages automatically by resolving their dependencies issues.

Yum will work only from Centos 5/ Red hat 5 and latest versions of fedora. For old releases like RHEL

4you need to use up2date command to update your rpm based packages. Yum uses confguration fle at

/etc/yum.conf

Install a Package with YUM

To install a package called Firefox, just run the below command it will automatically fnd and install all required dependencies for Firefox.

# yum install frefox

The above command will ask confrmation before installing any package on your system. If you want to install packages automatically without asking any confrmation, use option -y as shown in below example.

# yum -y install frefox

Removing a Package with YUM

To remove a package completely with their all dependencies, just run the following command as shown below.

# yum remove frefox

Above command will ask for confrmation before removing a package. To disable confrmation prompt just add option -y as shown in below.

# yum -y remove frefox

Updating a Package using YUM

If you have outdated version of MySQL package and you want to update it to the latest stable version, just

# yum update mysql

To install a package locally from directory or cd rom.

# yum localinstall fnger-0.17-39.el6.i686.rpm

List a package using YUM

Use the list function to search for the specifc package with name. For example to search for a package called openssh, use the command.

# yum list openssh

To search a specifc version openssh-4.3p2 of the package, use the command.

# yum list openssh-4.3p2

Search for a Package using YUM

If you don’t remember the exact name of the package, then use search function to search all the available packages to match the name of the package you specifed. For example, to search all the packages that matches the word .

# yum search vsftpd

If you would like to know information about a package before installing it, just run the below command.

# yum info frefox

List all available packages using YUM

To list all the available packages in the Yum database, use the below command.

# yum list | less

List all installed packages using YUM

To list all the installed packages on a system, just issue below command, it will display all the installed packages.

# yum list installed | less

Yum Provides Function

Yum provides a function that is used to fnd which package a specifc fle belongs to. For example, if you would like to know the name of the package that has the /etc/httpd/conf/httpd.conf.

# yum provides /etc/httpd/conf/httpd.conf

Check for available updates using Yum

# yum check-update

Update System using Yum

To keep your system up-to-date with all security and binary package updates, run the following command.

It will install all latest patches and security updates to your system.

# yum update

List all available Group Packages

In Linux, a number of packages are bundled to a particular group. Instead of installing individual packages with yum, you can install particular group that will install all the related packages that belongs to the group. For example to list all the available groups, just issue the following command.

# yum grouplist

Install a Group Packages

To install a particular package group, we use option groupinstall. Fore example, to install “MySQL

Database“, just execute the below command.

# yum groupinstall 'MySQL Database'

Update a Group Packages

To update any existing installed group packages, just run the following command as shown below.

# yum groupupdate 'DNS Name Server'

Remove a Group Packages

To delete or remove any existing installed group from the system, just use below command.

# yum groupremove 'DNS Name Server'

List enabled Yum Repositories

To list all enabled Yum repositories in your system, use the following option.

# yum repolist

List all enabled and disabled Yum Repositories

The following command will display all enabled and disabled yum repositories on the system.

# yum repolist all

To install a particular package from a specifc enabled or disabled repository, you must use –enablerepo option in your yum command. For example to Install PhpMyAdmin 3.5.2 package, just execute the command.

# yum --enablerepo=epel install phpmyadmin

Interactive Yum Shell

Yum utility provides a custom shell where you can execute multiple commands.

# yum shell

Clean Yum Cache

By default yum keeps all the repository enabled package data in /var/cache/yum/ with each sub-directory, to clean all cached fles from enabled repository, you need to run the following command regularly to clean up all the cache and make sure that there is nothing unnecessary space is using. We don’t want to give the output of the below command, because we like to keep cached data as it is.

# yum clean all

View History of Yum

To view all the past transactions of yum command, just use the following command.

# yum history

All software on a Red Hat Enterprise Linux system is divided into RPM packages which can be installed, upgraded or removed. The Red Hat Package

Manager (RPM) is an open packaging system, which runs on Red Hat

Enterprise Linux as well as other Linux and UNIX systems.

- Package installation is never interactive

- Applies to all SW [ core OS and add-ons ]

- No such thing as a patch to a package

RPM packages typically have fle names like foo-1.0-1.i386.rpm. For example,

The Java rpm package name is jre-8u111-linux-i586.rpm. The fle name includes the package name (foo, java), version (1.0, 8), release (1,u111), and architecture (i386,i586).

To install a package, log in as root and use the following command.

If the installation is successful, the following output is displayed

# rpm -Uvh jre-8u111-linux-i586.rpm

Preparing... ############## [100%]

1:jre1.8.0_111 ############## [100%]

If a package of the same name and version is already installed, following output will be displayed

# rpm -Uvh jre-8u111-linux-i586.rpm

Preparing... ############## [100%]

package jre1.8.0_111-1.8.0_111-fcs.i586 is already installed

However, if you want to install the package anyway, you can use the --replacepkgs option or --force, which tells

RPM to ignore the error.

# rpm -ivh --replacepkgs jre-8u111-linux-i586.rpm

Preparing... ############## [100%]

1:jre1.8.0_111 ############## [100%]

# rpm -ivh jre-8u111-linux-i586.rpm --force

Preparing... ############## [100%]

1:jre1.8.0_111 ############## [100%]

If you attempt to install a package that contains a fle which has already been installed by another package, the following is displayed.

# rpm -Uvh foo-1.0-1.i386.rpm

Preparing... ############## [100%] fle /usr/bin/foo from install of foo-1.0-1 conficts with fle from package bar-2.0.20

To make RPM ignore this error, use the -replacefles option

# rpm -ivh --replacefles foo-1.0-1.i386.rpm

RPM packages may sometimes depend on other packages, which means that they require other packages to be installed to run properly. If you try to install a package which has an unresolved dependency, output similar to the following.

error: Failed dependencies:

Suggested resolutions: bar-2.0.20-3.i386.rpm

If you are installing a package, it usually suggests the package(s) needed to resolve the dependency. Download suggested package(s) from Red Hat Network , and add it to the command.

# rpm -ivh foo-1.0-1.i386.rpm bar-2.0.20-3.i386.rpm

If installation of both packages is successful, output similar to the following is displayed

Preparing... ############## [100%]

1:foo ############## [ 100%]

2:bar ############## [100%]

If it does not suggest a package to resolve the dependency, you can try the -redhatprovides option to determine which package contains the required fle. You need the rpmdb-redhat package installed to use this option.

# rpm -q --redhatprovides bar.so.2

If the package that contains bar.so.2 is in the installed database from the rpmdb-redhat package, the name of the package is displayed: bar-2.0.20-3.i386.rpm

To force the installation anyway (which is not recommended since the package may not run correctly), use the

-nodeps option.

# rpm -ivh foo-1.0-1.i386.rpm --nodeps

Uninstalling a package is just as simple as installing one. Use the following command.

# rpm -e jre1.8.0_111

Upgrading a package is similar to installing one. use the following command

# rpm -Uvh jre-8u111-linux-i586.rpm

As part of upgrading a package, RPM automatically uninstalls any old versions.

Note : -U will also install a package even when there are no previous versions of the package installed.

Freshening is similar to upgrading, except that only existing packages are upgraded, use the following command.

# rpm -Fvh jre-8u111-linux-i586.rpm

The rpm -q package name command displays the package name, version, and release number of the installed package package name.

For example, using rpm -q to query installed package jre1.8.0_111 might generate the following output:

# rpm -q jre1.8.0_111 jre1.8.0_111-1.8.0_111-fcs.i586

To check all the installed package on the system, use following command

# rpm -qa nautilus-sendto-2.28.2-3.el6.x86_64 plymouth-scripts-0.8.3-27.el6.x86_64 pinentry-gtk-0.7.6-6.el6.x86_64 dejavu-fonts-common-2.30-2.el6.noarch gedit-2.28.4-3.el6.x86_64 genisoimage-1.1.9-12.el6.x86_64 festvox-slt-arctic-hts-0.20061229-18.el6.noarch foomatic-db-flesystem-4.0-7.20091126.el6.noarch sound-juicer-2.28.1-6.el6.x86_64 urw-fonts-2.4-10.el6.noarc

To check whether a particular package is installed or not, use following command

# rpm -q jre1.8.0_111

jre1.8.0_111-1.8.0_111-fcs.i586

To check whether a package is consistent or not before installing it, use following command on the shell prompt.

# rpm -ivh --test jre-8u111-linux-i586.rpm

Preparing... ############## [100%]

To see the information about installed package, use below command

# rpm -qi jre1.8.0_111

To see the confguration fles of the installed package, use following command

# rpm -qlc jre1.8.0_111

To see the directory with which a particular package is associated.

Chapter 38 Remote connection

There are many ways to access a shell remotely. One of the older ways is to use the telnet program, which is available on most network capable operating systems.

Accessing shell account through the telnet method is not at all secure because everything that you send and receive over the telnet method is visible in plain text on your local network. For this reason, you need a more secure program that telnet to connect to a remote host.

What is SSH

Secure Shell (SSH) is an open source and most trusted network protocol for operating network services securely over an unsecured network. The SSH protocol is a method for secure remote login from one computer to another. It is also used to transfer fles from one computer to another computer over the network using secure copy (SCP) Protocol.

It provides several alternative options for strong authentication, and it protects the communications security and integrity with strong encryption.

The SSH protocol is used in corporate networks for: providing secure access for users and automated processes interactive and automated fle transfers issuing remote commands managing network infrastructure and other mission-critical system components.

Confguration and demon

Confguration fle : /etc/ssh/sshd_confg

SSH demon or service is sshd

# service sshd status openssh-daemon (pid 2130) is running..

Accessing the remote machine using SSH.

To access the remote machine using ssh, the syntax is

# ssh usename@IP Address/Hostname

# ssh [email protected]

RSA key fngerprint is 88:33:32:74:08:aa:1c:2c:54:23:be:e2:ec:52:fb:ff.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added '192.168.150.133' (RSA) to the list of known hosts.

For the frst time, it will ask you if you wish to add the remote host to a list of know_hosts, go ahead and say yes. Then it will ask for a password to login.

To leave the session, just type exit or logout or use shortcut CTRL+D command and you will back to your own machine.

Trusted connection :

Password less login using SSH keys.

If you are dealing with a number of Linux remote servers, then SSH Password-less login is one of the best ways to automate tasks such as automatic backups with scripts, synchronization fles using scp and remote command execution.

By generating SSH keys, a public key and private key, it will work as authorized access and it will not prompt for password.

SSH keys are an implementation of public-key cryptography. They solve the problem of brute-force password attacks by making then computationally impractical. Public key cryptography uses a public key to encrypt data and private key to decrypt data.

Generating SSH key pair.

To generate a pair of public keys use following command.

# ssh-keygen

Generating public/private rsa key pair.

Enter fle in which to save the key (/root/.ssh/id_rsa): [Press enter key]

Enter passphrase (empty for no passphrase): [Press enter key]

Enter same passphrase again: [Press enter key]

Your identifcation has been saved in /root/.ssh/id_rsa.

Your public key has been saved in /root/.ssh/id_rsa.pub.

The key fngerprint is:

0b:51:d3:ac:96:74:0a:e9:b1:9c:f7:74:35:23:d1:15 [email protected]

The key's randomart image is:

+--[ RSA 2048]----+

| .oo .o Eo|

| o.* = o o |

| =.* . . |

| .oSo . |

| . .. |

| . |

| |

+------+

It will prompt above to mention the fle where these keys should be stored, to keep its default directory just press Enter. The default location will be /root/.ssh/ .

# cd /root/.ssh/

# ls id_rsa id_rsa.pub

Copying the public key on client machine.

To copy the server's public key in client system, the command is

# ssh-copy-id -i /root/.ssh/id_rsa.pub 192.168.150.133 [email protected]'s password:

Now try logging into the machine, with "ssh '192.168.150.133'", and check in:

.ssh/authorized_keys to make sure we haven't added extra keys that you weren't expecting.

Now check in the client machine.

# cat /root/.ssh/authorized_keys ssh-rsa

AAAAB3NzaC1yc2EAAAABIwAAAQEA5zWA3lFZ+ZL4Vnlpw8FdBTCMV7NBLSq/B9URVb5NUIMF8w+2zzq

TWh5jC2

+/2cjS1cIYtUPn03FyViKEKnRukI7iCuybTOcGWyoJW10sZIkhO61pRQjj2CH8M755Y970LkbjFU4WY7xXcsQ o5IA+Tymx

ZOKweerW9QDx

AH5E0GJbjZAbQsbhfkm790HEFKETuQsx5ElklChsXbPlkCFlHCnIonJN2ZxUb9wH32GmdHur3YKw5TtToyZ

CyONhxeBh+i

NQ== [email protected]

Try login to the client machine using SSH, check whether it is asking for password.

[root@myserver ~]# ssh 192.168.150.133

Last login: Thu Sep 14 04:26:31 2016 from 192.168.150.1

[root@localhost ~]#

It will not prompt for password once trusted connection is established.

Secure copy or SCP is a means of securely transferring computer fles between a local host and a remote host or between two remote hosts. It is based on the Secure Shell (SSH) protocol.

Copying fle to host, the syntax is scp SourceFile user@host:directory/TargetFile

Local Server:

# scp fle1.txt [email protected]:/root/fle1 fle1.txt 100% 0 0.0KB/s 00:00

Remote server:

# ls fle1 fle1

Copying fle from host, the syntax is scp user@host:directory/SourceFile TargetFile scp -r user@host:directory/SourceFolder TargetFolder

Remote server

# ls features.txt features.txt

Local server:

# ls features.txt features.txt

To copy all fles under directory :

Remote server:

# ll |wc -l

Local Server:

# ll |wc -l

# scp -r [email protected]:/root/ /root/ features.txt 100% 176 0.2KB/s 00:00

Install.log 100% 0 0.0KB/s 00:00 anaconda-ks.cfg 100% 3277 3.2KB/s 00:00

.xauthER2pZD 100% 66 0.1KB/s 00:00

.xauthntfvB2 100% 66 0.1KB/s 00:00 sample.txt 100% 109 0.1KB/s 00:00 fle2 100% 23 0.0KB/s 00:00

.cshrc 100% 100 0.1KB/s 00:00

......

# cd root/

# ll |wc -l

If the remote host uses a port other than the default of 22, it can be specifed in the command. For example, copying a fle from host: scp -P 2222 user@host:directory/SourceFile TargetFile

If we don't have trusted connection then it will prompt for password.

Rsync

Rsync is a utility for effciently transferring and synchronizing fles across computer systems, by checking the timestamp and size of fles.

Rsync is typically used for synchronizing fles and directories between two different systems.

For example, if the command rsync local-fle user@remote-host:remote-fle is run, rsync will use SSH to connect as user to remote-host. Once connected, it will invoke the remote host's rsync and then the two programs will determine what parts of the fle need to be transferred over the connection.

Rsync can also operate in a daemon mode, serving fles in the native rsync protocol.

Basic syntax of rsync command rsync options source destination

Some common options used with rsync commands

-v : verbose

-r : copies data recursively (but don't preserve timestamps and permission while transferring data

-a : archive mode, archive mode allows copying fles recursively and it also preserves symbolic links, fle permissions, user & group ownerships and timestamps

-z : compress fle data

-h : human-readable, output numbers in a human-readable format

Copy/Sync Files and Directory Locally

This following command will sync a single fle on a local machine from one location to another location.

# rsync -zvh backup.tar /tmp/backups/

The following command will transfer or sync all the fles of from one directory to a different directory in the same machine.

Copy/Sync Files and Directory to or From a Server

This following command will sync a directory from a local machine to a remote machine.

# rsync -avz rpmpkgs/ [email protected]:/home/

Copy/Sync a Remote Directory to a Local Machine

# rsync -avzh [email protected]:/root/rpmpkgs /tmp/myrpms

Rsync Over SSH

With rsync, we can use SSH (Secure Shell) for data transfer, using SSH protocol while transferring our data you can be ensured that your data is being transferred in a secured connection with encryption so that nobody can read your data while it is being transferred over the wire on the internet.

To specify a protocol with rsync you need to give the "-e" option with protocol name you want to use. Here in this example, We will be using ssh with -e option and perform data transfer.

# rsync -avzhe ssh [email protected]:/root/install.log /tmp/

Copy a File from a Local Server to a Remote Server with SSH

# rsync -avzhe ssh backup.tar [email protected]:/backups/

Show Progress While Transferring Data with rsync

# rsync -avzhe ssh --progress backup.tar [email protected]:/backups/

Use of -include and -exclude Options

These two options allows us to include and exclude fles by specifying parameters with these option helps us to specify those fles or directories which you want to include in your sync and exclude fles and folders with you don't want to be transferred.

# rsync -avzhe ssh --include '*.xml' --exclude '*.html' [email protected]:/home/sree/ /tmp/

Use of --delete Option

If a fle or directory not exist at the source, but already exists at the destination, you might want to delete that existing fle/directory at the target while syncing. We can use '--delete' option to delete fles that are not there in source directory.

# rsync -avz --delete [email protected]:/var/lib/rpm/

Set the Max Size of Files to be Transferred

You can specify the Max fle size to be transferred or sync. You can do it with "--max-size" option.

Set Bandwidth Limit and Transfer File

# rsync --bwlimit=100 -avzhe ssh [email protected]:/root/ /tmp/

Do a Dry Run with rsync

If you are a newbie and using rsync and don't know what exactly your command is going do. Rsync could really mess up the things in your destination folder and then doing an undo can be a tedious job.

# rsync --dry-run -zvh backup.tar /tmp/backups/

You can see man pages for more options.

FTP (File transfer protocol)

The File Transfer Protocol (FTP) is a standard network protocol used for the transfer fles between one host to another host over a TCP-based network, such as the Internet.

FTP is built on a client-server model architecture and uses separate control and data connections between the client and the server.

FTP users may authenticate themselves with a clear-text sign-in protocol, normally in the form of a username and password, but can connect anonymously if the server is confgured to allow it.

Usually, the FTP server which stores fles to be transferred uses two ports for the transferring purpose, one for commands and the other for sending and receiving data.

FTP may run in active or passive mode, which determines how the data connection is established.

In both cases, the client creates a TCP control connection from a random, usually unprivileged, port N to the FTP server command port 21.

Active mode: In active mode, command connection is initiated by the client, and data connection is initiated by the server. As it actively establishes the data connection with client, this mode is referred to as active. The client opens up a port higher than 1024, and through its connections to the port 21 or the command port of the server.Then the server opens up its port 20 and establishes a data connection to port higher than 1024 of the client.

Passive mode: In passive mode, the server acts entirely passively as the command connection and data connection both initiated and established by the client. In this mode, server listens for incoming requests through its port 21(command port), and when a request is received for the data connection from the client (using a high port), server randomly opens up one of its high ports. Confguration FTP

Confgure FTP Server for downloading fles.

Step 1: Install the package.

Check vsftpd is already installed or not with the below command.

# rpm -q vsftpd package vsftpd is not installed

Download vsftpd rpm package with below command.

# wget

86_64.rpm

After download is completed, install vsftpd with below command.

# yum localinstall vsftpd-2.2.2-6.el6_2.1.x86_64.rpm

Once installation is completed, verify it.

# rpm -q vsftpd vsftpd-2.2.2-6.el6_2.1.x86_64

Step 2: Go to /var/ftp/pub/ and create some fles.

# cd /var/ftp/pub/

# touch fle{1..5}

# ls fle1 fle2 fle3 fle4 fle5

Step 3: Restart the ftp service.

# service vsftpd restart

Shutting down vsftpd: [ OK ]

Starting vsftpd for vsftpd: [ OK ]

Make the service enable even after reboot of the system

# chkconfg vsftpd on

# chkconfg --list vsftpd vsftpd 0:off 1:off 2:on 3:on 4:on 5:on 6:off

Step 4: Check whether ftp package is installed or not.

# rpm -q ftp package ftp is not installed

Download the rpm fle and install.

# wget

# rpm -ivh ftp-0.17-51.1.el6.x86_64.rpm

# rpm -q ftp ftp-0.17-51.1.el6.x86_64

Step 5: Now connect to the ftp server.

Syntax : # ftp << IP Address >>

# ftp 192.168.150.133

Connected to 192.168.150.133 (192.168.150.133).

220 (vsFTPd 2.2.2)

Name (192.168.150.133:root): ftp

331 Please specify the password.Password:

230 Login successful.

Remote system type is UNIX.

Using binary mode to transfer fles. ftp>

Step 6: Go to pub directory, check the fles available. ftp> cd pub

250 Directory successfully changed. ftp> ftp> ls

227 Entering Passive Mode (192,168,32,134,143,45).

150 Here comes the directory listing.

-rw-r--r-- 1 0 0 0 Aug 13 10:03 fle1

-rw-r--r-- 1 0 0 0 Aug 13 10:03 fle2

-rw-r--r-- 1 0 0 0 Aug 13 10:03 fle3

-rw-r--r-- 1 0 0 0 Aug 13 10:03 fle4

-rw-r--r-- 1 0 0 0 Aug 13 10:03 fle5

226 Directory send OK.

Step 7: Download fles.

Syntax: get flename mget fle1 fle2 ... ftp> get fle1 local: fle1 remote: fle1

227 Entering Passive Mode (192,168,150,133,242,175).

150 Opening BINARY mode data connection for fle1 (0 bytes).

226 Transfer complete. ftp> quit

221 Goodbye.

# ls fle1 fle1

# ftp 192.168.150.133

Connected to 192.168.150.133 (192.168.150.133).

220 (vsFTPd 2.2.2)

Name (192.168.150.133:root): ftp

331 Please specify the password.

Password:

230 Login successful.

Remote system type is UNIX.

Using binary mode to transfer fles. ftp> ftp> mget fle2 fle3 ftp> quit

221 Goodbye.

# ls fle2 fle3

# get - to download single fle mget - to download multiple fles.

To exit from ftp server, use exit or bye ftp> quit

221 Goodbye. ftp> bye

221 Goodbye.

To connect to the ftp server graphically, open web browser and type ftp server's ip address as following ftp://192.168.150.133/

To upload the fles in the ftp server, the steps are

Step 1: Create an upload directory in /var/ftp/.

# cd /var/ftp/

# mkdir upload

Step 2: Change the group to ftp and write permissions to the upload directory.

# chgrp ftp upload

# ls -ld upload drwxr-xr-x. 2 root ftp 4096 Sep 19 01:53 upload

# chmod g+w upload

# ls -ld upload drwxrwxr-x. 2 root ftp 4096 Sep 19 01:53 upload

Step 3: Now goto /etc/vsftpd/ and uncomment anon_upload_enable attribute.

# vi vsftpd.conf

# has an effect if the above global write enable is activated. Also, you will

# obviously need to create a directory writable by the FTP user.

# anon_upload_enable=YES

Remove # character.

# obviously need to create a directory writable by the FTP user. anon_upload_enable=YES

Step 4: Restart the ftp service.

# service vsftpd restart

Shutting down vsftpd: [ OK ]

Starting vsftpd for vsftpd: [ OK ]

If SELinux is enabled in the ftp server, Could not Create fle error will be displayed.

To solve this,

Step 5: Check the Booleans for ftp.

# getsebool -a |grep ftp allow_ftpd_anon_write --> off allow_ftpd_full_access --> off allow_ftpd_use_cifs --> off allow_ftpd_use_nfs --> off ftp_home_dir --> on ftpd_connect_db --> off ftpd_use_passive_mode --> off httpd_enable_ftp_server --> off tftp_anon_write --> off

Make "allow_ftpd_anon_write" on

# setsebool -P allow_ftpd_anon_write on

# getsebool -a |grep ftp allow_ftpd_anon_write --> on

Step 6: Add read write permission in context of upload directory using below command.

# cd /var/ftp/

# chcon -t public_content_rw_t upload

Finally, login to the client machine, access the ftp server and try uploading the fles in it.

# ftp 192.168.150.133

Connected to 192.168.150.133 (192.168.150.133).

220 (vsFTPd 2.2.2)

Name (192.168.150.133:root): ftp

331 Please specify the password.

Password:

230 Login successful.

Remote system type is UNIX.

Using binary mode to transfer fles. ftp> ftp> cd upload

250 Directory successfully changed. ftp> ftp> put test local: test remote: test

227 Entering Passive Mode (192,168,150,133,252,220).

150 Ok to send data.

226 Transfer complete.

24 bytes sent in 0.0172 secs (1.39 Kbytes/sec) ftp> ftp> ftp> ls

227 Entering Passive Mode (192,168,150,133,36,247).

150 Here comes the directory listing.

-rw------1 14 50 24 Sep 19 09:13 test

226 Directory send OK.

227 Entering Passive Mode (192,168,150,133,41,64).

150 Ok to send data.

226 Transfer complete.

77 bytes sent in 0.014 secs (5.50 Kbytes/sec) local: emp.txt remote: emp.txt

227 Entering Passive Mode (192,168,150,133,113,70).

150 Ok to send data.

226 Transfer complete.

215 bytes sent in 0.00895 secs (24.03 Kbytes/sec) ftp> ftp> ls

227 Entering Passive Mode (192,168,150,133,185,14).

150 Here comes the directory listing.

-rw------1 14 50 77 Sep 19 09:15 dept.txt

-rw------1 14 50 215 Sep 19 09:15 emp.txt

-rw------1 14 50 27 Sep 19 09:14 fle3

-rw------1 14 50 24 Sep 19 09:13 test

226 Directory send OK. ftp> put - to upload single fle. mput - to upload multiple fles.

Allowing root access to the ftp.

# ftp 192.168.150.133

Connected to 192.168.150.133 (192.168.150.133).

220 (vsFTPd 2.2.2)

Name (192.168.150.133:root): root

To allow root user, comment root in /etc/vsftpd/user_list and /etc/vsftpd/ftpusers

# vi /etc/vsftpd/ftpusers

# Users that are not allowed to login via ftp

#root

# vi /etc/vsftpd/user_list

# for users that are denied.

#root

Make sure ftp_home_dir is on in ftp

Booleans.

# getsebool -a |grep ftp allow_ftpd_anon_write --> on allow_ftpd_full_access --> off allow_ftpd_use_cifs --> off allow_ftpd_use_nfs --> offftp_home_dir --> on

Try logging again as root in the ftp server.

# ftp 192.168.150.133

Connected to 192.168.150.133

(192.168.150.133).

220 (vsFTPd 2.2.2)

Name (192.168.150.133:root): root

331 Please specify the password.

Password:

230 Login successful.

Remote system type is UNIX.

Network File System (NFS) is a distributed fle system protocol originally developed by Sun Microsystems in 1984, allowing a user on a client computer to access fles over a computer network much like local storage is accessed.

Linux can be both NFS server and NFS client. Which means that it can export fle systems to other systems and mount fle systems exported from the other machines.

With the help of NFS, we can set up fle sharing between Unix to Linux systems and Linux to Unix systems.

A manual refresh is not needed because the client accesses the remote fle system as if it were local. Because access is granted by IP address, username and password are not required.

However, there are security risks to consider because the NFS server knows nothing about the users on the client system.

Step 1. Install NFS server

Use the following command to install NFS

# yum -y install nfs-utils nfs-utils-lib

Step 2. Create a directory called /nfs_share with the following command create some fles.

# mkdir /nfs_share

# touch nsf{1..5}

Step 3. edit the fle /etc/exports and add the following line.

# vi /etc/exports

/nfs_share *(rw,sync)

/nfs_share - The directory to be shared

* - The clients were allowed to access the share. You can restrict it by IP address. For example, you could, instead of the asterisk, put

192.168.0.0/24 to restrict it to clients on the 192.168.0.0/24 network. rw - Read and Write access sync - Sync the disk before command completion.

Step 4. Now run the exportfs command to export the directory.

# exportrf -avr

-r : Reexport all directories

-u : Unexports one or more directories

-v : Provides verbose output

Step 5. Restart the services and make it permanent.

# service nfs restart

If you want NFS to start at boot, use the following command

# chkconfg --levels 235 nfs on

Confguring NFS Client

Step 1. You must install the nfs package on the client with this command

# yum install nfs-utils nfs-utils-lib -y

Step 2. Start the service.

# service nfs restart

# chkconfg --levels 235 nfs on

Step 3. Once the package is installed, you can use the showmount command to view exports on an NFS server.

# showmount -e 192.168.10.90

192.168.10.90 -- Server IP address

Step 4. Make a directory and mount NFS over it.

# mkdir /nfs_client1

# mount -t nfs 192.168.10.90:/nfs_share /nfs_client1

Step 5: To make it permanent mount, edit /etc/fstab and add the following line.

# vi /etc/fstab

192.168.10.90:/nfs_share /nfs_client1 nfs defaults 0 0

After it is mounted, we are able to view the contents of the exported fles/directories locally. Samba

Samba is a free software re-implementation of the Common Internet File System (CIFS, also known as SMB) protocol. Microsoft clients will use this protocol to access fles and printers located on your Linux/Unix server just as if it were a native windows server.

Samba allows fle and print sharing between computers running Microsoft Windows and computers running Unix.

Step 1 : Install SAMBA package

Download samba packages and install by using following command.

# yum install samba* -y

Step 2: Once installation completed, make directory and assign full permissions to it.

# mkdir /samba_share

# chmod 777 /samba_share

Step 3: Check the context of the directory and change it to according to samba

# ls -ldZ /samba_share drwxrwxrwx. root root unconfned_u:object_r:default_t:s0 /samba_share/

# chcon -t samba_share_t /samba_share/

# ls -ldZ /samba_share/ drwxrwxrwx. root root unconfned_u:object_r:samba_share_t:s0 /samba_share/

Step 4: Create a user or existing use who will be allowed to login in as samba use.

# smbpasswd -a smbuser

New SMB password:

Retype new SMB password:

Added user smbuser.

To delete a user from samba, use smbpasswd -x username

To check all the samba users

# pdbedit -L smbuser:501:

Step 5 : Open confguration fle (/etc/samba/smb.conf) and the following lines.

[fleshare] comment = Public Stuff path = /samba_share

Step 6: Test the samba parameters.

# testparm

Load smb confg fles from /etc/samba/smb.conf rlimit_max: increasing rlimit_max (1024) to minimum Windows limit (16384)

Processing section "[homes]"

Processing section "[printers]"

Processing section "[smbhare]"

Loaded services fle OK.

Server role: ROLE_STANDALONE

Press enter to see a dump of your service defnitions

[printers]

comment = All Printers

path = /var/spool/samba

printable = Yes

print ok = Yes

browseable = No

[fleshare]

comment = Public Stuff

path = /samba_share

valid users = smbuser

read only = No

hosts allow = 192.168.1.

Step 6: Restart the Samba service with the commands service smb restart and service nmb restart.

# service smb restart

# chkconfg smb on

# service nmb restart

# chkconfg nmb on

Windows as client:

To connect to the samba server, right click on my computer and select Map Network Drive.

Give the address of the samba server and click on Finish button to continue.

When properly confgured, it will prompt for username and password and you can see the fles and you can add the fles to samba server. One time schedule Recurring schedule

One time job schedule

"at" command is used to schedule a job for a particular time or interval. But we can't use at command for any recurring tasks. For recurring tasks use Linux crontab.

"at" command can be useful for taking one time backup, sending email as reminder at specifed time, executing a script at specifed time etc.

Schedule job using at command

Syntax :

# at time

Task

Ctrl+d to save it.

Schedule "sh load.sh" command to be executed on next 6:00 AM once.

# at 6:00 AM at> sh load.sh at> < EOT > job 1 at 2019-05-10 06:00

Schedule "sh load.sh" command to be executed at 6:00 AM till 7 days from now.

To check list of jobs.

# at -l

1 2019-05-10 06:00 a root

3 2019-05-17 06:00 a root or

# atq

1 2019-05-10 06:00 a root

3 2019-05-17 06:00 a root

To check what is scheduled

# at -c job id

# at -c 3

# atrm job id(To remove a job)

# atq

1 2019-05-10 06:00 a root

3 2019-05-17 06:00 a root

# atrm 3

# atq

1 2019-05-10 06:00 a root

Schedule task at 09:00 AM.

# at 09:00 AM

Schedule task at 01:00 AM on coming Sunday.

# at 01:00 AM Sun

Schedule task at 06:00 AM on coming 5th July.

Schedule task to execute at midnight.

# at midnight

Schedule task at 09:00 AM on coming 12th Dec 2016.

# at 09:00 AM 12/12/2019

# at 09:00 AM 12.12.2019

Schedule task at 12:00 PM on same date next month.

# at 12:00 PM next month

Schedule task at 07:00 AM tomorrow.

# at 07:00 AM tomorrow

Schedule task to execute just after 1 hour.

# at now + 1 hour

Schedule task to execute just after 30 minutes.

# at now + 30 minutes

Schedule task to execute just after 1 and 2 weeks.

# at now + 1 week

# at now + 2 weeks

Schedule task to execute just after 1 and 2 years.

# at now + 1 year

# at now + 2 years

In Linux Operating system, it is possible to create a schedule for commands or script fles to be executed automatically within a specifed period of time, on a specifed date.

Linux comes with the automated task utility task i.e cron. The cron daemon on Linux runs jobs in the background at specifc times.

The jobs that you want to run with cron can be scheduled in various ways. The most common way is to edit a fle which is known as your crontab.

Normally, each user has his/her own and is able to schedule jobs by editing it.

/etc or a subdirectory of /etc) that only system administrators can edit.

Each line of a crontab fle represents a job, and looks like this

$ cd /etc/

$ cat crontab

SHELL=/bin/bash

PATH=/sbin:/bin:/usr/sbin:/usr/bin

MAILTO=root

# For details see man 4 crontabs

# Example of job defnition:

# .------minute (0 - 59)

# | .------hour (0 - 23)

# | | .------day of month (1 - 31)

# | | | .------month (1-12) OR jan,feb,mar,apr ..

# | | | | .---- day of week (0 - 6) (Sunday=0 or 7)

# | | | | | OR sun,mon,tue,wed,thu,fri,sat

# * * * * * user-name command to be executed

Nonstandard predefned scheduling defnitions

Some cron implementations support the following non-standard macros

EntryDescription Equivalent to

@yearly (or @annually) Run once a year at midnight of 1 January 0 0 1 1 *

@monthly Run once a month at midnight of the frst day of the month 0 0 1 * *

@weekly Run once a week at midnight on Sunday morning 0 0 * * 0

@daily Run once a day at midnight 0 0 * * *

@hourly Run once an hour at the beginning of the hour 0 * * * *

@reboot Run at startup N/A

Cron permissions

Below two fles play an important role

/etc/cron.allow - If this fle exists, it must contain your username for you to use cron jobs.

To display the current crontab

$ crontab -l

To edit the crontab, use below command

$ crontab -e

To delete crontab, use below command

$ crontab -r

Run backup.sh script at 01:30 AM daily.

30 01 * * * /bin/bash /home/ops/scripts/backup.sh

Run load.sh script daily every hour from 02 AM to 08 AM

00 02-08 * * * /bin/bash /home/ops/load.sh

Run monday.sh every Monday, at 9 A.M. and 6 P.M

00 09-18 * * Mon /bin/bash /home/ops/monday.sh

CEHPOINT E-LEARNING cehpoint.co.in Jit Banerjee