UNIX, GNU/Linux and simple tools for data manipulation Dr Jean-Baka DOMELEVO ENTFELLNER BecA-ILRI Hub Basic Bioinformatics Training Workshop @ILRI Addis Ababa Wednesday December 13th 2017 Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 1 / 37 1 UNIX & GNU/Linux: brief history and introduction 2 Using the Bash shell Your first commands Filesystems and permissions Bash special characters and features Quoting in Bash 3 So many tools You CANNOT live without your man Data manipulation commandline tools Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 2 / 37 Outline 1 UNIX & GNU/Linux: brief history and introduction 2 Using the Bash shell Your first commands Filesystems and permissions Bash special characters and features Quoting in Bash 3 So many tools You CANNOT live without your man Data manipulation commandline tools Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 3 / 37 UNIX & GNU/Linux: introduction GNU/Linux is an operating system (OS). GNU/Linux fully belongs to a broad family of OSes, the UNIX family. Operating system: definition unique interface between the computer (hardware) and the different programs (software) users run on it allows different programs and different users to use concurrently the same machine implements a filesystem, a console environment, a graphical environment, drivers for keyboard and mouse, etc examples of operating systems: Windows (Microsoft), Mac OS X (Apple), Android (Google), GNU/Linux, FreeBSD, etc “Linux” is only the kernel of GNU/Linux systems, responsible for granting access to the resources on the host and for time-sharing between processes. Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 4 / 37 UNIX & GNU/Linux systems: timeline GNU/Linux: a fairly recent member of an old and huge family (see http://www.levenez.com/unix/) 1969: UNICS 1971: UNIX Time-Sharing System V1 1982: SunOS 1.0 1983: UNIX System V 1991: GNU project (GNU/Hurd) ; Linux 0.01 1994: Linux 1.0 1999: Darwin 0.1 ; Mac OS X Server 1.0 2008: Android 1.0 (derived from Linux 2.6.23) 2013: Linux 3.9 Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 5 / 37 Linux distributions: different flavours of the same OS The GNU/Linux operatring system comes in different distributions. Three distributions have ever been true beacons and gave many offsprings: 1 Debian (1993) ) Ubuntu, 2004 and Linux Mint, 2010 2 Slackware (1993), from SLS (1992) ) SuSE, 1998 3 RedHat (late 1994) ) CentOS and Fedora, both 2003 For a full account, see http://futurist.se/gldt Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 6 / 37 UNIX environments are free from viruses. UNIX enables you to harness the full computational power of your machine. UNIX systems have been designed from their origin to be massively multi-user and multi-process systems. UNIX systems are much more secure than any Windows. Take-home message The true power of UNIX (and so of GNU/Linux) lies in its commandline interface. What makes UNIX systems superior to the Windows family UNIX gives you more control over your computer (no hidden actions, no undesired pieces of software). Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 7 / 37 UNIX enables you to harness the full computational power of your machine. UNIX systems have been designed from their origin to be massively multi-user and multi-process systems. UNIX systems are much more secure than any Windows. Take-home message The true power of UNIX (and so of GNU/Linux) lies in its commandline interface. What makes UNIX systems superior to the Windows family UNIX gives you more control over your computer (no hidden actions, no undesired pieces of software). UNIX environments are free from viruses. Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 7 / 37 UNIX systems have been designed from their origin to be massively multi-user and multi-process systems. UNIX systems are much more secure than any Windows. Take-home message The true power of UNIX (and so of GNU/Linux) lies in its commandline interface. What makes UNIX systems superior to the Windows family UNIX gives you more control over your computer (no hidden actions, no undesired pieces of software). UNIX environments are free from viruses. UNIX enables you to harness the full computational power of your machine. Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 7 / 37 UNIX systems are much more secure than any Windows. Take-home message The true power of UNIX (and so of GNU/Linux) lies in its commandline interface. What makes UNIX systems superior to the Windows family UNIX gives you more control over your computer (no hidden actions, no undesired pieces of software). UNIX environments are free from viruses. UNIX enables you to harness the full computational power of your machine. UNIX systems have been designed from their origin to be massively multi-user and multi-process systems. Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 7 / 37 Take-home message The true power of UNIX (and so of GNU/Linux) lies in its commandline interface. What makes UNIX systems superior to the Windows family UNIX gives you more control over your computer (no hidden actions, no undesired pieces of software). UNIX environments are free from viruses. UNIX enables you to harness the full computational power of your machine. UNIX systems have been designed from their origin to be massively multi-user and multi-process systems. UNIX systems are much more secure than any Windows. Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 7 / 37 What makes UNIX systems superior to the Windows family UNIX gives you more control over your computer (no hidden actions, no undesired pieces of software). UNIX environments are free from viruses. UNIX enables you to harness the full computational power of your machine. UNIX systems have been designed from their origin to be massively multi-user and multi-process systems. UNIX systems are much more secure than any Windows. Take-home message The true power of UNIX (and so of GNU/Linux) lies in its commandline interface. Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 7 / 37 Outline 1 UNIX & GNU/Linux: brief history and introduction 2 Using the Bash shell Your first commands Filesystems and permissions Bash special characters and features Quoting in Bash 3 So many tools You CANNOT live without your man Data manipulation commandline tools Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 8 / 37 interact with the installed software (install, run, etc), login to distant hosts (telnet, ssh), perform all of the above through automated processes ) scripts. Shells are at the same time commandline environments (run one command at a time) and scripting environments (write and run scripts). On most GNU/Linux distributions, Bash is accessible through the "Terminal" icon . Bash: a shell environment Bash is the most popular shell environment on GNU/Linux systems. It stands for "Bourne Again Shell". Shell environments are designed to: interact with the host filesystem (browse and create directories, see the content of files, etc), Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 9 / 37 login to distant hosts (telnet, ssh), perform all of the above through automated processes ) scripts. Shells are at the same time commandline environments (run one command at a time) and scripting environments (write and run scripts). On most GNU/Linux distributions, Bash is accessible through the "Terminal" icon . Bash: a shell environment Bash is the most popular shell environment on GNU/Linux systems. It stands for "Bourne Again Shell". Shell environments are designed to: interact with the host filesystem (browse and create directories, see the content of files, etc), interact with the installed software (install, run, etc), Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 9 / 37 perform all of the above through automated processes ) scripts. Shells are at the same time commandline environments (run one command at a time) and scripting environments (write and run scripts). On most GNU/Linux distributions, Bash is accessible through the "Terminal" icon . Bash: a shell environment Bash is the most popular shell environment on GNU/Linux systems. It stands for "Bourne Again Shell". Shell environments are designed to: interact with the host filesystem (browse and create directories, see the content of files, etc), interact with the installed software (install, run, etc), login to distant hosts (telnet, ssh), Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 9 / 37 Shells are at the same time commandline environments (run one command at a time) and scripting environments (write and run scripts). On most GNU/Linux distributions, Bash is accessible through the "Terminal" icon . Bash: a shell environment Bash is the most popular shell environment on GNU/Linux systems. It stands for "Bourne Again Shell". Shell environments are designed to: interact with the host filesystem (browse and create directories, see the content of files, etc), interact with the installed software (install, run, etc), login to distant hosts (telnet, ssh), perform all of the above through automated processes ) scripts. Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 9 / 37 On most GNU/Linux distributions, Bash is accessible through the "Terminal" icon . Bash: a shell environment Bash is the most popular shell environment on GNU/Linux systems.