KET/KTL 2020

Introduction to the Command Language and Standard Tools Overview with Examples

Jan Bělohoubek Aims of this Lecture

3 Command Line Interface – CLI 1 Aims of this Lecture Motivation (Anketa) 4 (Our) Reference

2 Unix Operating Systems

1/26 Aims of this Lecture Motivation (Anketa)

provide an overview on "classical" engineering SW tools no more "What is this?" ! hands on . . . but gently our time is limited speed-up the learning curve as you will face it

Area/Tool GNU/Linux CLI Scripting VCS Make SSH Embedded SW maybe probably of course of course of course maybe Application SW maybe probably of course of course of course probably Digital Design probably probably of course of course probably probably PCB Design maybe maybe probably of course maybe maybe Mechanical maybe maybe maybe probably rather not rather not Design Research+1 of course of course of course of course of course of course

1Simulations, Supercomputing, Networking, Control, AI . . . 1/26 Unix Operating Systems

Linux Distributions Popular in Industry 1 Aims of this Lecture (EDA) Linux Distributions 2 Unix Operating Systems Popular for Home Use Brief History Unix/Linux 3 Command Line Interface – Surprising/New Features CLI How To Start . . . for Windows Users 4 (Our) BASH Reference

2/26 Unix Operating Systems Brief History

Image source: https://en.wikipedia.org/wiki/History_of_Unix

1969 Unnamed PDP-7 1969 Open source 1971 to 1973 Unix 1971 to 1973 Version 1 to 4 Mixed/shared source

1974 to 1975 Unix 1974 to 1975 Closed source Version 5 to 6 PWB/Unix 1978 1978

BSD 1.0 to 2.0 Unix 1979 Version 7 1979

Unix/32V 1980 1980 BSD 3.0 to 4.1 Xenix 1981 System III 1981 1.0 to 2.3 1982 1982 Xenix 3.0 1983 BSD 4.2 SunOS 1983 1 to 1.1 System V R1 to R2 1984 SCO Xenix 1984

Unix 1985 Version 8 SCO Xenix 1985 AIX V/286 System V 1986 Unix-like systems BSD 4.3 1.0 R3 HP-UX 1986 SunOS SCO Xenix 1.0 to 1.2 Unix 1.2 to 3.0 1987 V/386 1987 9 and 10 HP-UX 1988 (last versions BSD 4.3 System V 1988 2.0 to 3.0 from Tahoe R4 1989 Bell Labs) SCO Xenix 1989 BSD Net/1 V/386 BSD 4.3 1990 Reno 1990 1991 BSD Net/2 1991 Linux 0.0.1 SunOS 4 Minix 386BSD 1.x NexTSTEP/ 1992 OPENSTEP HP-UX 1992 1.0 to 4.0 NetBSD 6 to 11 0.8 to 1.0 Linux BSD 1993 SCO UNIX UnixWare 1993 0.95 to 1.2.x 4.4-Lite 3.2.4 1.x to 2.x & 1994 FreeBSD (System V 1994 1.0 to Lite Release 2 R4.2) 1995 2.2.x NetBSD 1995 OpenBSD OpenServer 1.1 to 1.2 Solaris 1.0 to 2.2 5.0 to 5.04 1996 2.1 to 9 1996 1997 1997 NetBSD 1.3 1998 FreeBSD 1998 3.0 to 3.2 OpenServer 1999 Minix 1999 Mac OS X 5.0.5 to 5.0.7 2.x Server 2000 2000

2001 to 2004 AIX 2001 to 2004 Linux 3.0-7.2 2005 2.x UnixWare 2005 7.x 2006 to 2007 (System V 2006 to 2007 R5) 2008 Solaris 2008 10 OpenServer Mac OS X, 2009 OpenBSD 6.x 2009 OS X, FreeBSD NetBSD 2.3-6.8 HP-UX 2010 macOS 3.3-12.x 1.3-8.1 2010 11i+ Minix 10.0 to 10.15 DragonFly 2011 3.1.0-3.4.0 (Darwin BSD 2011 1.2.1 to 19) 1.0 to 4.8 Linux OpenSolaris 2012 to 2015 2012 to 2015 3.x & derivatives (illumos, etc.) 2016 2016 Solaris Linux 11.0-11.4 2/26 2017 2017 4.x OpenServer 2018 10.x 2018 Linux 5.x 2019 2019 Unix Operating Systems Unix/Linux Surprising/New Features

Multi- multiple users work concurently (terminals); root user Multi-Tasking/Time-Sharing multiple tasks run concurently Hierarchical filesystem – tree structure with root "/" everything is a file (directory, device, . . . ) Software Management Tools and Repositories Linux distributions have software management tools and repositories with ability to install/update software automatically since mid 90s (1997 RPM, 1995 dselect) Android Store comes in . . . 2008 Windows Store comes in . . . 2012 How you search for a new software (or updates)?

3/26 Unix Operating Systems How To Start . . . for Windows Users

connect to a remote machine (our case!) select a Linux Distribution and install it: allong your Windows Installation (dual-boot) in a virtual environment (e.g. VirtualBox) Windows Subsystem for Linux currently, there are more than 500 actively developed Linux Distributions2

2https://www.tecmint.com/linux-distro-for-power-users/ 4/26 Unix Operating Systems Linux Distributions Popular in Industry (EDA)

Red Hat Enterprise Linux (RHEL) CentOS – RHEL "clone" SUSE Linux Enterprise (SLE) (Oracle Linux)

5/26 Unix Operating Systems Linux Distributions Popular for Home Use

Fedora – driven by community around RedHat ("test enviroment" for RHEL) OpenSUSE – driven by community around SUSE – commercial but free (with a paid support option) Debian – greatest community driven distribution Many others: Linux Mint, Slackware, , Arch, . . .

6/26 Command Line Interface – CLI

Why to Use . . . How to Use . . . 1 Aims of this Lecture Shell Command Language – Unix 2 Unix Operating Systems Standard SSH – Secure Shell 3 Command Line Interface – CLI 4 (Our) BASH Reference It is Used for . . .

7/26 Command Line Interface – CLI It is Used for . . .

traditional interface for interactive access to computers (super)computer terminals starting in the mid-1960s PCs in/since 1980s nowedays mostly (typical terminal host systems) servers and supercomputers – standardized embedded devices – standardized (e.g. AT) or custom/proprietary (e.g. KETCube) network equipment – routers, probes, . . . &Ds personal computers – often mix TUI & GUI for efficiency & automation

7/26 Command Line Interface – CLI Why to Use . . .

less resources remote access, terminal access . . . → "embedded" scripting – automation & repetitive tasks CLI vs. script language: program execution vs. function call; work with text vs. work with data (objects) efficient work with files & text filter certain lines from a huge csv file (Excell will crash!) browse & search log files bulk file rename software execution control handle return value access "hidden" features ...

8/26 Command Line Interface – CLI How to Use . . .

Terminal (emulator) HW device to access a terminal host system SW application (terminal emulator), that mimics HW terminal mostly on PC Command Interpreter:

Exec CMD Prompt example – (ba)sh:

[jan@host]:/home/jan$#[user@host]:path$ PROMPT Exec Batch Prompt example – KETCube: Assign variable >>

9/26 Command Line Interface – CLI Shell Command Language – Unix Standard

shell is a command interpreter there are many Shell Command Language implementations with a superset of features sh (Bourne SHell), bash (Bourne Again SHell) – most popular ash (Almquist SHell), dash (Debian Almquist SHell) – leightweight, scripts (may be) compatible with bash csh (C shell), tcsh – increased readability (scripting!); C-inspired syntax sh is nowedays a link to (in most cases) bash, tcsh or dash We will discover only a small part of the Unix tool universe: Many details are hidden or simplified intentionally!

10/26 Command Line Interface – CLI SSH – Secure Shell

the cryptographic network protocol – establishes a secured client-server connection/channel capabilities (openSSH, SSHv2): remote SHell & tunneling user authentication by /private-public key pairs Secure Copy (SCP) and File Transfer (SFTP) protocols Software support: Un*x: openSSH client/server (ssh, sshfs, scp) Windows: Putty, WinSCP

$ ssh [email protected] -i .ssh/id_rsa# access remote with $# pub/priv. key pair $ ssh -L8080:localhost:80 ws.net# access ws.net:80 $# from localhost:8080 $ ssh -X [email protected] xeyes# display xeyes locally

11/26 (Our) BASH Reference

Filesystem Tree – Logic Filesystem 1 Aims of this Lecture Standard Unix Tools – Filesystem 2 Unix Operating Systems Standard Unix Tools – Text (Files) and Filters 3 Command Line Interface – Standard Unix Tools – CLI Control Flow Standard Unix Tools – 4 (Our) BASH Reference Get Help Command Execution Bash Tricks Builtin Commands Midnight Commander Shell Variables Tricks

12/26 (Our) BASH Reference Command Execution

standard output (default is screen)

standard input standard error output PROCESS (default is keyboard) (default is screen)

?

CMD [parameters] > output < input

inputs may be redirected from file: {< , «} outputs may be redirected to file: {> , »} commands may be concatenated: {; , &&, ||, ...} cmd1 ; cmd2 – run cmd1, then run cmd2 cmd1 | cmd2 – filter cmd1 && cmd2 – run cmd2 only if cmd1 OK cmd1 || cmd2 – run cmd2 only if cmd1 ERROR

12/26 (Our) BASH Reference Builtin Commands

source – source file echo – print args to stdout pwd – get absloute path of the working directory exit – exit shell + return value

13/26 (Our) BASH Reference Builtin Commands

$ source .bashrc $ $pwd / home / jan $ $ echo Hello World! Hello World! $ d bash: d: Command not found... $ alias d="date" $ d Po 2. listopadu 2020, 15:39:00 CET $ $ exit0# no error

14/26 (Our) BASH Reference Shell Variables

Important variables: HOME – The current user’s home directory PATH – list of directories in which the shell looks for commands PS1 – prompt definition ? – last command status

15/26 (Our) BASH Reference Shell Variables

$ echo $HOME / home / jan $ $ echo $PATH / usr /local/bin:/usr/local/sbin:/usr/bin:/usr/sbin $ $ PATH=${PATH}:/sbin# add new directory toPATH $ echo $PATH / usr /local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/sbin $ $ echo $PS1 [\u@\h \W]\$# special characters are evaluated byBASH $

16/26 (Our) BASH Reference Filesystem Tree – Logic Filesystem

relative path: ../.. absolute path: / /

HDD0

etc home dev bin usr mnt ...

HDD1 bin local nfs

working directory relative path: . relative path: ../../usr/local/bin absolute path: /home/jan absolute path: /usr/local/bin Remote jan bin File Server

backup

17/26 (Our) BASH Reference Standard Unix Tools – Filesystem

cd – Change Directory cp – CoPy file mv – MoVe file rm – ReMove file mkdir – Make DIRectory touch – change modify time of file, create empty file if the file does not exist (pwd – get absloute path) + mc – Midnight Commander :-)

18/26 (Our) BASH Reference Standard Unix Tools – Filesystem

$ ls -l / ... drwxr-xr-x 2 root root 4096 Mar 14 2020 bin dr-x------3 root root 4096 Oct 23 03:51 boot ... drwxr-xr-x 2 root root 6 Sep 23 2013 home ... $ $ pwd /users/b/belohoub/home $ mkdir KTL $ cdKTL $ ls $ touch test $ ls test $ rm test $ ls $

19/26 (Our) BASH Reference Standard Unix Tools – Text (Files) and Filters

cat – CATenate head and tail more and less wget sort and uniq grep – like ed command ’g/re/p’ sed – Stream EDitor (ed – 1973) awk – Aho–Weinberger–Kernighan wc – Word Count diff – DIFFerences vi – VIsual (learn how to exit :-))

20/26 (Our) BASH Reference Standard Unix Tools – Text (Files) and Filters

Dataset description: http://opendata.zcu.cz/Energeticky-dispecink.html

$ wget http://openstore.zcu.cz/OD_ZCU_09_2020/\ OD_ZCU_POCASI_09_2020_CSV.zip $ $ unzip OD_ZCU_POCASI_09_2020_CSV.zip Archive: OD_ZCU_POCASI_09_2020_CSV.zip inflating: OD_ZCU_POCASI_09_2020.CSV $ $ cat OD_ZCU_POCASI_09_2020.CSV | wc -l $ $ less OD_ZCU_POCASI_09_2020.CSV ... $ $ tail -n 10 OD_ZCU_POCASI_09_2020.CSV $ $# get data for 1.9.2020 $ grep \"01.09 OD_ZCU_POCASI_09_2020.CSV $

21/26 (Our) BASH Reference Standard Unix Tools – Text (Files) and Filters

Dataset description: http://opendata.zcu.cz/Energeticky-dispecink.html

$ $# get first 10 temperatures $ cat OD_ZCU_POCASI_09_2020.CSV | awk-F";"’{print␣$2}’\ | head -n 10 $ $# print date× when temperature was 14- 15C $ awk-F";"’{if␣(($2␣>␣14)␣&&␣($2␣<␣15))␣print␣$1}’\ OD_ZCU_POCASI_09_2020.CSV $ $# now compute in how many days in October $# the temperature approached 25C $ ... 12 $

22/26 (Our) BASH Reference Standard Unix Tools – Control Flow

Conditional branch: $ touch /file $ if["$?"="0"] > then > echo"Last␣command␣execution␣OK!" > else > echo"Last␣command␣execution␣FAILED!" > fi For loop: $ cdKTL#$( cmd) -> execute cmd and use its std output $ fora in $( ls /etc | head -n 10 ) > do > echo $a ... > touch ${a}_$( date +%s ) > done + while, case (switch)

23/26 (Our) BASH Reference Standard Unix Tools – Get Help

-h | –help command parameters man pages with 9 sections: 1 – Executable programs or shell commands 3 – Library calls (functions within program libraries) info pages (more complex) Example:

$ man printf ...

$ man 1 printf ...

$ man 3 printf ...

24/26 (Our) BASH Reference Bash Tricks

TAB – (auto)complete command – browse command history ↑ ↓ CTRL+R – search command history CTRL+C – interrupt the running process ...

25/26 (Our) BASH Reference Midnight Commander Tricks

TAB – switch focus between panels INSERT – mark/remove mark ALT+S – search CTRL+O – switch to terminal ALT+O – in inactive panel display parent directory (for file) or directory content (for directory) ALT+ENTER – insert file name to cmdline ...

26/26 (Our) BASH Reference Midnight Commander Tricks

Featured Reading and Resources: https://pubs.opengroup.org/onlinepubs/009695399/utilities/ xcu_chap02.html https://www.gnu.org/doc/doc.html https://www.gnu.org/software/bash/manual https://docs.microsoft.com/en-us/windows/wsl/install-win10 https://www.linuxexpres.cz/praxe/serial-o-bashi (cs) https://learnxinyminutes.com/docs/awk/ https://learnxinyminutes.com/docs/bash/

Thank you for your attention!

Jan Bělohoubek UWB, Czech Republic [email protected] +420 377 63 4514