this talk

• show the historical background • introduce, explain and demonstrate , , • solve real life problems

Old Software Treasuries please interrupt me at any time in case of questions

markus schnalke please tell me what I should demonstrate

goal: motivate you to learn and use these programs

old software treasuries origin

• older than I am • classic tools • standardized • from • various implementations • representations of the Unix Philosophy • freely available • good software historical background historical background

1968: 1970: • fails • Kernighan suggests ‘‘Unix’’ • the last people working on it were Thompson, Ritchie, • deal: a PDP-11 for a document preparation system McIlroy, Ossanna 1973: 1969: • rewrite of Unix in C • they still think about operating systems 1974: • pushing idea: a file system (mainly Thompson) • first publication • in search for hardware: a rarely used PDP-7

computer hardware back then impact on software

• computers are as large as closets • don’t waste characters • users work on terminals • be quiet • terminals are: a keyboard and a line printer • nothing like ‘‘screen-orientation’’ • line printers have a speed of 10–15 chars/s

ed

• by in early 70s, based on qed • the Unix text editor • the standard text editor! • Unix was written with ed ed • first implementation of Regular Expressions

ed: old-fashioned ed: reasons

• already old-fashioned in 1984 • the standard (text editor) •‘‘So why are we spending time on such a old-fashioned • available as /bin/ed program?’’ • line oriented → works on any terminal •‘‘Although many readers will prefer some other editor for • needs few bandwidth ed daily use, is universally available, efficient and effec- • good for automated editing (scripts) tive.’’ • good for presentations (?)

it’s not so much about using it, but about knowing it ed: overview ed: commands

• usage: ed [-s] [file] i insert p print • commands: [a1[,a2]] cmd [params] a append n print with line number • default addresses, default params c change l list characters

• success → no feedback d delete e edit file • problems → ‘‘?’’ m move w write file • about 30 commands, half of them important t transfer (copy) q quit

s substitute u undo g global command

ed: addresses

4 4th line $ last line . current line

+ next line (.+1) -- the same as .-2

/RE/ next line matching RE ?RE? previous line matching RE

1,$ all lines -,/foo/ from previous line to next /foo/ ed: derived software

grep

ed vi

sed sed

sed sed: differences to ed

• by Lee McMahon in 1973 • line processing cycle •‘‘stream editor’’ • no forward/backward references • started as a one-night hack on ed (AFAIR) • hold space (g,G,h,H,x) • makes ed suitable for pipeline processing • labels and branches • irony: today, people use sed -i for in-place editing • bad i and a syntax awk

awk awk: purpose

• by Aho, Weinberger, Kernighan • for text processing • oawk in 1977, nawk in 1985 • avoids complex constructs in sh • name is abbreviation of surnames (but also ‘‘awkward’’) • floating point arithmetic • the second scripting language on Unix (besides sh) • ‘‘ sed meets C’’ awk: usage awk: features

• awk ’commands’ out • variables, assoc arrays • awk -f cmdfile out • functions • program flow like in sed • pattern matching (EREs) • input automatically split in records and fields • dynamic typing • program is a list of blocks: cond { commands } • more high-level: no pointers

awk: statements awk: functions much like C arithmetic much like in C • if, but no switch-case I/O getline(), system() • while, do-while string sub(), gsub() • for (;;) {} like in C substr(), index(), match(), • for (i in array) {} foreach split(), sprintf() • break, continue, exit, return tolower(), toupper(), • print and printf are statements! redirection possi- length(), int() ble with >, >>, | awk: variables awk: conditions

RS, FS record/field separator examples: NR number of current record • NR == 1 {...} NF $1 $NF number of fields ( ... ) • NR == 1, NR == 5 {...} ARGC, ARGV like in C • /RE/ {...} FILENAME name of current input file • $1 == "string" {...} OFMT output format for numbers • $1 ˜ /RE/ {...} ORS, OFS output record/field separator SUBSEP a[1,2] equals a[1 SUBSEP 2] • BEGIN {...} • END {...}

real life examples literature

• ‘‘The Unix Programming Environment’’ by Kernighan this talk was prepared using tools of the Heirloom project: and Pike is highly recommended to support this talk http://heirloom.sf.net • it’s one of my favorite computer books the slides macros were taken from • you really should read it! http://repo.cat-v.org/troff-slider/ all editing was done with ed, of course :-) • ‘‘SED & AWK ge-packt’’ by Stephan Thesing • if you want a german book about sed and awk then get the slides and examples are available on my website this one http://marmaro.de/docs and on • you can get it for about 3 Euro http://ulm.ccc.de/ChaosSeminar/2009/10_Softwareschaetze

2009-10-12