GAWK: Effective AWK Programming a User’S Guide for GNU Awk Edition 3 October, 2007
Total Page:16
File Type:pdf, Size:1020Kb
GAWK: Effective AWK Programming A User's Guide for GNU Awk Edition 3 October, 2007 Arnold D. Robbins \To boldly go where no man has gone before" is a Registered Trademark of Paramount Pictures Corporation. Published by: Free Software Foundation 51 Franklin Street, Fifth Floor Boston, MA 02110-1301 USA Phone: +1-617-542-5942 Fax: +1-617-542-2652 Email: [email protected] URL: http://www.gnu.org/ ISBN 1-882114-28-0 Copyright c 1989, 1991, 1992, 1993, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2007 Free Software Foundation, Inc. This is Edition 3 of GAWK: Effective AWK Programming: A User's Guide for GNUAwk, for the 3.1.6 (or later) version of the GNU implementation of AWK. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with the Invariant Sections being \GNU General Public License", the Front-Cover texts being (a) (see below), and with the Back-Cover Texts being (b) (see below). A copy of the license is included in the section entitled \GNU Free Documentation License". a. \A GNU Manual" b. \You have freedom to copy and modify this GNU Manual, like GNU software. Copies published by the Free Software Foundation raise funds for GNU development." Cover art by Etienne Suvasa. i Short Contents Foreword::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 1 Preface :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 3 1 Getting Started with awk ::::::::::::::::::::::::::::::::::::::::::::: 10 2 Regular Expressions ::::::::::::::::::::::::::::::::::::::::::::::::: 21 3 Reading Input Files ::::::::::::::::::::::::::::::::::::::::::::::::: 32 4 Printing Output :::::::::::::::::::::::::::::::::::::::::::::::::::: 51 5 Expressions :::::::::::::::::::::::::::::::::::::::::::::::::::::::: 65 6 Patterns, Actions, and Variables ::::::::::::::::::::::::::::::::::::::: 84 7 Arrays in awk ::::::::::::::::::::::::::::::::::::::::::::::::::::: 104 8 Functions::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 113 9 Internationalization with gawk :::::::::::::::::::::::::::::::::::::::: 139 10 Advanced Features of gawk :::::::::::::::::::::::::::::::::::::::::: 147 11 Running awk and gawk :::::::::::::::::::::::::::::::::::::::::::::: 154 12 A Library of awk Functions :::::::::::::::::::::::::::::::::::::::::: 161 13 Practical awk Programs ::::::::::::::::::::::::::::::::::::::::::::: 187 A The Evolution of the awk Language ::::::::::::::::::::::::::::::::::: 224 B Installing gawk :::::::::::::::::::::::::::::::::::::::::::::::::::: 231 C Implementation Notes :::::::::::::::::::::::::::::::::::::::::::::: 248 D Basic Programming Concepts :::::::::::::::::::::::::::::::::::::::: 262 Glossary ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 267 GNU General Public License ::::::::::::::::::::::::::::::::::::::::::::: 275 GNU Free Documentation License :::::::::::::::::::::::::::::::::::::::: 285 Index :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 291 ii Table of Contents Foreword ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 1 Preface ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 3 History of awk and gawk :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 3 A Rose by Any Other Name :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 4 Using This Book:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 4 Typographical Conventions ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 6 The GNU Project and This Book ::::::::::::::::::::::::::::::::::::::::::::::::::::::: 6 How to Contribute:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 8 Acknowledgments::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 8 1 Getting Started with awk :::::::::::::::::::::::::::::::::::: 10 1.1 How to Run awk Programs ::::::::::::::::::::::::::::::::::::::::::::::::::::::: 10 1.1.1 One-Shot Throwaway awk Programs ::::::::::::::::::::::::::::::::::::::::: 10 1.1.2 Running awk Without Input Files:::::::::::::::::::::::::::::::::::::::::::: 11 1.1.3 Running Long Programs ::::::::::::::::::::::::::::::::::::::::::::::::::::: 11 1.1.4 Executable awk Programs:::::::::::::::::::::::::::::::::::::::::::::::::::: 12 1.1.5 Comments in awk Programs ::::::::::::::::::::::::::::::::::::::::::::::::: 12 1.1.6 Shell-Quoting Issues ::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 13 1.2 Data Files for the Examples :::::::::::::::::::::::::::::::::::::::::::::::::::::: 14 1.3 Some Simple Examples ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 15 1.4 An Example with Two Rules ::::::::::::::::::::::::::::::::::::::::::::::::::::: 17 1.5 A More Complex Example:::::::::::::::::::::::::::::::::::::::::::::::::::::::: 17 1.6 awk Statements Versus Lines:::::::::::::::::::::::::::::::::::::::::::::::::::::: 18 1.7 Other Features of awk :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 20 1.8 When to Use awk::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 20 2 Regular Expressions:::::::::::::::::::::::::::::::::::::::::: 21 2.1 How to Use Regular Expressions:::::::::::::::::::::::::::::::::::::::::::::::::: 21 2.2 Escape Sequences :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 22 2.3 Regular Expression Operators :::::::::::::::::::::::::::::::::::::::::::::::::::: 23 2.4 Using Character Lists :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 25 2.5 gawk-Specific Regexp Operators :::::::::::::::::::::::::::::::::::::::::::::::::: 27 2.6 Case Sensitivity in Matching ::::::::::::::::::::::::::::::::::::::::::::::::::::: 28 2.7 How Much Text Matches? :::::::::::::::::::::::::::::::::::::::::::::::::::::::: 29 2.8 Using Dynamic Regexps :::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 29 2.9 Where You Are Makes A Difference::::::::::::::::::::::::::::::::::::::::::::::: 30 3 Reading Input Files :::::::::::::::::::::::::::::::::::::::::: 32 3.1 How Input Is Split into Records :::::::::::::::::::::::::::::::::::::::::::::::::: 32 3.2 Examining Fields::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 35 3.3 Nonconstant Field Numbers :::::::::::::::::::::::::::::::::::::::::::::::::::::: 35 3.4 Changing the Contents of a Field ::::::::::::::::::::::::::::::::::::::::::::::::: 36 3.5 Specifying How Fields Are Separated ::::::::::::::::::::::::::::::::::::::::::::: 38 3.5.1 Using Regular Expressions to Separate Fields :::::::::::::::::::::::::::::::: 39 3.5.2 Making Each Character a Separate Field::::::::::::::::::::::::::::::::::::: 39 3.5.3 Setting FS from the Command Line :::::::::::::::::::::::::::::::::::::::::: 40 iii 3.5.4 Field-Splitting Summary::::::::::::::::::::::::::::::::::::::::::::::::::::: 41 3.6 Reading Fixed-Width Data ::::::::::::::::::::::::::::::::::::::::::::::::::::::: 42 3.7 Multiple-Line Records :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 43 3.8 Explicit Input with getline :::::::::::::::::::::::::::::::::::::::::::::::::::::: 45 3.8.1 Using getline with No Arguments :::::::::::::::::::::::::::::::::::::::::: 46 3.8.2 Using getline into a Variable ::::::::::::::::::::::::::::::::::::::::::::::: 46 3.8.3 Using getline from a File ::::::::::::::::::::::::::::::::::::::::::::::::::: 47 3.8.4 Using getline into a Variable from a File ::::::::::::::::::::::::::::::::::: 47 3.8.5 Using getline from a Pipe :::::::::::::::::::::::::::::::::::::::::::::::::: 48 3.8.6 Using getline into a Variable from a Pipe::::::::::::::::::::::::::::::::::: 49 3.8.7 Using getline from a Coprocess::::::::::::::::::::::::::::::::::::::::::::: 49 3.8.8 Using getline into a Variable from a Coprocess ::::::::::::::::::::::::::::: 49 3.8.9 Points to Remember About getline ::::::::::::::::::::::::::::::::::::::::: 50 3.8.10 Summary of getline Variants :::::::::::::::::::::::::::::::::::::::::::::: 50 4 Printing Output :::::::::::::::::::::::::::::::::::::::::::::: 51 4.1 The print Statement :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 51 4.2 Examples of print Statements ::::::::::::::::::::::::::::::::::::::::::::::::::: 51 4.3 Output Separators ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 52 4.4 Controlling Numeric Output with print :::::::::::::::::::::::::::::::::::::::::: 53 4.5 Using printf Statements for Fancier Printing::::::::::::::::::::::::::::::::::::: 53 4.5.1 Introduction to the printf Statement ::::::::::::::::::::::::::::::::::::::: 53 4.5.2 Format-Control Letters :::::::::::::::::::::::::::::::::::::::::::::::::::::: 54 4.5.3 Modifiers for printf Formats :::::::::::::::::::::::::::::::::::::::::::::::: 55 4.5.4 Examples Using printf ::::::::::::::::::::::::::::::::::::::::::::::::::::: 57 4.6 Redirecting Output of print and printf ::::::::::::::::::::::::::::::::::::::::: 58 4.7 Special File Names in gawk ::::::::::::::::::::::::::::::::::::::::::::::::::::::: 60 4.7.1 Special Files for Standard Descriptors:::::::::::::::::::::::::::::::::::::::: 60 4.7.2 Special Files for Process-Related Information :::::::::::::::::::::::::::::::: 61 4.7.3 Special Files for Network Communications ::::::::::::::::::::::::::::::::::: 62 4.7.4 Special File Name Caveats ::::::::::::::::::::::::::::::::::::::::::::::::::: 62 4.8 Closing Input and Output Redirections ::::::::::::::::::::::::::::::::::::::::::: 62 5 Expressions ::::::::::::::::::::::::::::::::::::::::::::::::::: 65 5.1 Constant Expressions:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::