TEX BY TOPIC, A TEXNICIAN’S REFERENCE VICTOR EIJKHOUT DOCUMENT REVISION 1.5, 2019 Copyright c 1991-2013 Victor Eijkhout. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled ”GNU Free Documentation License”. This document is based on the book TEX by Topic, copyright 1991-2019 Victor Eijkhout. This book was printed in 1991 by Addison-Wesley UK, ISBN 0-201-56882-9, reprinted in 1993, pdf version first made freely available in 2001. Cover design (lulu.com version): Joanna K. Wozniak ([email protected]) Victor Eijkhout – TEX by Topic 1 2 Victor Eijkhout – TEX by Topic Contents License 15 Preface 21 1 The Structure of the TEX Processor 23 1.1 Four TEX processors 23 1.2 The input processor 24 1.2.1 Character input 24 1.2.2 Two-level input processing 24 1.3 The expansion processor 25 1.3.1 The process of expansion 25 1.3.2 Special cases: \expandafter, \noexpand, and \the 25 1.3.3 Braces in the expansion processor 26 1.4 The execution processor 26 1.5 The visual processor 27 1.6 Examples 28 1.6.1 Skipped spaces 28 1.6.2 Internal quantities and their representations 28 2 Category Codes and Internal States 29 2.1 Introduction 29 2.2 Initial processing 29 2.3 Category codes 30 2.4 From characters to tokens 32 2.5 The input processor as a finite state automaton 32 2.5.1 State N: new line 32 2.5.2 State S: skipping spaces 32 2.5.3 State M: middle of line 33 2.6 Accessing the full character set 33 2.7 Transitions between internal states 33 2.7.1 0: escape character 34 2.7.2 1–4, 7–8, 11–13: non-blank characters 34 2.7.3 5: end of line 34 2.7.4 6: parameter 34 2.7.5 7: superscript 34 2.7.6 9: ignored character 35 2.7.7 10: space 35 2.7.8 14: comment 35 2.7.9 15: invalid 35 2.8 Letters and other characters 35 3 2.9 The \par token 36 2.10 Spaces 37 2.10.1 Skipped spaces 37 2.10.2 Optional spaces 37 2.10.3 Ignored and obeyed spaces 38 2.10.4 More ignored spaces 38 2.10.5 hspace tokeni 39 2.10.6 Control space 39 2.10.7 ‘ ’ 39 2.11 More about line ends 40 2.11.1 Obeylines 40 2.11.2 Changing the \endlinechar 40 2.11.3 More remarks about the end-of-line character 41 2.12 More about the input processor 41 2.12.1 The input processor as a separate process 41 2.12.2 The input processor not as a separate process 42 2.12.3 Recursive invocation of the input processor 42 2.13 The @ convention 43 3 Characters 45 3.1 Character codes 45 3.2 Control sequences for characters 46 3.2.1 Denoting characters to be typeset: \char 46 3.2.2 Implicit character tokens: \let 47 3.3 Accents 48 3.4 Testing characters 49 3.5 Uppercase and lowercase 50 3.5.1 Uppercase and lowercase codes 50 3.5.2 Uppercase and lowercase commands 50 3.5.3 Uppercase and lowercase forms of keywords 50 3.5.4 Creative use of \uppercase and \lowercase 51 3.6 Codes of a character 51 3.7 Converting tokens into character strings 51 3.7.1 Output of control sequences 52 3.7.2 Category codes of a \string 52 4 Fonts 53 4.1 Fonts 53 4.2 Font declaration 54 4.2.1 Fonts and tfm files 54 4.2.2 Querying the current font and font names 54 4.2.3 \nullfont 55 4.3 Font information 55 4.3.1 Font dimensions 55 4.3.2 Kerning 56 4.3.3 Italic correction 56 4.3.4 Ligatures 57 4.3.5 Boundary ligatures 57 5 Boxes 59 4 Victor Eijkhout – TEX by Topic 5.1 Boxes 60 5.2 Box registers 60 5.2.1 Allocation: \newbox 60 5.2.2 Usage: \setbox, \box, \copy 61 5.2.3 Testing: \ifvoid, \ifhbox, \ifvbox 61 5.2.4 The \lastbox 61 5.3 Natural dimensions of boxes 62 5.3.1 Dimensions of created horizontal boxes 62 5.3.2 Dimensions of created vertical boxes 62 5.3.3 Examples 63 5.4 More about box dimensions 64 5.4.1 Predetermined dimensions 64 5.4.2 Changes to box dimensions 65 5.4.3 Moving boxes around 65 5.4.4 Box dimensions and box placement 65 5.4.5 Boxes and negative glue 66 5.5 Overfull and underfull boxes 67 5.6 Opening and closing boxes 67 5.7 Unboxing 68 5.8 Text in boxes 69 5.9 Assorted remarks 70 5.9.1 Forgetting the \box 70 5.9.2 Special-purpose boxes 70 5.9.3 The height of a vertical box in horizontal mode 70 5.9.4 More subtleties with vertical boxes 70 5.9.5 Hanging the \lastbox back in the list 71 5.9.6 Dissecting paragraphs with \lastbox 72 6 Horizontal and Vertical Mode 73 6.1 Horizontal and vertical mode 73 6.1.1 Horizontal mode 73 6.1.2 Vertical mode 74 6.2 Horizontal and vertical commands 74 6.3 The internal modes 75 6.3.1 Restricted horizontal mode 75 6.3.2 Internal vertical mode 75 6.4 Boxes and modes 76 6.4.1 What box do you use in what mode? 76 6.4.2 What mode holds in what box? 76 6.4.3 Mode-dependent behaviour of boxes 76 6.5 Modes and glue 76 6.6 Migrating material 77 6.6.1 \vadjust 77 6.7 Testing modes 77 7 Numbers 79 7.1 Numbers and hnumberis 79 7.2 Integers 79 7.2.1 Denotations: integers 80 Victor Eijkhout – TEX by Topic 5 7.2.2 Denotations: characters 80 7.2.3 Internal integers 81 7.2.4 Internal integers: other codes of a character 82 7.2.5 hspecial integeri 82 7.2.6 Other internal quantities: coersion to integer 82 7.2.7 Trailing spaces 82 7.3 Numbers 82 7.4 Integer registers 83 7.5 Arithmetic 83 7.5.1 Arithmetic statements 84 7.5.2 Floating-point arithmetic 84 7.5.3 Fixed-point arithmetic 84 7.6 Number testing 84 7.7 Remarks 85 7.7.1 Character constants 85 7.7.2 Expanding too far / how far 85 8 Dimensions and Glue 87 8.1 Definition of hgluei and hdimeni 88 8.1.1 Definition of dimensions 88 8.1.2 Definition of glue 89 8.1.3 Conversion of hgluei to hdimeni 90 8.1.4 Registers for \dimen and \skip 90 8.1.5 Arithmetic: addition 90 8.1.6 Arithmetic: multiplication and division 91 8.2 More about dimensions 91 8.2.1 Units of measurement 91 8.2.2 Dimension testing 92 8.2.3 Defined dimensions 92 8.3 More about glue 92 8.3.1 Stretch and shrink 93 8.3.2 Glue setting 94 8.3.3 Badness 94 8.3.4 Glue and breaking 95 8.3.5 \kern 95 8.3.6 Glue and modes 95 8.3.7 The last glue item in a list: backspacing 96 8.3.8 Examples of backspacing 96 8.3.9 Glue in trace output 97 9 Rules and Leaders 99 9.1 Rules 99 9.1.1 Rule dimensions 100 9.2 Leaders 100 9.2.1 Rule leaders 101 9.2.2 Box leaders 102 9.2.3 Evenly spaced leaders 102 9.3 Assorted remarks 103 9.3.1 Rules and modes 103 6 Victor Eijkhout – TEX by Topic 9.3.2 Ending a paragraph with leaders 103 9.3.3 Leaders and box registers 103 9.3.4 Output in leader boxes 104 9.3.5 Box leaders in trace output 104 9.3.6 Leaders and shifted margins 104 10 Grouping 105 10.1 The grouping mechanism 105 10.2 Local and global assignments 106 10.3 Group delimiters 106 10.4 More about braces 107 10.4.1 Brace counters 107 10.4.2 The brace as a token 108 10.4.3 Open and closing brace control symbols 108 11 Macros 109 11.1 Introduction 109 11.2 Layout of a macro definition 110 11.3 Prefixes 110 11.4 The definition type 111 11.5 The parameter text 111 11.5.1 Undelimited parameters 112 11.5.2 Delimited parameters 112 11.5.3 Examples with delimited arguments 113 11.5.4 Empty arguments 114 11.5.5 The macro parameter character 114 11.5.6 Brace delimiting 115 11.6 Construction of control sequences 115 11.7 Token assignments by \let and \futurelet 116 11.7.1 \let 116 11.7.2 \futurelet 117 11.8 Assorted remarks 117 11.8.1 Active characters 117 11.8.2 Macros versus primitives 117 11.8.3 Tail recursion and the \loop macro 118 11.9 Macro techniques 119 11.9.1 Unknown number of arguments 119 11.9.2 Examining the argument 119 11.9.3 Optional macro parameters with \futurelet 121 11.9.4 Two-step macros 121 11.9.5 A comment environment 122 12 Expansion 125 12.1 Introduction 125 12.2 Ordinary expansion 125 12.3 Reversing expansion order 126 12.3.1 One step expansion: \expandafter 126 12.3.2 Total expansion: \edef 127 12.3.3 \afterassignment 127 12.3.4 \aftergroup 128 Victor Eijkhout – TEX by Topic 7 12.4 Preventing expansion 129 12.4.1 \noexpand 129 12.4.2 \noexpand and active characters 129 12.5 \relax 130 12.5.1 \relax and \csname 130 12.5.2 Preventing expansion with \relax 131 12.5.3 TEX inserts a \relax 131 12.5.4 The value of non-macros; \the 132 12.6 Examples 132 12.6.1 Expanding after 132 12.6.2 Defining inside an \edef 133 12.6.3 Expansion and \write 134 12.6.4 Controlled expansion inside an \edef 135 12.6.5 Multiple prevention of expansion 135 12.6.6 More examples with \relax 136 12.6.7 Example: category code saving and restoring 136 12.6.8 Combining \aftergroup and boxes 137 12.6.9 More expansion 138 13 Conditionals 139 13.1 The shape of conditionals 139 13.2 Character and control sequence tests 140 13.2.1 \if 140 13.2.2 \ifcat 140 13.2.3 \ifx 141 13.3 Mode tests 141 13.4 Numerical tests 142 13.5 Other tests 142 13.5.1 Dimension testing 142 13.5.2 Box tests 142 13.5.3 I/O tests 142 13.5.4 Case statement 142 13.5.5 Special tests 143 13.6 The \newif macro 143 13.7 Evaluation of conditionals 144 13.8 Assorted remarks 145 13.8.1 The test gobbles up tokens 145 13.8.2 The test wants to gobble up the \else or \fi 145 13.8.3 Macros and conditionals; the use of \expandafter 146 13.8.4 Incorrect matching 147 13.8.5 Conditionals and grouping 147 13.8.6 A trick 148 13.8.7 More examples of expansion in conditionals 148 14 Token Lists 151 14.1 Token lists 151 14.2 Use of token lists 151 14.3 htoken parameteri 152 14.4 Token list registers 152 8 Victor Eijkhout – TEX by Topic 14.5 Examples 153 14.5.1 Operations on token lists: stack macros 153 14.5.2 Executing token lists 154 15 Baseline Distances 155 15.1 Interline glue 155 15.2 The perceived depth of boxes