Universal Ctags Documentation Release 0.3.0
Total Page:16
File Type:pdf, Size:1020Kb
Universal Ctags Documentation Release 0.3.0 Universal Ctags Team 22 September 2021 Contents 1 Building ctags 2 1.1 Building with configure (*nix including GNU/Linux)........................2 1.2 Building/hacking/using on MS-Windows..............................4 1.3 Building on Mac OS.........................................7 2 Man pages 9 2.1 ctags.................................................9 2.2 tags.................................................. 33 2.3 ctags-optlib.............................................. 40 2.4 ctags-client-tools........................................... 46 2.5 ctags-incompatibilities........................................ 53 2.6 ctags-faq............................................... 56 2.7 ctags-lang-inko............................................ 62 2.8 ctags-lang-iPythonCell........................................ 62 2.9 ctags-lang-julia............................................ 63 2.10 ctags-lang-python.......................................... 65 2.11 ctags-lang-r.............................................. 69 2.12 ctags-lang-sql............................................. 70 2.13 ctags-lang-tcl............................................. 72 2.14 ctags-lang-verilog.......................................... 73 2.15 readtags................................................ 76 3 Parsers 81 3.1 Asm parser.............................................. 81 3.2 CMake parser............................................. 81 3.3 The new C/C++ parser........................................ 82 3.4 The new HTML parser........................................ 86 3.5 puppetManifest parser........................................ 87 3.6 The new Python parser........................................ 87 3.7 The new Tcl parser.......................................... 87 3.8 The Vim parser............................................ 88 3.9 XSLT parser............................................. 88 4 Option files 89 4.1 Order of loading option files..................................... 89 4.2 --options=PATHNAME option.................................. 90 4.3 Tips for writing an option file.................................... 90 4.4 Difference from Exuberant Ctags.................................. 91 5 Output formats 92 5.1 Changes to the tags file format.................................... 92 i 5.2 Xref output.............................................. 99 5.3 JSON output............................................. 101 6 Running multiple parsers on an input file 102 6.1 Guest parser: Applying a parser to specified areas of input file................... 102 6.2 Subparser: Tagging definitions of higher (upper) level language.................. 103 7 Interactive mode 108 7.1 generate-tags............................................. 108 7.2 sandbox submode.......................................... 109 8 Other changes 110 8.1 New and extended options...................................... 111 8.2 Incompatible changes in command line............................... 115 8.3 Changes imported from Exuberant Ctags.............................. 115 8.4 Parser related changes........................................ 116 8.5 Readtags............................................... 118 9 Extending ctags with Regex parser (optlib) 120 9.1 Regular expression (regex) engine.................................. 121 9.2 Regex option argument flags..................................... 122 9.3 Overriding the letter for file kind................................... 127 9.4 Generating fully qualified tags automatically from scope information............... 128 9.5 Multi-line pattern match....................................... 130 9.6 Advanced pattern matching with multiple regex tables....................... 133 9.7 Scheduling a guest parser with _guest regex flag......................... 138 9.8 Defining a subparser......................................... 140 9.9 Translating an option file into C source code (optlib2c)....................... 142 10 Optscript, a programming language for extending optlib parsers 143 10.1 Preparation for learning....................................... 143 10.2 Syntax extension........................................... 144 10.3 The optscript command..................................... 144 10.4 Optscript in ctags........................................... 146 10.5 Recipes................................................ 147 10.6 Difference between Optscript and PostScript............................ 148 11 Extending ctags with a parser written in C 149 11.1 Writing a parser in C......................................... 149 11.2 Input text stream........................................... 151 11.3 Output tag stream.......................................... 151 11.4 tokenInfo API............................................ 153 11.5 Multiple parsers........................................... 154 11.6 PackCC compiler-compiler..................................... 161 11.7 Automatic parser guessing (TBW).................................. 162 11.8 Managing regular expression parsers (TBW)............................ 162 11.9 Ghost kind in regex parser (TBW).................................. 162 12 Testing ctags 163 12.1 Tmain: a facility for testing main part................................ 163 12.2 Tinst: installation test......................................... 164 12.3 Fussy syntax checking........................................ 164 12.4 Finding performance bottleneck................................... 165 12.5 Checking coverage.......................................... 165 12.6 Running cppcheck.......................................... 165 13 Testing a parser 166 13.1 Units test facility........................................... 167 13.2 Reviewing the result of Units test.................................. 171 ii 13.3 Semi-fuzz(Fuzz) testing....................................... 171 13.4 Noise testing............................................. 172 13.5 Chop and slap testing........................................ 172 13.6 Input validation for Units ...................................... 172 13.7 Testing examples in language specific man pages.......................... 174 14 Testing readtags 176 15 Request for extending a parser (or Reporting a bug of parser) 177 15.1 Before reporting........................................... 177 15.2 The content of report......................................... 178 15.3 An example of good report...................................... 179 16 Contributions 181 16.1 Basic rules.............................................. 182 16.2 Before You Start........................................... 182 16.3 Developing a parser......................................... 183 16.4 Testing................................................ 185 16.5 Writing Documents......................................... 186 16.6 Committing and submitting a pull request.............................. 187 16.7 Rules for reviewing a pull request.................................. 189 17 Relationship between other projects 190 17.1 Other tagging engines........................................ 191 17.2 Software using ctags......................................... 191 17.3 Other interesting ctags repositories................................. 192 17.4 Tracking other projects........................................ 192 18 Who we are 199 Bibliography 201 iii Universal Ctags Documentation, Release 0.3.0 Version Draft Authors Universal Ctags developers Web Page https://ctags.io/ Universal Ctags (abbreviated as u-ctags) is a maintained implementation of ctags. ctags generates an index (or tag) file of language objects found in source files for programming languages. This index makes it easy for text editors and other tools to locate the indexed items. Exuberant Ctags (e-ctags) maintained by Darren Hiebert, the ancestor of Universal Ctags, improved traditional ctags with multi-language support, the ability for the user to define new languages searched by regular expressions (called optlib in Universal Ctags), and the ability to generate emacs-style TAGS files. But the activity of the project unfortunately stalled. Universal Ctags has the objective of continuing the development of Exuberant Ctags. Reza Jelveh <[email protected]> initially created a personal fork of Exuberant Ctags on GitHub. As interest and par- ticipation grew, it was decided to move development to a dedicated project as Universal Ctags. The goal of this project is to maintain a common/unified working space where people interested in making ctags better can work together. Some of major features of Universal Ctags are; • more numbers of improved language support – new extended C/C++ language parser, etc. • fully extended optlib (a feature to define a new language parser from a command line) • interactive mode (experimental) The primary documents of Universal Ctags are man pages. Users should first consult the ctags(1), and other man pages if necessary. This guide, which also includes the man pages, is primarily for developers and provides additional information to the man pages, including experimental features. This is a draft document. Proofreading and pull-requests are welcome! Contents 1 CHAPTER 1 Building ctags 1.1 Building with configure (*nix including GNU/Linux) If you are going to build Universal Ctags on a popular GNU/Linux distribution, you can install the tools and libraries that Universal Ctags requires (or may use) as packages. See GNU/Linux distributions about the packages. Like most Autotools-based projects, you need to do: $ git clone https://github.com/universal-ctags/ctags.git