Universal Ctags Documentation Release 0.3.0
Universal Ctags Team
25 September 2021 Contents
1 Building ctags 2 1.1 Building with configure (*nix including GNU/Linux)...... 2 1.2 Building/hacking/using on MS-Windows...... 4 1.3 Building on Mac OS...... 7
2 Man pages 9 2.1 ctags...... 9 2.2 tags...... 33 2.3 ctags-optlib...... 40 2.4 ctags-client-tools...... 46 2.5 ctags-incompatibilities...... 53 2.6 ctags-faq...... 56 2.7 ctags-lang-inko...... 62 2.8 ctags-lang-iPythonCell...... 62 2.9 ctags-lang-julia...... 63 2.10 ctags-lang-python...... 65 2.11 ctags-lang-r...... 69 2.12 ctags-lang-sql...... 70 2.13 ctags-lang-tcl...... 72 2.14 ctags-lang-verilog...... 73 2.15 readtags...... 76
3 Parsers 81 3.1 Asm parser...... 81 3.2 CMake parser...... 81 3.3 The new C/C++ parser...... 82 3.4 The new HTML parser...... 86 3.5 puppetManifest parser...... 87 3.6 The new Python parser...... 87 3.7 The new Tcl parser...... 87 3.8 The Vim parser...... 88 3.9 XSLT parser...... 88
4 Option files 89 4.1 Order of loading option files...... 89 4.2 --options=PATHNAME option...... 90 4.3 Tips for writing an option file...... 90 4.4 Difference from Exuberant Ctags...... 91
5 Output formats 92 5.1 Changes to the tags file format...... 92
i 5.2 Xref output...... 99 5.3 JSON output...... 101
6 Running multiple parsers on an input file 102 6.1 Guest parser: Applying a parser to specified areas of input file...... 102 6.2 Subparser: Tagging definitions of higher (upper) level language...... 103
7 Interactive mode 108 7.1 generate-tags...... 108 7.2 sandbox submode...... 109
8 Other changes 110 8.1 New and extended options...... 111 8.2 Incompatible changes in command line...... 115 8.3 Changes imported from Exuberant Ctags...... 115 8.4 Parser related changes...... 116 8.5 Readtags...... 118
9 Extending ctags with Regex parser (optlib) 120 9.1 Regular expression (regex) engine...... 121 9.2 Regex option argument flags...... 122 9.3 Overriding the letter for file kind...... 127 9.4 Generating fully qualified tags automatically from scope information...... 128 9.5 Multi-line pattern match...... 130 9.6 Advanced pattern matching with multiple regex tables...... 133 9.7 Scheduling a guest parser with _guest regex flag...... 138 9.8 Defining a subparser...... 140 9.9 Translating an option file into C source code (optlib2c)...... 142
10 Optscript, a programming language for extending optlib parsers 143 10.1 Preparation for learning...... 143 10.2 Syntax extension...... 144 10.3 The optscript command...... 144 10.4 Optscript in ctags...... 146 10.5 Recipes...... 147 10.6 Difference between Optscript and PostScript...... 148
11 Extending ctags with a parser written in C 149 11.1 Writing a parser in C...... 149 11.2 Input text stream...... 151 11.3 Output tag stream...... 151 11.4 tokenInfo API...... 153 11.5 Multiple parsers...... 154 11.6 PackCC compiler-compiler...... 161 11.7 Automatic parser guessing (TBW)...... 162 11.8 Managing regular expression parsers (TBW)...... 162 11.9 Ghost kind in regex parser (TBW)...... 162
12 Testing ctags 163 12.1 Tmain: a facility for testing main part...... 163 12.2 Tinst: installation test...... 164 12.3 Fussy syntax checking...... 164 12.4 Finding performance bottleneck...... 165 12.5 Checking coverage...... 165 12.6 Running cppcheck...... 165
13 Testing a parser 166 13.1 Units test facility...... 167 13.2 Reviewing the result of Units test...... 171
ii 13.3 Semi-fuzz(Fuzz) testing...... 171 13.4 Noise testing...... 172 13.5 Chop and slap testing...... 172 13.6 Input validation for Units ...... 172 13.7 Testing examples in language specific man pages...... 174
14 Testing readtags 176
15 Request for extending a parser (or Reporting a bug of parser) 177 15.1 Before reporting...... 177 15.2 The content of report...... 178 15.3 An example of good report...... 179
16 Contributions 181 16.1 Basic rules...... 182 16.2 Before You Start...... 182 16.3 Developing a parser...... 183 16.4 Testing...... 185 16.5 Writing Documents...... 186 16.6 Committing and submitting a pull request...... 187 16.7 Rules for reviewing a pull request...... 189
17 Relationship between other projects 190 17.1 Other tagging engines...... 191 17.2 Software using ctags...... 191 17.3 Other interesting ctags repositories...... 192 17.4 Tracking other projects...... 192
18 Who we are 199
Bibliography 201
iii Universal Ctags Documentation, Release 0.3.0
Version Draft Authors Universal Ctags developers Web Page https://ctags.io/ Universal Ctags (abbreviated as u-ctags) is a maintained implementation of ctags. ctags generates an index (or tag) file of language objects found in source files for programming languages. This index makes it easy for text editors and other tools to locate the indexed items. Exuberant Ctags (e-ctags) maintained by Darren Hiebert, the ancestor of Universal Ctags, improved traditional ctags with multi-language support, the ability for the user to define new languages searched by regular expressions (called optlib in Universal Ctags), and the ability to generate emacs-style TAGS files. But the activity of the project unfortunately stalled. Universal Ctags has the objective of continuing the development of Exuberant Ctags. Reza Jelveh
Contents 1 CHAPTER 1
Building ctags
1.1 Building with configure (*nix including GNU/Linux)
If you are going to build Universal Ctags on a popular GNU/Linux distribution, you can install the tools and libraries that Universal Ctags requires (or may use) as packages. See GNU/Linux distributions about the packages. Like most Autotools-based projects, you need to do:
$ git clone https://github.com/universal-ctags/ctags.git $ cd ctags $ ./autogen.sh $ ./configure --prefix=/where/you/want # defaults to /usr/local $ make $ make install # may require extra privileges depending on where to install
After installation the ctags executable will be in $prefix/bin/. autogen.sh runs autoreconf internally. If you use a (binary oriented) GNU/Linux distribution, autoreconf may be part of the autoconf package. In addition you may have to install automake and/or pkg-config, too.
1.1.1 GNU/Linux distributions
Before running ./autogen.sh, install some packages. On Debian-based systems (including Ubuntu), do:
$ sudo apt install \ gcc make \ pkg-config autoconf automake \ python3-docutils \ libseccomp-dev \ libjansson-dev \ libyaml-dev \ libxml2-dev
On Fedora systems, do:
2 Universal Ctags Documentation, Release 0.3.0
$ sudo dnf install \ gcc make \ pkgconfig autoconf automake \ python3-docutils \ libseccomp-devel \ jansson-devel \ libyaml-devel \ libxml2-devel
1.1.2 Changing the executable’s name
On some systems, like certain BSDs, there is already a ‘ctags’ program in the base system, so it is somewhat inconvenient to have the same name for Universal Ctags. During the configure stage you can now change the name of the created executable. To add a prefix ‘ex’ which will result in ‘ctags’ being renamed to ‘exctags’:
$ ./configure --program-prefix=ex
To completely change the program’s name run the following:
$ ./configure --program-transform-name='s/ctags/my_ctags/; s/etags/myemacs_tags/'
Please remember there is also an ‘etags’ installed alongside ‘ctags’ which you may also want to rename as shown above.
1.1.3 Cross-compilation
The way of cross-compilation is a bit complicated because the build-system of ctags uses packcc, a code generator written in C language. It means that two C compilers should be installed on you build machine; one for compiling packcc, another for compiling ctags. We provide two sets of configure variables to affect these two C compilers: CC, CFLAGS, CPPFLAGS, LDFLAGS variables affect the compiler who compiles ctags. CC_FOR_BUILD, CPPFLAGS_FOR_BUILD, CPPFLAGS_FOR_BUILD, LDFLAGS_FOR_BUILD variables affect the compiler who compiles packcc. When native-compiling, FOO_FOR_BUILD is the same as FOO. Here is an example show you how to use these configure variables:
$ mkdir ./out $ configure \ --host=armv7a-linux-androideabi \ --prefix=`pwd`/out \ --enable-static \ --disable-seccomp \ CC=/usr/local/opt/android-sdk/ndk-bundle/toolchains/llvm/prebuilt/darwin-
˓→x86_64/bin/armv7a-linux-androideabi21-clang \ CFLAGS='-v' \ CPP='/usr/local/opt/android-sdk/ndk-bundle/toolchains/llvm/prebuilt/darwin-
˓→x86_64/bin/armv7a-linux-androideabi21-clang -E' \ CPPFLAGS='-I/Users/leleliu008/.ndk-pkg/pkg/jansson/armeabi-v7a/include -I/
˓→Users/leleliu008/.ndk-pkg/pkg/libyaml/armeabi-v7a/include -I/Users/leleliu008/.
˓→ndk-pkg/pkg/libxml2/armeabi-v7a/include -I/Users/leleliu008/.ndk-pkg/pkg/
˓→libiconv/armeabi-v7a/include --sysroot /usr/local/opt/android-sdk/ndk-bundle/
˓→toolchains/llvm/prebuilt/darwin-x86_64/sysroot -Qunused-arguments -Dftello=ftell
˓→-Dfseeko=fseek' \ LDFLAGS='-L/Users/leleliu008/.ndk-pkg/pkg/jansson/armeabi-v7a/lib -L/Users/
˓→leleliu008/.ndk-pkg/pkg/libyaml/armeabi-v7a/lib -L/Users/leleliu008/.ndk-pkg/pkg/
˓→libxml2/armeabi-v7a/lib -L/Users/leleliu008/.ndk-pkg/pkg/libiconv/armeabi-v7a/ ˓→lib --sysroot /usr/local/opt/android-sdk/ndk-bundle/toolchains/llvm/prebuilt/(continues on next page) ˓→darwin-x86_64/sysroot' \
1.1. Building with configure (*nix including GNU/Linux) 3 Universal Ctags Documentation, Release 0.3.0
(continued from previous page) AR=/usr/local/opt/android-sdk/ndk-bundle/toolchains/llvm/prebuilt/darwin-
˓→x86_64/bin/arm-linux-androideabi-ar \ RANLIB=/usr/local/opt/android-sdk/ndk-bundle/toolchains/llvm/prebuilt/
˓→darwin-x86_64/bin/arm-linux-androideabi-ranlib \ CC_FOR_BUILD=/usr/bin/cc \ CFLAGS_FOR_BUILD='-v' \ PKG_CONFIG_PATH=/Users/leleliu008/.ndk-pkg/pkg/libiconv/armeabi-v7a/lib/
˓→pkgconfig:/Users/leleliu008/.ndk-pkg/pkg/libxml2/armeabi-v7a/lib/pkgconfig:/
˓→Users/leleliu008/.ndk-pkg/pkg/libyaml/armeabi-v7a/lib/pkgconfig:/Users/
˓→leleliu008/.ndk-pkg/pkg/jansson/armeabi-v7a/lib/pkgconfig \ PKG_CONFIG_LIBDIR=/Users/leleliu008/.ndk-pkg/pkg ... $ make ... $ make install ... $ ls out/bin ctags readtags
Simpler example for aarch64-linux-gnu can be found in circle.yml in the source tree.
1.2 Building/hacking/using on MS-Windows
Maintainer Frank Fesevur
This part of the documentation is written by Frank Fesevur, co-maintainer of Universal Ctags and the maintainer of the Windows port of this project. It is still very much a work in progress. Things still need to be written down, tested or even investigated. When building for Windows you should be aware that there are many compilers and build environments available. This is a summary of available options and things that have been tested so far.
1.2.1 Compilers
There are many compilers for Windows. Compilers not mentioned here may work but are not tested.
Microsoft Visual Studio https://www.visualstudio.com/ Obviously there is Microsoft Visual Studio 2013. Many professional developers targeting Windows use Visual Studio. Visual Studio comes in a couple of different editions. Their Express and Community editions are free to use, but a Microsoft-account is required to download the .iso and when you want to continue using it after a 30-days trial period. Other editions of Visual Studio must be purchased. Installing Visual Studio will give you the IDE, the command line compilers and the MS-version of make named nmake. Note that ctags cannot be built with Visual Studio older than 2013 anymore. There is C99 (or C11) coding used that generates syntax errors with VS2012 and older. This could affect compilers from other vendors as well.
GCC
There are three flavors of GCC for Windows: • MinGW http://www.mingw.org • MinGW-w64 http://mingw-w64.sourceforge.net
1.2. Building/hacking/using on MS-Windows 4 Universal Ctags Documentation, Release 0.3.0
• TDM-GCC http://tdm-gcc.tdragon.net MinGW started it all, but development stalled for a while and no x64 was available. Then the MinGW-w64 fork emerged. It started as a 64-bit compiler, but soon they included both a 32-bit and a 64-bit compiler. But the name remained, a bit confusing. Another fork of MinGW is TDM-GCC. It also provides both 32-bit and 64-bit compilers. All have at least GCC 4.8. MinGW-w64 appears to be the most used flavor of MinGW at this moment. Many well known programs that originate from GNU/Linux use MinGW-w64 to compile their Windows port.
1.2.2 Building ctags from the command line
Microsoft Visual Studio
Most users of Visual Studio will use the IDE and not the command line to compile a project. But by default a shortcut to the command prompt that sets the proper path is installed in the Start Menu. When this command prompt is used nmake -f mk_mvc.mak will compile ctags. You can also go into the win32 subdirectory and run msbuild ctags_vs2013.sln for the default build. Use msbuild ctags_vs2013.sln / p:Configuration=Release to specifically build a release build. MSBuild is what the IDE uses internally and therefore will produce the same files as the IDE. If you want to build an iconv enabled version, you must specify WITH_ICONV=yes and ICONV_DIR like below:
nmake-f mk_mvc.mak WITH_ICONV=yes ICONV_DIR=path/to/iconvlib
If you want to build a debug version using mk_mvc.mak, you must specify DEBUG=1 like below:
nmake-f mk_mvc.mak DEBUG=1
If you want to create PDB files for debugging even for a release version, you must specify PDB=1 like below:
nmake-f mk_mvc.mak PDB=1
GCC
General All the GCC’s come with installers or with zipped archives. Install or extract them in a directory without spaces. GNU Make builds for Win32 are available as well, and sometimes are included with the compilers. Make sure it is in your path, for instance by copying the make.exe in the bin directory of your compiler. Native win32 versions of the GNU/Linux commands cp, rm and mv can be useful. rm is almost always used in by the clean target of a makefile. CMD Any Windows includes a command prompt. Not the most advanced, but it is enough to do the build tasks. Make sure the path is set properly and make -f mk_mingw.mak should do the trick. If you want to build an iconv enabled version, you must specify WITH_ICONV=yes like below:
make-f mk_mingw.mak WITH_ICONV=yes
If you want to build a debug version, you must specify DEBUG=1 like below:
make-f mk_mingw.mak DEBUG=1
MSYS / MSYS2 From their site: MSYS is a collection of GNU utilities such as bash, make, gawk and grep to allow building of applications and programs which depend on traditional UNIX tools to be present. It is intended to supplement MinGW and the deficiencies of the cmd shell.
1.2. Building/hacking/using on MS-Windows 5 Universal Ctags Documentation, Release 0.3.0
MSYS comes in two flavors; the original from MinGW and MSYS2. See https://www.msys2.org/ about MSYS2. MSYS is old but still works. You can build ctags with it using make -f mk_mingw.mak. The Autotools are too old on MSYS so you cannot use them. MSYS2 is a more maintained version of MSYS, but specially geared towards MinGW-w64. You can also use Autotools to build ctags. If you use Autotools you can enable parsers which require jansson, libxml2 or libyaml, and can also do the Units testing with make units. The following packages are needed to build a full-featured version: • base-devel (make, autoconf) • mingw-w64-{i686,x86_64}-toolchain (mingw-w64-{i686,x86_64}-gcc, mingw-w64-{i686,x86_64}-pkg- config) • mingw-w64-{i686,x86_64}-jansson • mingw-w64-{i686,x86_64}-libxml2 • mingw-w64-{i686,x86_64}-libyaml • mingw-w64-{i686,x86_64}-xz If you want to build a single static-linked binary, you can use the following command:
./autogen.sh ./configure --disable-external-sort --enable-static make
--disable-external-sort is a recommended option for Windows builds. Cygwin Cygwin provides ports of many GNU/Linux tools and a POSIX API layer. This is the most complete way to get the GNU/Linux terminal feel under Windows. Cygwin has a setup that helps you install all the tools you need. One drawback of Cygwin is that it has poor performance. It is easy to build a Cygwin version of ctags using the normal GNU/Linux build steps. This ctags.exe will depend on cygwin1.dll and should only be used within the Cygwin ecosystem. The following packages are needed to build a full-featured version: • libiconv-devel • libjansson-devel • libxml2-devel • libyaml-devel Cygwin has packages with a recent version of MinGW-w64 as well. This way it is easy to cross-compile a native Windows application with make -f mk_mingw.mak CC=i686-w64-mingw32-gcc. You can also build a native Windows version using Autotools.
./autogen.sh ./configure --host=i686-w64-mingw32 --disable-external-sort make
If you use Autotools you can also do the Units testing with make units. Some anti-virus software slows down the build and test process significantly, especially when ./configure is running and during the Units tests. In that case it could help to temporarily disable them. But be aware of the risks when you disable your anti-virus software. Cross-compile from GNU/Linux All major distributions have both MinGW and MinGW-w64 packages. Cross-compiling works the same way as with Cygwin. You cannot do the Windows based Units tests on GNU/Linux.
1.2. Building/hacking/using on MS-Windows 6 Universal Ctags Documentation, Release 0.3.0
1.2.3 Building ctags with IDEs
I have no idea how things work for most GNU/Linux developers, but most Windows developers are used to IDEs. Not many use a command prompt and running the debugger from the command line is not a thing a Windows developers would normally do. Many IDEs exist for Windows, I use the two below.
Microsoft Visual Studio
As already mentioned Microsoft Visual Studio 2013 has the free Express and Community editions. For ctags the Windows Desktop Express Edition is enough to get the job done. The IDE has a proper debugger. Project files for VS2013 can be found in the win32 directory. Please know that when files are added to the sources.mak, these files need to be added to the .vcxproj and .vcx- proj.filters files as well. The XML of these files should not be a problem.
Code::Blocks http://www.codeblocks.org/ Code::Blocks is a decent GPL-licensed IDE that has good gcc and gdb integration. The TDM-GCC that can be installed together with Code::Blocks works fine and I can provide a project file. This is an easy way to have a free - free as in beer as well as in speech - solution and to have the debugger within the GUI as well.
1.2.4 Other differences between Microsoft Windows and GNU/Linux
There other things where building ctags on Microsoft Windows differs from building on GNU/Linux. • Filenames on Windows file systems are case-preserving, but not case-sensitive. • Windows file systems use backslashes "\" as path separators, but paths with forward slashes "/" are no problem for a Windows program to recognize, even when a full path (include drive letter) is used. • The default line-ending on Windows is CRLF. A tags file generated by the Windows build of ctags will contain CRLF. • The tools used to build ctags do understand Unix-line endings without problems. There is no need to convert the line-ending of existing files in the repository. • Due to the differences between the GNU/Linux and Windows C runtime library there are some things that need to be added to ctags to make the program as powerful as it is on GNU/Linux. At this moment regex and fnmatch are borrowed from glibc. mkstemp() is taken from MinGW-w64’s runtime library. scandir() is taken from Pacemaker. • Units testing needs a decent bash shell, some unix-like tools (e.g. diff, sed) and Python 3.5 or later. It is only tested using Cygwin or MSYS2.
1.3 Building on Mac OS
Maintainer Cameron Eagans
This part of the documentation is written by Cameron Eagans, a co-maintainer of Universal Ctags and the main- tainer of the OSX packaging of this project.
1.3. Building on Mac OS 7 Universal Ctags Documentation, Release 0.3.0
1.3.1 Build Prerequisites
Building ctags on OSX should be no different than building on GNU/Linux. The same toolchains are used, and the Mac OS packaging scripts use autotools and make (as you’d expect). You may need to install the xcode command line tools. You can install the entire xcode distribution from the App Store, or for a lighter install, you can simply run xcode-select --install to only install the compilers and such. See https://stackoverflow.com/a/9329325 for more information. Once your build toolchain is installed, proceed to the next section. At this point, if you’d like to build from an IDE, you’ll have to figure it out. Building ctags is a pretty straightfor- ward process that matches many other projects and most decent IDEs should be able to handle it.
Building Manually (i.e. for development)
You can simply run the build instructions in README.md.
Building with Homebrew
Homebrew (https://brew.sh/) is the preferred method for installing Universal Ctags for end users. Currently, the process for installing with Homebrew looks like this: brew tap universal-ctags/universal-ctags brew install--HEAD universal-ctags
Eventually, we hope to move the Universal-ctags formula to the main Homebrew repository, but since we don’t have any tagged releases at this point, it’s a head-only formula and wouldn’t be accepted. When we have a tagged release, we’ll submit a PR to Homebrew. If you’d like to help with the Homebrew formula, you can find the repository here: https://github.com/ universal-ctags/homebrew-universal-ctags
1.3.2 Differences between OSX and GNU/Linux
There other things where building ctags on OSX differs from building on GNU/Linux. • Filenames on HFS+ (the Mac OS filesystem) are case-preserving, but not case-sensitive in 99% of configu- rations. If a user manually formats their disk with a case sensitive version of HFS+, then the filesystem will behave like normal GNU/Linux systems. Depending on users doing this is not a good thing.
1.3.3 Contributing
This documentation is very much a work in progress. If you’d like to contribute, submit a PR and mention @cweagans for review.
1.3. Building on Mac OS 8 CHAPTER 2
Man pages
2.1 ctags
Generate tag files for source code Version 5.9.0 Manual group Universal Ctags Manual section 1
2.1.1 SYNOPSIS ctags [
2.1.2 DESCRIPTION
The ctags and etags (see -e option) programs (hereinafter collectively referred to as ctags, except where distin- guished) generate an index (or “tag”) file for a variety of language objects found in source file(s). This tag file allows these items to be quickly and easily located by a text editor or other utilities (client tools). A tag signifies a language object for which an index entry is available (or, alternatively, the index entry created for that object). Alternatively, ctags can generate a cross reference file which lists, in human readable form, information about the various language objects found in a set of source files. Tag index files are supported by numerous editors, which allow the user to locate the object associated with a name appearing in a source file and jump to the file and line which defines the name. See the manual of your favorite editor about utilizing ctags command and the tag index files in the editor. ctags is capable of generating different kinds of tags for each of many different languages. For a complete list of supported languages, the names by which they are recognized, and the kinds of tags which are generated for each, see the --list-languages and --list-kinds-full options. This man page describes Universal Ctags, an implementation of ctags derived from Exuberant Ctags. The major incompatible changes between Universal Ctags and Exuberant Ctags are enumerated in ctags-incompatibilities(7).
9 Universal Ctags Documentation, Release 0.3.0
One of the advantages of Exuberant Ctags is that it allows a user to define a new parser from the command line. Extending this capability is one of the major features of Universal Ctags. ctags-optlib(7) describes how the capability is extended. Newly introduced experimental features are not explained here. If you are interested in such features and ctags internals, visit https://docs.ctags.io/.
2.1.3 COMMAND LINE INTERFACE
Despite the wealth of available options, defaults are set so that ctags is most commonly executed without any options (e.g. “ctags *”, or “ctags -R”), which will create a tag file in the current directory for all recognized source files. The options described below are provided merely to allow custom tailoring to meet special needs. Note that spaces separating the single-letter options from their parameters are optional. Note also that the boolean parameters to the long form options (those beginning with -- and that take a [=(yes|no)] parameter) may be omitted, in which case =yes is implied. (e.g. --sort is equivalent to --sort=yes). Note further that =1, =on, and =true are considered synonyms for =yes, and that =0, =off, and =false are considered synonyms for =no. Some options are either ignored or useful only when used while running in etags mode (see -e option). Such options will be noted.
Letters and names
Some options take one-letter flags as parameters (e.g. --kinds-
--kinds-C=+pLl --kinds-C=+{prototype}{label}{local} --kinds-C=+{prototype}L{local}
Note that braces may be meta characters in your shell. Put single quotes in such case. --list-... options shows one-letter flags and associated long-name flags.
List options
Universal Ctags introduces many --list-... options that provide the internal data of Universal Ctags (See “Listing Options”). Both users and client tools may use the data. --with-list-header and --machinable options adjust the output of the most of --list-... options. The default setting (--with-list-header=yes and --machinable=no) is for using interactively from a terminal. The header that explains the meaning of columns is simply added to the output, and each column is aligned in all lines. The header line starts with a hash (’#’) character. For scripting in a client tool, --with-list-header=no and --machinable=yes may be useful. The header is not added to the output, and each column is separated by tab characters. Note the order of columns will change in the future release. However, labels in the header will not change. So by scanning the header, a client tool can find the index for the target column.
2.1. ctags 10 Universal Ctags Documentation, Release 0.3.0
2.1.4 OPTIONS
ctags has more options than listed here. Options starting with an underscore character, such as --_echo=
Input/Output File Options
--exclude=
2.1. ctags 11 Universal Ctags Documentation, Release 0.3.0
behaviors, either explicitly specify the files or pipe the output of find(1) into “ctags -L -” instead. See, also, the --exclude and --maxdepth to limit recursion. Note: This option is not supported on all platforms at present. It is available if the output of the --help option includes this option. -R Equivalent to --recurse. -L
Output Format Options
--format=(1|2) Change the format of the output tag file. Currently the only valid values for level are 1 or 2. Level 1 specifies the original tag file format and level 2 specifies a new extended format containing extension fields (but in a manner which retains backward-compatibility with original vi(1) implementations). The default level is 2. [Ignored in etags mode] --output-format=(u-ctags|e-ctags|etags|xref|json) Specify the output format. The default is u-ctags. See tags(5) for u-ctags and e-ctags. See -e for etags, and -x for xref. json format is available only if the ctags executable is built with libjansson. See ctags-client-tools(7) for more about json format. -e Same as --output-format=etags. Enable etags mode, which will create a tag file for use with the Emacs editor. Alternatively, if ctags is invoked by a name containing the string “etags” (either by renaming, or creating a link to, the executable), etags mode will be enabled. -x Same as --output-format=xref. Print a tabular, human-readable cross reference (xref) file to standard output instead of generating a tag file. The information contained in the output includes: the tag name; the kind of tag; the line number, file name, and source line (with extra white space condensed) of the file which defines the tag. No tag file is written and all options affecting tag file output will be ignored. Example applications for this feature are generating a listing of all functions located in a source file (e.g. “ctags -x --kinds-c=f file”), or generating a list of all externally visible global variables located in a source file (e.g. “ctags -x --kinds-c=v --extras=-F file”). --sort=(yes|no|foldcase) Indicates whether the tag file should be sorted on the tag name (default is yes). Note that the original vi(1) required sorted tags. The foldcase value specifies case insensitive
2.1. ctags 12 Universal Ctags Documentation, Release 0.3.0
(or case-folded) sorting. Fast binary searches of tag files sorted with case-folding will require special support from tools using tag files, such as that found in the ctags readtags library, or Vim version 6.2 or higher (using “set ignorecase”). [Ignored in etags mode] -u Equivalent to --sort=no (i.e. “unsorted”). --etags-include=
Language Selection and Mapping Options
--language-force=(|all) Specifies the languages for which tag generation is enabled, with
containing a comma-separated list of language names (case-insensitive; either built-in or user- defined). If the first language of
is not preceded by either a ‘+’ or ‘-’, the current list (the current settings of enabled/disabled languages managed in ctags internally) will be cleared before adding or removing the languages in
. Until a ‘-’ is encountered, each language in the
will be added to the current list. As either the ‘+’ or ‘-’ is encountered in the
, the languages following it are added or removed from the current list, respectively. Thus, it becomes simple to replace the current list with a new one, or to add or remove languages from the current list. The actual list of files for which tags will be generated depends upon the language extension mapping in effect (see the --langmap option). Note that the most of languages, including user-defined languages, are enabled unless explicitly disabled using this option. Language names included in list may be any built-in language or one previously defined with --langdef. The default is all, which is also accepted as a valid argument. See the --list-languages option for a list of the all (built-in and user-defined) language names. Note --languages= option works cumulative way; the option can be specified with different arguments multiple times in a command line. --alias-
2.1. ctags 13 Universal Ctags Documentation, Release 0.3.0
--alias-all= This clears aliases setting of all languages. --alias-all=default This restores the default languages aliases for all languages. --guess-language-eagerly Looks into the file contents for heuristically guessing the proper language parser. See “Determining file language”. -G Equivalent to --guess-language-eagerly. --langmap=
Tags File Contents Options
See “TAG ENTRIES” about fields, kinds, roles, and extras. --excmd=(number|pattern|mix|combine) Determines the type of EX command used to locate tags in the source file. [Ignored in etags mode]
2.1. ctags 14 Universal Ctags Documentation, Release 0.3.0
The valid values for type (either the entire word or the first letter is accepted) are: number Use only line numbers in the tag file for locating tags. This has four advantages: 1. Significantly reduces the size of the resulting tag file. 2. Eliminates failures to find tags because the line defining the tag has changed, causing the pattern match to fail (note that some editors, such as vim, are able to recover in many such instances). 3. Eliminates finding identical matching, but incorrect, source lines (see “BUGS”). 4. Retains separate entries in the tag file for lines which are identical in content. In pattern mode, duplicate entries are dropped because the search patterns they generate are identical, making the duplicate entries useless. However, this option has one significant drawback: changes to the source files can cause the line numbers recorded in the tag file to no longer correspond to the lines in the source file, causing jumps to some tags to miss the target definition by one or more lines. Basically, this option is best used when the source code to which it is applied is not subject to change. Selecting this option type causes the following options to be ignored: -B, -F. number type is ignored in Xref and JSON output formats. Use --_xformat="...%n" for Xref output format, or --fields=+n-P for JSON output format. pattern Use only search patterns for all tags, rather than the line numbers usually used for macro defini- tions. This has the advantage of not referencing obsolete line numbers when lines have been added or removed since the tag file was generated. mixed In this mode, patterns are generally used with a few exceptions. For C, line numbers are used for macro definition tags. For Fortran, line numbers are used for common blocks because their corre- sponding source lines are generally identical, making pattern searches useless for finding all matches. This was the default format generated by the original ctags and is, therefore, retained as the default for this option. combine Concatenate the line number and pattern with a semicolon in between. -n Equivalent to --excmd=number. -N Equivalent to --excmd=pattern. --extras=[+|-][
2.1. ctags 15 Universal Ctags Documentation, Release 0.3.0
to exclude it. In the absence of any preceding ‘+’ or ‘-’ sign, only those fields explicitly listed in flags will be included in the output (i.e. overriding the default set). All fields are included if ‘*’ is given. This option is ignored if the option --format=1 (legacy tag file format) has been specified. Use --fields-
2.1. ctags 16 Universal Ctags Documentation, Release 0.3.0
In such a situation, there is a concern about conflicting field names; mixing tags files generated by multiple tags generators including Universal Ctags is difficult. This option provides a workaround for such station.
$ ctags --fields='{line}{end}' -o - hello.c main hello.c /^main(int argc, char **argv)$/;" f line:3 end:6 $ ctags --put-field-prefix --fields='{line}{end}' -o - hello.c main hello.c /^main(int argc, char **argv)$/;" f line:3 ˓→UCTAGSend:6
In the above example, the prefix is put to end field which is newly introduced in Universal Ctags. --roles-(
Option File Options
--options=
2.1. ctags 17 Universal Ctags Documentation, Release 0.3.0
--options-maybe=
optlib Options
See ctags-optlib(7) for details of each option. --kinddef-
Language Specific Options
--if0[=(yes|no)] Indicates a preference as to whether code within an “#if 0” branch of a preprocessor conditional should be examined for non-macro tags (macro tags are always included). Because the intent of this construct is to disable code, the default value of this option is no (disabled). Note that this indicates a preference only and does not guarantee skipping code within an “#if 0” branch, since the fall-back algorithm used to generate tags when preprocessor conditionals are too complex follows all branches of a conditional. --line-directives[=(yes|no)] Specifies whether #line directives should be recognized. These are present in the output of a preprocessor and contain the line number, and possibly the file name, of the original source file(s) from which the preprocessor output file was generated. This option is off by default. When enabled, this option will cause ctags to generate tag entries marked with the file names and line numbers of their locations original source file(s), instead of their actual locations in the preprocessor output. The actual file names placed into the tag file will have the same leading path components as the preprocessor output file, since it is assumed that the original source files are located relative to the preprocessor output file (unless, of course, the #line directive specifies an absolute path). Note: This option is generally only useful when used together with the --excmd=number (-n) option. Also, you may have to use either the --langmap or --language-force option if the extension of the preprocessor output file is not known to ctags. -D |default) Specifies a
of file extensions, separated by periods, which are to be interpreted as include (or header) files. To indicate files having no extension, use a period not followed by a non-period character (e.g. ‘.’, ..x, .x.). This option only affects how the scoping of particular kinds of tags are interpreted (i.e. whether or not they are considered as globally visible or visible only within the file in which they are defined); it does not map the extension to any particular language. Any tag which is located in a non-include file and cannot be seen (e.g. linked to) from another file is considered to have file-limited (e.g. static) scope. No kind of tag appearing in an include file will be considered to have file-limited scope. If the first character in the list is ‘+’, then the extensions in the list will be appended to the current list; otherwise, the list will replace the current list. See, also, the fileScope/F flag of --extras option.
2.1. ctags 18 Universal Ctags Documentation, Release 0.3.0
The default list is .h.H.hh.hpp.hxx.h++.inc.def. To restore the default list, specify “-h default”. Note that if an extension supplied to this option is not already mapped to a particular language (see “De- termining file language”, above), you will also need to use either the --map-
int foo ARGDECL4(void *, ptr, long int, nbytes)
In the above example, the macro ARGDECL4 would be mistakenly interpreted to be the name of the function instead of the correct name of foo. Specifying “-I ARGDECL4” results in the correct behavior.
/* creates an RCS version string in module */ MODULE_VERSION("$Revision$")
In the above example the macro invocation looks too much like a function definition because it is not followed by a semicolon (indeed, it could even be followed by a global variable definition that would look much like a K&R style function parameter declaration). In fact, this seeming function definition could possibly even cause the rest of the file to be skipped over while trying to complete the definition. Specifying “-I MODULE_VERSION+” would avoid such a problem.
CLASS Example { // your content here };
The example above uses CLASS as a preprocessor macro which expands to something different for each platform. For instance CLASS may be defined as class __declspec(dllexport) on Win32 plat- forms and simply class on UNIX. Normally, the absence of the C++ keyword class would cause the source file to be incorrectly parsed. Correct behavior can be restored by specifying “-I CLASS=class”. --param-
2.1. ctags 19 Universal Ctags Documentation, Release 0.3.0
Listing Options
--list-aliases[=(
2.1. ctags 20 Universal Ctags Documentation, Release 0.3.0
--list-kinds-full[=(
$ ctags --kinds-C=+'{local}' --list-kinds-full \ | grep -E '^(#|C\+\+ .* local)' #LANGUAGE LETTER NAME ENABLED REFONLY NROLES MASTER DESCRIPTION C++ l local yes no 0 C local variables $ ctags --kinds-C=-'{local}' --list-kinds-full \ | grep -E '^(#|C\+\+ .* local)' #LANGUAGE LETTER NAME ENABLED REFONLY NROLES MASTER DESCRIPTION C++ l local no no 0 C local variables
You see ENABLED field of local kind of C++ language is changed Though local kind of C lan- guage is enabled/disabled. If you swap the languages, you see the same result. DESCRIPTION Human readable description for the kind. --list-languages Lists the names of the languages understood by ctags, and then exits. These lan- guage names are case insensitive and may be used in many other options like --language-force, --languages, --kinds-
2.1. ctags 21 Universal Ctags Documentation, Release 0.3.0
--list-maps[=(
Miscellaneous Options
--help Prints to standard output a detailed usage description, and then exits. -? Equivalent to --help. --help-full Prints to standard output a detailed usage description including experimental features, and then exits. Visit https://docs.ctags.io/ for information about the latest exciting experimental features. --license Prints a summary of the software license to standard output, and then exits. --print-language Just prints the language parsers for specified source files, and then exits. --quiet[=(yes|no)] Write fewer messages (default is no). --totals[=(yes|no|extra)] Prints statistics about the source files read and the tag file written during the current invocation of ctags. This option is no by default. The extra value prints parser specific statistics for parsers gathering such information.
2.1. ctags 22 Universal Ctags Documentation, Release 0.3.0
--verbose[=(yes|no)] Enable verbose mode. This prints out information on option processing and a brief message describing what action is being taken for each file considered by ctags. Normally, ctags does not read command line arguments until after options are read from the configuration files (see “FILES”, below). However, if this option is the first argument on the command line, it will take effect before any options are read from these sources. The default is no. -V Equivalent to --verbose. --version Prints a version identifier for ctags to standard output, and then exits. This is guaranteed to always contain the string “Universal Ctags”.
Obsoleted Options
These options are kept for backward-compatibility with Exuberant Ctags. -w This option is silently ignored for backward-compatibility with the ctags of SVR4 Unix. --file-scope[=(yes|no)] This options is removed. Use --extras=[+|-]F or --extras=[+|-]{fileScope} instead. --extra=[+|-][
2.1.5 OPERATIONAL DETAILS
As ctags considers each source file name in turn, it tries to determine the language of the file by applying tests described in “Determining file language”. If a language was identified, the file is opened and then the appropriate language parser is called to operate on the currently open file. The parser parses through the file and adds an entry to the tag file for each language object it is written to handle. See “TAG FILE FORMAT”, below, for details on these entries.
Notes for C/C++ Parser
This implementation of ctags imposes no formatting requirements on C code as do legacy implementations. Older implementations of ctags tended to rely upon certain formatting assumptions in order to help it resolve coding dilemmas caused by preprocessor conditionals. In general, ctags tries to be smart about conditional preprocessor directives. If a preprocessor conditional is encountered within a statement which defines a tag, ctags follows only the first branch of that conditional (except in the special case of #if 0, in which case it follows only the last branch). The reason for this is that failing to pursue only one branch can result in ambiguous syntax, as in the following example:
#ifdef TWO_ALTERNATIVES struct { #else union { #endif short a; long b; }
Both branches cannot be followed, or braces become unbalanced and ctags would be unable to make sense of the syntax. If the application of this heuristic fails to properly parse a file, generally due to complicated and inconsistent pairing within the conditionals, ctags will retry the file using a different heuristic which does not selectively
2.1. ctags 23 Universal Ctags Documentation, Release 0.3.0
follow conditional preprocessor branches, but instead falls back to relying upon a closing brace (’}’) in column 1 as indicating the end of a block once any brace imbalance results from following a #if conditional branch. ctags will also try to specially handle arguments lists enclosed in double sets of parentheses in order to accept the following conditional construct:
extern void foo __ARGS((int one, char two));
Any name immediately preceding the ‘((’ will be automatically ignored and the previous name will be used. C++ operator definitions are specially handled. In order for consistency with all types of operators (overloaded and conversion), the operator name in the tag file will always be preceded by the string “operator ” (i.e. even if the actual operator definition was written as “operator<<”). After creating or appending to the tag file, it is sorted by the tag name, removing identical tag lines.
Determining file language
File name mapping
Unless the --language-force option is specified, the language of each source file is automatically selected based upon a mapping of file names to languages. The mappings in effect for each language may be displayed using the --list-maps option and may be changed using the --langmap or --map-
Heuristically guessing
If ctags cannot select a parser from the mapping of file names, various heuristic tests are conducted to determine the language: template file name testing If the file name has an .in extension, ctags applies the mapping to the file name without the extension. For example, config.h is tested for a file named config.h.in. “interpreter” testing The first line of the file is checked to see if the file is a #! script for a recognized language. ctags looks for a parser having the same name. If ctags finds no such parser, ctags looks for the name in alias lists. For example, consider if the first line is #!/bin/sh. Though ctags has a “shell” parser, it doesn’t have a “sh” parser. However, sh is listed as an alias for shell, therefore ctags selects the “shell” parser for the file. An exception is env. If env is specified (for example “#!/usr/bin/env python”), ctags reads more lines to find real interpreter specification. To display the list of aliases, use --list-aliases option. To add an item to the list or to remove an item from the list, use the --alias-
2.1. ctags 24 Universal Ctags Documentation, Release 0.3.0
“emacs mode at the first line” testing The Emacs editor has multiple editing modes specialized for program- ming languages. Emacs can recognize a marker called modeline in a file and utilize the marker for the mode selection. This heuristic test does the same as what Emacs does. ctags treats MODE as a name of interpreter and applies the same rule of “interpreter” testing if the first line has one of the following patterns:
-*- mode: MODE- *-
or
-*- MODE- *-
“emacs mode at the EOF” testing Emacs editor recognizes another marker at the end of file as a mode specifier. This heuristic test does the same as what Emacs does. ctags treats MODE as a name of an interpreter and applies the same rule of “interpreter” heuristic testing, if the lines at the tail of the file have the following pattern:
Local Variables: ... mode: MODE ... End:
3000 characters are sought from the end of file to find the pattern. “vim modeline” testing Like the modeline of the Emacs editor, Vim editor has the same concept. ctags treats TYPE as a name of interpreter and applies the same rule of “interpreter” heuristic testing if the last 5 lines of the file have one of the following patterns:
filetype=TYPE
or
ft=TYPE
“PHP marker” testing If the first line is started with
$ ctags --print-language config.h.in input.m input.unknown config.h.in: C++ input.m: MatLab input.unknown: NONE
NONE means that ctags does not select any parser for the file.
2.1.6 TAG FILE FORMAT
This section describes the tag file format briefly. See tags(5) and ctags-client-tools(7) for more details. When not running in etags mode, each entry in the tag file consists of a separate line, each looking like this, called regular tags, in the most general case:
2.1. ctags 25 Universal Ctags Documentation, Release 0.3.0
The fields and separators of these lines are specified as follows: 1.
!_TAG_FILE_FORMAT 2 /extended format; --format=1 will not append ;" to
˓→lines/ !_TAG_FILE_SORTED 1 /0=unsorted, 1=sorted, 2=foldcase/ ...
--pseudo-tags=[+|-](
2.1.7 TAG ENTRIES
A tag is an index for a language object. The concept of a tag and related items in Exuberant Ctags are refined and extended in Universal Ctags. A tag is categorized into definition tags or reference tags. In general, Exuberant Ctags only tags definitions of language objects: places where newly named language objects are introduced. Universal Ctags, on the other hand, can also tag references of language objects: places where named language objects are used. However, support for generating reference tags is new and limited to specific areas of specific languages in the current version.
Extension fields
A tag can record various information, called extension fields. Extension fields are tab-separated key-value pairs appended to the end of the EX command as a comment, as described above. These key value pairs appear in the general form key:value. In addition, information on the scope of the tag definition may be available, with the key portion equal to some language-dependent construct name and its value the name declared for that construct in the program. This scope entry indicates the scope in which the tag was found. For example, a tag generated for a C structure member would have a scope looking like struct:myStruct.
2.1. ctags 26 Universal Ctags Documentation, Release 0.3.0
--fields=[+|-][
Kinds kind is a field which represents the kind of language object specified by a tag. Kinds used and defined are very different between parsers. For example, C language defines macro, function, variable, typedef, etc. --kinds-(
2.1. ctags 27 Universal Ctags Documentation, Release 0.3.0
$ ctags -o - kinds.c foo kinds.c /^int foo() {$/;" f typeref:typename:int $ ctags --fields=+k -o - kinds.c foo kinds.c /^int foo() {$/;" f typeref:typename:int $ ctags --fields=+K -o - kinds.c foo kinds.c /^int foo() {$/;" function typeref:typename:int $ ctags --fields=+z -o - kinds.c foo kinds.c /^int foo() {$/;" kind:f typeref:typename:int $ ctags --fields=+zK -o - kinds.c foo kinds.c /^int foo() {$/;" kind:function typeref:typename:int
Roles
Role is a newly introduced concept in Universal Ctags. Role is a concept associated with reference tags, and is not implemented widely yet. As described previously in “Kinds”, the kind field represents the type of language object specified with a tag, such as a function vs. a variable. Specific kinds are defined for reference tags, such as the C++ kind header for header file, or Java kind package for package statements. For such reference kinds, a roles field can be added to distinguish the role of the reference kind. In other words, the kind field identifies the what of the language object, whereas the roles field identifies the how of a referenced language object. Roles are only used with specific kinds. For a definition tag, this field takes def as a value. For example, Baz is tagged as a reference tag with kind package and with role imported with the following code.
package Bar; import Baz;
class Foo { // ... }
$ ctags --fields=+KEr -uo - roles.java Bar roles.java /^package Bar;$/;" package roles:def Foo roles.java /^class Foo {$/;" class roles:def $ ctags --fields=+EKr --extras=+r -uo - roles.java Bar roles.java /^package Bar;$/;" package roles:def Baz roles.java /^import Baz;$/;" package roles:imported
˓→extras:reference Foo roles.java /^class Foo {$/;" class roles:def
--roles-(
Extras
Generally, ctags tags only language objects appearing in source files, as is. In other words, a value for a name: field should be found on the source file associated with the name:. An extra type tag (extra) is for tagging a language object with a processed name, or for tagging something not associated with a language object. A typical extra tag is qualified, which tags a language object with a class-qualified or scope-qualified name. --extras-(
2.1. ctags 28 Universal Ctags Documentation, Release 0.3.0
Inquire the output of ctags --list-extras for the list of extras. The meaning of major extras is as follows (long-name flag/one-letter flag): anonymous/none Include an entry for the language object that has no name like lambda function. This extra has no one-letter flag and is enabled by default. The extra tag is useful as a placeholder to fill scope fields for language objects defined in a language object with no name.
struct { double x, y; } p= { .x= 0.0, .y= 0.0};
‘x’ and ‘y’ are the members of a structure. When filling the scope fields for them, ctags has trouble because the struct where ‘x’ and ‘y’ belong to has no name. For overcoming the trouble, ctags generates an anonymous extra tag for the struct and fills the scope fields with the name of the extra tag.
$ ctags --fields=-f -uo - input.c __anon9f26d2460108 input.c /^struct {$/;" s x input.c /^ double x, y;$/;" m struct:__
˓→anon9f26d2460108 y input.c /^ double x, y;$/;" m struct:__
˓→anon9f26d2460108 p input.c /^} p = { .x = 0.0, .y = 0.0 };$/;" v
˓→typeref:struct:__anon9f26d2460108
The above tag output has __anon9f26d2460108 as an anonymous extra tag. The typeref field of ‘p’ also receives the benefit of it. fileScope/F Indicates whether tags scoped only for a single file (i.e. tags which cannot be seen outside of the file in which they are defined, such as language objects with static modifier of C language) should be included in the output. See also the -h option. This extra tag is enabled by default. Add --extras=-F option not to output tags scoped only for a single-file. This is the replacement for --file-scope option of Exuberant Ctags.
static int f() { return 0; } int g() { return 0; }
$ ctags -uo - filescope.c f filescope.c /^static int f() {$/;" f typeref:typename:int
˓→ file: g filescope.c /^int g() {$/;" f typeref:typename:int $ ctags --extras=-F -uo - filescope.c g filescope.c /^int g() {$/;" f typeref:typename:int
inputFile/f Include an entry for the base file name of every source file (e.g. example.c), which addresses the first line of the file. This flag is the replacement for --file-tags hidden option of Exuberant Ctags. If the end: field is enabled, the end line number of the file can be attached to the tag. (However, ctags omits the end: field if no newline is in the file like an empty file.) By default, ctags doesn’t create the inputFile/f extra tag for the source file when ctags doesn’t find a parser for it. Enabling Unknown parser with --languages=+Unknown forces ctags to create the extra tags for any source files. The etags mode enables the Unknown parser implicitly. pseudo/p Include pseudo-tags. Enabled by default unless the tag file is written to standard output. See ctags- client-tools(7) about the detail of pseudo-tags.
2.1. ctags 29 Universal Ctags Documentation, Release 0.3.0 qualified/q Include an extra class-qualified or namespace-qualified tag entry for each tag which is a member of a class or a namespace. This may allow easier location of a specific tags when multiple occurrences of a tag name occur in the tag file. Note, however, that this could potentially more than double the size of the tag file. The actual form of the qualified tag depends upon the language from which the tag was derived (using a form that is most natural for how qualified calls are specified in the language). For C++ and Perl, it is in the form class::member; for Eiffel and Java, it is in the form class.member. Note: Using backslash characters as separators forming qualified name in PHP. However, in tags output of Universal Ctags, a backslash character in a name is escaped with a backslash character. See tags(5) about the escaping. The following example demonstrates the qualified extra tag.
class point { double x; };
For the above source file, ctags tags point and x by default. If the qualified extra is enabled from the command line (--extras=+q), then point.x is also tagged even though the string “point.x” is not in the source code.
$ ctags --fields=+K -uo - qualified.java point qualified.java /^class point {$/;" class x qualified.java /^ double x;$/;" field class:point $ ctags --fields=+K --extras=+q -uo - qualified.java point qualified.java /^class point {$/;" class x qualified.java /^ double x;$/;" field class:point point.x qualified.java /^ double x;$/;" field class:point reference/r Include reference tags. See “TAG ENTRIES” about reference tags. The following example demonstrates the reference extra tag.
#include
The roles:system or roles:local fields will be added depending on whether the include file name begins with ‘<’ or not. “#define X” emits a definition tag. On the other hand “#undef X” emits a reference tag.
$ ctags --fields=+EKr -uo - inc.c X inc.c /^#define X$/;" macro file: roles:def
˓→extras:fileScope $ ctags --fields=+EKr --extras=+r -uo - inc.c stdio.h inc.c /^#include
˓→extras:reference utils.h inc.c /^#include "utils.h"/;" header roles:local
˓→extras:reference X inc.c /^#define X$/;" macro file: roles:def
˓→extras:fileScope X inc.c /^#undef X$/;" macro file: roles:undef
˓→extras:fileScope,reference
Language-specific fields and extras
Exuberant Ctags has the concept of fields and extras. They are common between parsers of different languages. Universal Ctags extends this concept by providing language-specific fields and extras.
2.1. ctags 30 Universal Ctags Documentation, Release 0.3.0
2.1.8 HOW TO USE WITH VI
vi(1) will, by default, expect a tag file by the name tags in the current directory. Once the tag file is built, the following commands exercise the tag indexing feature: vi -t tag Start vi and position the cursor at the file and line where tag is defined. :ta tag Find a tag. Ctrl-] Find the tag under the cursor. Ctrl-T Return to previous location before jump to tag (not widely implemented).
2.1.9 HOW TO USE WITH GNU EMACS emacs(1) will, by default, expect a tag file by the name TAGS in the current directory. Once the tag file is built, the following commands exercise the tag indexing feature: M-x visit-tags-table
2.1.10 HOW TO USE WITH NEDIT
NEdit version 5.1 and later can handle the new extended tag file format (see --format). • To make NEdit use the tag file, select “File->Load Tags File”. • To jump to the definition for a tag, highlight the word, then press Ctrl-D. NEdit 5.1 can read multiple tag files from different directories. Setting the X resource nedit.tagFile to the name of a tag file instructs NEdit to automatically load that tag file at startup time.
2.1.11 CAVEATS
Because ctags is neither a preprocessor nor a compiler, use of preprocessor macros can fool ctags into either miss- ing tags or improperly generating inappropriate tags. Although ctags has been designed to handle certain common cases, this is the single biggest cause of reported problems. In particular, the use of preprocessor constructs which alter the textual syntax of C can fool ctags. You can work around many such problems by using the -I option. Note that since ctags generates patterns for locating tags (see the --excmd option), it is entirely possible that the wrong line may be found by your editor if there exists another source line which is identical to the line containing the tag. The following example demonstrates this condition:
int variable;
/* ... */ void foo(variable) int variable; { /* ... */ }
Depending upon which editor you use and where in the code you happen to be, it is possible that the search pattern may locate the local parameter declaration before it finds the actual global variable definition, since the lines (and therefore their search patterns) are identical. This can be avoided by use of the --excmd=n option.
2.1. ctags 31 Universal Ctags Documentation, Release 0.3.0
2.1.12 BUGS
ctags has more options than ls(1). ctags assumes the input file is written in the correct grammar. Otherwise output of ctags is undefined. In other words it has garbage in, garbage out (GIGO) feature. When parsing a C++ member function definition (e.g. className::function), ctags cannot determine whether the scope specifier is a class name or a namespace specifier and always lists it as a class name in the scope portion of the extension fields. Also, if a C++ function is defined outside of the class declaration (the usual case), the access specification (i.e. public, protected, or private) and implementation information (e.g. virtual, pure virtual) contained in the function declaration are not known when the tag is generated for the function definition. It will, however be available for prototypes (e.g. --kinds-c++=+p). No qualified tags are generated for language objects inherited into a class.
2.1.13 ENVIRONMENT VARIABLES
TMPDIR On Unix-like hosts where mkstemp(3) is available, the value of this variable specifies the directory in which to place temporary files. This can be useful if the size of a temporary file becomes too large to fit on the partition holding the default temporary directory defined at compilation time. ctags creates temporary files only if either (1) an emacs-style tag file is being generated, (2) the tag file is being sent to standard output, or (3) the program was compiled to use an internal sort algorithm to sort the tag files instead of the sort(1) utility of the operating system. If the sort(1) utility of the operating system is being used, it will generally observe this variable also. Note that if ctags is setuid, the value of TMPDIR will be ignored.
2.1.14 FILES
tags The default tag file created by ctags. TAGS The default tag file created by etags. $XDG_CONFIG_HOME/ctags/*.ctags, or $HOME/.config/ctags/*.ctags if $XDG_CONFIG_HOME is not defined (on other than MS Windows) $HOME/.ctags.d/*.ctags $HOMEDRIVE$HOMEPATH/ctags.d/*.ctags (on MS Windows only) .ctags.d/*.ctags ctags.d/*.ctags If any of these configuration files exist, each will be expected to contain a set of default options which are read in the order listed when ctags starts, but before any command line options are read. This makes it possible to set up personal or project-level defaults. It is possible to compile ctags to read an additional configuration file before any of those shown above, which will be indicated if the output produced by the --version option lists the custom-conf feature. Options appearing on the command line will override options specified in these files. Only options will be read from these files. Note that the option files are read in line-oriented mode in which spaces are significant (since shell quoting is not possible) but spaces at the beginning of a line are ignored. Each line of the file is read as one command line parameter (as if it were quoted with single quotes). Therefore, use new lines to indicate separate command-line arguments. A line starting with ‘#’ is treated as a comment. *.ctags files in a directory are loaded in alphabetical order.
2.1. ctags 32 Universal Ctags Documentation, Release 0.3.0
2.1.15 SEE ALSO
See ctags-optlib(7) for defining (or extending) a parser in a configuration file. See tags(5) for the format of tag files. See ctags-incompatibilities(7) about known incompatible changes with Exuberant Ctags. See ctags-client-tools(7) if you are interested in writing a tool for processing tags files. See ctags-lang-python(7) about python input specific notes. See readtags(1) about a client tool for binary searching a name in a sorted tags file. The official Universal Ctags web site at: https://ctags.io/ Also ex(1), vi(1), elvis(1), or, better yet, vim(1), the official editor of ctags. For more information on vim(1), see the Vim web site at: https://www.vim.org/
2.1.16 AUTHOR
Universal Ctags project https://ctags.io/ Darren Hiebert
2.1.17 MOTIVATION
“Think ye at all times of rendering some service to every member of the human race.” “All effort and exertion put forth by man from the fullness of his heart is worship, if it is prompted by the highest motives and the will to do service to humanity.” – From the Baha’i Writings
2.1.18 CREDITS
This version of ctags (Universal Ctags) derived from the repository, known as fishman-ctags, started by Reza Jelveh. The fishman-ctags was derived from Exuberant Ctags. Some parsers are taken from tagmanager of the Geany (https://www.geany.org/) project. Exuberant Ctags was originally derived from and inspired by the ctags program by Steve Kirkendall
2.2 tags
Vi tags file format extended in ctags projects Version 2+ Manual group Universal Ctags Manual section 5
2.2. tags 33 Universal Ctags Documentation, Release 0.3.0
2.2.1 DESCRIPTION
The contents of next section is a copy of FORMAT file in Exuberant Ctags source code in its subversion repository at sourceforge.net. Exceptions introduced in Universal Ctags are explained inline with “EXCEPTION” marker.
2.2.2 Proposal for extended Vi tags file format
Version: 0.06 DRAFT Date: 1998 Feb 8 Author: Bram Moolenaar
Introduction
The file format for the “tags” file, as used by Vi and many of its descendants, has limited capabilities. This additional functionality is desired: 1. Static or local tags. The scope of these tags is the file where they are defined. The same tag can appear in several files, without really being a duplicate. 2. Duplicate tags. Allow the same tag to occur more then once. They can be located in a different file and/or have a different command. 3. Support for C++. A tag is not only specified by its name, but also by the context (the class name). 4. Future extension. When even more additional functionality is desired, it must be possible to add this later, without breaking programs that don’t support it.
From proposal to standard
To make this proposal into a standard for tags files, it needs to be supported by most people working on versions of Vi, ctags, etc.. Currently this standard is supported by: Darren Hiebert
2.2. tags 34 Universal Ctags Documentation, Release 0.3.0
Backwards compatibility
A tags file that is generated in the new format should still be usable by Vi. This makes it possible to distribute tags files that are usable by all versions and descendants of Vi. This restricts the format to what Vi can handle. The format is: 1. The tags file is a list of lines, each line in the format:
{tagname}
{tagname} Any identifier, not containing white space.. EXCEPTION: Universal Ctags violates this item of the proposal; tagname may contain spaces. How- ever, tabs are not allowed.
The new lines can be: main main.c /^main(argc, argv)$/;"any additional text DEBUG defines.c 89;"any additional text
Note that the ‘;’ is required to put the cursor in the right line, and then the ‘”’ is recognized as the start of a comment. For Posix compliant Vi versions this will NOT work, since only a line number or a search command is recognized. I hope Posix can be adjusted. Nvi suffers from this.
Security
Vi allows the use of any Ex command in a tags file. This has the potential of a trojan horse security leak. The proposal is to allow only Ex commands that position the cursor in a single file. Other commands, like editing another file, quitting the editor, changing a file or writing a file, are not allowed. It is therefore logical to call the command a tagaddress. Specifically, these two Ex commands are allowed: • A decimal line number:
89
• A search command. It is a regular expression pattern, as used by Vi, enclosed in // or ??:
/^int c;$/ ?main()?
There are two combinations possible:
2.2. tags 35 Universal Ctags Documentation, Release 0.3.0
• Concatenation of the above, with ‘;’ in between. The meaning is that the first line number or search com- mand is used, the cursor is positioned in that line, and then the second search command is used (a line number would not be useful). This can be done multiple times. This is useful when the information in a single line is not unique, and the search needs to start in a specified line.
/struct xyz {/;/int count;/ 389;/struct foo/;/char *s;/
• A trailing comment can be added, starting with ‘;”’ (two characters: semi-colon and double-quote). This is used below.
89;" foo bar
This might be extended in the future. What is currently missing is a way to position the cursor in a certain column.
Goals
Now the usage of the comment text has to be defined. The following is aimed at: 1. Keep the text short, because: • The line length that Vi can handle is limited to 512 characters. • Tags files can contain thousands of tags. I have seen tags files of several Mbytes. • More text makes searching slower. 2. Keep the text readable, because: • It is often necessary to check the output of a new ctags program. • Be able to edit the file by hand. • Make it easier to write a program to produce or parse the file. 3. Don’t use special characters, because: • It should be possible to treat a tags file like any normal text file.
Proposal
Use a comment after the {tagaddress} field. The format would be:
{tagname}
{tagname} Any identifier, not containing white space.. EXCEPTION: Universal Ctags violates this item of the proposal; name may contain spaces. However, tabs are not allowed. Conversion, for some characters including
2.2. tags 36 Universal Ctags Documentation, Release 0.3.0
• The name consist only out of alphabetical characters. Upper and lower case are allowed. Lower case is recommended. Case matters (“kind:” and “Kind: are different tagfields). EXCEPTION: Universal Ctags allows users to use a numerical character in the name other than its initial letter. • The value may be empty. It cannot contain a
2.2. tags 37 Universal Ctags Documentation, Release 0.3.0
FIELD-NAME DESCRIPTION arity Number of arguments for a function tag. class Name of the class for which this tag is a member or method. enum Name of the enumeration in which this tag is an enu- merator. file Static (local) tag, with a scope of the specified file. When the value is empty, {tagfile} is used. function Function in which this tag is defined. Useful for local variables (and functions). When functions nest (e.g., in Pascal), the function names are concatenated, sep- arated with ‘/’, so it looks like a path. kind Kind of tag. The value depends on the language. For C and C++ these kinds are recommended: c class name d define (from #define XXX) e enumerator f function or method name F file name g enumeration name m member (of structure or class data) p function prototype s structure name t typedef u union name v variable When this field is omitted, the kind of tag is unde- fined. struct Name of the struct in which this tag is a member. union Name of the union in which this tag is a member.
Note that these are mostly for C and C++. When tags programs are written for other languages, this list should be extended to include the used field names. This will help users to be independent of the tags program used. Examples: asdf sub.cc /^asdf()$/;" new_field:some\svalue file: foo_t sub.h /^typedef foo_t$/;" kind:t func3 sub.p /^func3()$/;" function:/func1/func2 file: getflag sub.c /^getflag(arg)$/;" kind:f file: inc sub.cc /^inc()$/;" file: class:PipeBuf
The name of the “kind:” field can be omitted. This is to reduce the size of the tags file by about 15%. A program reading the tags file can recognize the “kind:” field by the missing ‘:’. Examples: foo_t sub.h /^typedef foo_t$/;" t getflag sub.c /^getflag(arg)$/;" f file:
Additional remarks: • When a tagfield appears twice in a tag line, only the last one is used. Note about line separators: Vi traditionally runs on Unix systems, where the line separator is a single linefeed character
2.2. tags 38 Universal Ctags Documentation, Release 0.3.0
Summary:
line separator generated on accepted on
The characters
:%s/^\([^^I] *\)[^I] *\([^^I] *\)[^I] */\1^I\2^I/
(replace ^I with a real
!_TAG_FILE_FORMAT {version-number} /optional comment/ !_TAG_FILE_SORTED {0|1} /0=unsorted, 1=sorted/
The {version-number} used in the tag file format line reserves the value of “1” for tag files complying with the original UNIX vi/ctags format, and reserves the value “2” for tag files complying with this proposal. This value may be used to determine if the extended features described in this proposal are present. Tag lines providing information about the program used to generate the tag file, and provided solely for documen- tation purposes:
!_TAG_PROGRAM_AUTHOR {author-name} /{email-address}/ !_TAG_PROGRAM_NAME {program-name} /optional comment/ !_TAG_PROGRAM_URL {URL} /optional comment/ !_TAG_PROGRAM_VERSION {version-id} /optional comment/
EXCEPTION: Universal Ctags introduces more kinds of pseudo-tags. See ctags-client-tools(7) about them.
2.2. tags 39 Universal Ctags Documentation, Release 0.3.0
2.2.3 Exceptions in Universal Ctags
Universal Ctags supports this proposal with some exceptions.
Exceptions
1. {tagname} in tags file generated by Universal Ctags may contain spaces and several escape sequences. Parsers for documents like Tex and reStructuredText, or liberal languages such as JavaScript need these exceptions. See {tagname} of Proposal section for more detail about the conversion. 2. “name” part of {tagfield} in a tag generated by Universal Ctags may contain numeric characters, but the first character of the “name” must be alphabetic.
Compatible output and weakness
Default behavior (--output-format=u-ctags option) has the exceptions. In other hand, with --output-format=e-ctags option ctags has no exception; Universal Ctags command may use the same file format as Exuberant Ctags. However, --output-format=e-ctags throws away a tag entry which name includes a space or a tab character. TAG_OUTPUT_MODE pseudo-tag tells which format is used when ctags generating tags file.
2.2.4 SEE ALSO ctags(1), ctags-client-tools(7), ctags-incompatibilities(7), readtags(1)
2.3 ctags-optlib
Universal Ctags parser definition language Version 5.9.0 Manual group Universal Ctags Manual section 7
2.3.1 SYNOPSIS ctags [options] [file(s)] etags [options] [file(s)]
2.3.2 DESCRIPTION
Exuberant Ctags, the ancestor of Universal Ctags, has provided the way to define a new parser from command line. Universal Ctags extends and refines this feature. optlib parser is the name for such parser in Universal Ctags. “opt” intends a parser is defined with combination of command line options. “lib” intends an optlib parser can be more than ad-hoc personal configuration. This man page is for people who want to define an optlib parser. The readers should read ctags(1) of Universal Ctags first. Following options are for defining (or customizing) a parser: • --langdef=
2.3. ctags-optlib 40 Universal Ctags Documentation, Release 0.3.0
• --kinddef-
Storing a parser definition to a file
Though it is possible to define a parser from command line, you don’t want to type the same command line each time when you need the parser. You can store options for defining a parser into a file. ctags loads files (preload files) listed in “FILES” section of ctags(1) at program starting up. You can put your parser definition needed usually to the files. --options=
Overview for defining a parser
1. Design the parser You need know both the target language and the ctags’ concepts (definition, reference, kind, role, field, extra). About the concepts, ctags(1) of Universal Ctags may help you. 2. Give a name to the parser Use --langdef=
2.3. ctags-optlib 41 Universal Ctags Documentation, Release 0.3.0
of specifying --regex-
2.3.3 OPTIONS
--langdef=
2.3. ctags-optlib 42 Universal Ctags Documentation, Release 0.3.0
--regex-
FLAGS FOR --regex-
You can specify more than one flag,
2.3. ctags-optlib 43 Universal Ctags Documentation, Release 0.3.0
Exuberant Ctags uses a
2.3.4 EXAMPLES
2.3. ctags-optlib 44 Universal Ctags Documentation, Release 0.3.0
Perl Pod
This is the definition (pod.ctags) used in ctags for parsing Pod (https://perldoc.perl.org/perlpod.html) file.
--langdef=pod --map-pod=+.pod
--kinddef-pod=c,chapter,chapters --kinddef-pod=s,section,sections --kinddef-pod=S,subsection,subsections --kinddef-pod=t,subsubsection,subsubsections
--regex-pod=/^=head1[ \t]+(.+)/\1/c/ --regex-pod=/^=head2[ \t]+(.+)/\1/s/ --regex-pod=/^=head3[ \t]+(.+)/\1/S/ --regex-pod=/^=head4[ \t]+(.+)/\1/t/
Using scope regex flags
Let’s think about writing a parser for a very small subset of the Ruby language. input source file (input.srb): class Example def methodA puts"in class_method" end def methodB puts"in class_method" end end
The parser for the input should capture Example with class kind, methodA, and methodB with method kind. methodA and methodB should have Example as their scope. end: fields of each tag should have proper values. optlib file (sub-ruby.ctags):
--langdef=subRuby --map-subRuby=.srb --kinddef-subRuby=c,class,classes --kinddef-subRuby=m,method,methods --regex-subRuby=/^class[ \t]+([a-zA-Z][a-zA-Z0-9]+)/\1/c/{scope=push} --regex-subRuby=/^end///{scope=pop}{placeholder} --regex-subRuby=/^[ \t]+def[ \t]+([a-zA-Z][a-zA-Z0-9_]+)/\1/m/{scope=push} --regex-subRuby=/^[ \t]+end///{scope=pop}{placeholder} command line and output:
$ ctags --quiet --fields=+eK \ --options=./sub-ruby.ctags -o - input.srb Example input.srb /^class Example$/;" class end:8 methodA input.srb /^ def methodA$/;" method class:Example end:4 methodB input.srb /^ def methodB$/;" method class:Example end:7
2.3.5 SEE ALSO
The official Universal Ctags web site at: https://ctags.io/
2.3. ctags-optlib 45 Universal Ctags Documentation, Release 0.3.0
ctags(1), tags(5), regex(3), regex(7), egrep(1)
2.3.6 AUTHOR
Universal Ctags project https://ctags.io/ (This man page partially derived from ctags(1) of Executable-ctags) Darren Hiebert
2.4 ctags-client-tools
Hints for developing a tool using ctags command and tags output Version 5.9.0 Manual group Universal Ctags Manual section 7
2.4.1 SYNOPSIS ctags [options] [file(s)] etags [options] [file(s)]
2.4.2 DESCRIPTION
Client tool means a tool running the ctags command and/or reading a tags file generated by ctags command. This man page gathers hints for people who develop client tools.
2.4.3 PSEUDO-TAGS
Pseudo-tags, stored in a tag file, indicate how ctags generated the tags file: whether the tags file is sorted or not, which version of tags file format is used, the name of tags generator, and so on. The opposite term for pseudo-tags is regular-tags. A regular-tag is for a language object in an input file. A pseudo-tag is for the tags file itself. Client tools may use pseudo-tags as reference for processing regular-tags. A pseudo-tag is stored in a tags file in the same format as regular-tags as described in tags(5), except that pseudo- tag names are prefixed with “!_”. For the general information about pseudo-tags, see “TAG FILE INFORMA- TION” in tags(5). An example of a pseudo tag:
!_TAG_PROGRAM_NAME Universal Ctags /Derived from Exuberant Ctags/
The value, “2”, associated with the pseudo tag “TAG_PROGRAM_NAME”, is used in the field for input file. The description, “Derived from Exuberant Ctags”, is used in the field for pattern. Universal Ctags extends the naming scheme of the classical pseudo-tags available in Exuberant Ctags for emitting language specific information as pseudo tags:
!_{pseudo-tag-name}!{language-name} {associated-value} /{description}/
The language-name is appended to the pseudo-tag name with a separator, “!”. An example of pseudo tag with a language suffix:
!_TAG_KIND_DESCRIPTION!C f,function /function definitions/
2.4. ctags-client-tools 46 Universal Ctags Documentation, Release 0.3.0
This pseudo-tag says “the function kind of C language is enabled when generating this tags file.” --pseudo-tags is the option for enabling/disabling individual pseudo-tags. When enabling/disabling a pseudo tag with the option, specify the tag name only “TAG_KIND_DESCRIPTION”, without the prefix (“!_”) or the suffix (“!C”).
Options for Pseudo-tags
--extras=+p (or --extras=+{pseudo}) Forces writing pseudo-tags. ctags emits pseudo-tags by default when writing tags to a regular file (e.g. “tags’.) However, when specify- ing -o - or -f - for writing tags to standard output, ctags doesn’t emit pseudo-tags. --extras=+p or --extras=+{pseudo} will force pseudo-tags to be written. --list-pseudo-tags Lists available types of pseudo-tags and shows whether they are enabled or disabled. Running ctags with --list-pseudo-tags option lists available pseudo-tags. Some of pseudo-tags newly introduced in Universal Ctags project are disabled by default. Use --pseudo-tags=... to enable them. --pseudo-tags=[+|-]names|* Specifies a list of pseudo-tag types to include in the output. The parameters are a set of pseudo tag names. Valid pseudo tag names can be listed with --list-pseudo-tags. Surround each name in the set with braces, like “{TAG_PROGRAM_AUTHOR}”. You don’t have to include the “!_” pseudo tag prefix when spec- ifying a name in the option argument for --pseudo-tags= option. pseudo-tags don’t have a notation using one-letter flags. If a name is preceded by either the ‘+’ or ‘-’ characters, that tags’s effect has been added or removed. Otherwise the names replace any current settings. All entries are included if ‘*’ is given. --fields=+E (or --fields=+{extras}) Attach “extras:pseudo” field to pseudo-tags. An example of pseudo tags with the field:
!_TAG_PROGRAM_NAME Universal Ctags /Derived from Exuberant Ctags/
˓→extras:pseudo
If the name of a normal tag in a tag file starts with “!_”, a client tool cannot distinguish whether the tag is a regular-tag or pseudo-tag. The fields attached with this option help the tool distinguish them.
List of notable pseudo-tags
Running ctags with --list-pseudo-tags option lists available types of pseudo-tags with short descriptions. This subsection shows hints for using notable ones. TAG_EXTRA_DESCRIPTION (new in Universal Ctags) Indicates the names and descriptions of enabled ex- tras:
!_TAG_EXTRA_DESCRIPTION {extra-name} /description/ !_TAG_EXTRA_DESCRIPTION!{language-name} {extra-name} /description/
If your tool relies on some extra tags (extras), refer to the pseudo-tags of this type. A tool can reject the tags file that doesn’t include expected extras, and raise an error in an early stage of processing. An example of the pseudo-tags:
$ ctags --extras=+p --pseudo-tags='{TAG_EXTRA_DESCRIPTION}' -o - input.c !_TAG_EXTRA_DESCRIPTION anonymous /Include tags for non-named
˓→objects like lambda/ !_TAG_EXTRA_DESCRIPTION fileScope /Include tags of file scope/ !_TAG_EXTRA_DESCRIPTION pseudo /Include pseudo tags/ (continues on next page)
2.4. ctags-client-tools 47 Universal Ctags Documentation, Release 0.3.0
(continued from previous page) !_TAG_EXTRA_DESCRIPTION subparser /Include tags generated by
˓→subparsers/ ...
A client tool can know “{anonymous}”, “{fileScope}”, “{pseudo}”, and “{subparser}” extras are enabled from the output. TAG_FIELD_DESCRIPTION (new in Universal Ctags) Indicates the names and descriptions of enabled fields:
!_TAG_FIELD_DESCRIPTION {field-name} /description/ !_TAG_FIELD_DESCRIPTION!{language-name} {field-name} /description/
If your tool relies on some fields, refer to the pseudo-tags of this type. A tool can reject a tags file that doesn’t include expected fields, and raise an error in an early stage of processing. An example of the pseudo-tags:
$ ctags --fields-C=+'{macrodef}' --extras=+p --pseudo-tags='{TAG_FIELD_
˓→DESCRIPTION}' -o - input.c !_TAG_FIELD_DESCRIPTION file /File-restricted scoping/ !_TAG_FIELD_DESCRIPTION input /input file/ !_TAG_FIELD_DESCRIPTION name /tag name/ !_TAG_FIELD_DESCRIPTION pattern /pattern/ !_TAG_FIELD_DESCRIPTION typeref /Type and name of a variable or typedef/ !_TAG_FIELD_DESCRIPTION!C macrodef /macro definition/ ...
A client tool can know “{file}”, “{input}”, “{name}”, “{pattern}”, and “{typeref}” fields are enabled from the output. The fields are common in languages. In addition to the common fields, the tool can known “{macrodef}” field of C language is also enabled. TAG_FILE_ENCODING (new in Universal Ctags) TBW TAG_FILE_FORMAT See also tags(5). TAG_FILE_SORTED See also tags(5). TAG_KIND_DESCRIPTION (new in Universal Ctags) Indicates the names and descriptions of enabled kinds:
!_TAG_KIND_DESCRIPTION!{language-name} {kind-letter},{kind-name} /
˓→description/
If your tool relies on some kinds, refer to the pseudo-tags of this type. A tool can reject the tags file that doesn’t include expected kinds, and raise an error in an early stage of processing. Kinds are language specific, so a language name is always appended to the tag name as suffix. An example of the pseudo-tags:
$ ctags --extras=+p --kinds-C=vfm --pseudo-tags='{TAG_KIND_DESCRIPTION}' -o -
˓→input.c !_TAG_KIND_DESCRIPTION!C f,function /function definitions/ !_TAG_KIND_DESCRIPTION!C m,member /struct, and union members/ !_TAG_KIND_DESCRIPTION!C v,variable /variable definitions/ ...
A client tool can know “{function}”, “{member}”, and “{variable}” kinds of C language are enabled from the output. TAG_KIND_SEPARATOR (new in Universal Ctags) TBW TAG_OUTPUT_EXCMD (new in Universal Ctags) Indicates the specified type of EX command with --excmd option. TAG_OUTPUT_FILESEP (new in Universal Ctags) TBW
2.4. ctags-client-tools 48 Universal Ctags Documentation, Release 0.3.0
TAG_OUTPUT_MODE (new in Universal Ctags) TBW TAG_PATTERN_LENGTH_LIMIT (new in Universal Ctags) TBW TAG_PROC_CWD (new in Universal Ctags) Indicates the working directory of ctags during processing. This pseudo-tag helps a client tool solve the absolute paths for the input files for tag entries even when they are tagged with relative paths. An example of the pseudo-tags:
$ cat tags !_TAG_PROC_CWD /tmp/ // main input.c /^int main (void) { return 0; }$/;" f
˓→typeref:typename:int ...
From the regular tag for “main”, the client tool can know the “main” is at “input.c”. However, it is a relative path. So if the directory where ctags run and the directory where the client tool runs are different, the client tool cannot find “input.c” from the file system. In that case, TAG_PROC_CWD gives the tool a hint; “input.c” may be at “/tmp”. TAG_PROGRAM_NAME TBW TAG_ROLE_DESCRIPTION (new in Universal Ctags) Indicates the names and descriptions of enabled roles:
!_TAG_ROLE_DESCRIPTION!{language-name}!{kind-name} {role-name} /
˓→description/
If your tool relies on some roles, refer to the pseudo-tags of this type. Note that a role owned by a disabled kind is not listed even if the role itself is enabled.
2.4.4 REDUNDANT-KINDS
TBW
2.4.5 MULTIPLE-LANGUAGES FOR AN INPUT FILE
Universal ctags can run multiple parsers. That means a parser, which supports multiple parsers, may output tags for different languages. language/l field can be used to show the language for each tag.
$ cat /tmp/foo.html
title
$ ./ctags -o - --extras=+g /tmp/foo.html title /tmp/foo.html /^title<\/h1>$/;" h x /tmp/foo.html /var x = 1/;" v $ ./ctags -o - --extras=+g --fields=+l /tmp/foo.html title /tmp/foo.html /^ title<\/h1>$/;" h language:HTML x /tmp/foo.html /var x = 1/;" v language:JavaScript
2.4.6 UTILIZING READTAGS
See readtags(1) to know how to use readtags. This section is for discussing some notable topics for client tools.
2.4. ctags-client-tools 49 Universal Ctags Documentation, Release 0.3.0
Build Filter/Sorter Expressions
Certain escape sequences in expressions are recognized by readtags. For example, when searching for a tag that matches a\?b, if using a filter expression like '(eq? $name "a\?b")', since \? is translated into a single ? by readtags, it actually searches for a?b. Another problem is if a single quote appear in filter expressions (which is also wrapped by single quotes), it terminates the expression, producing broken expressions, and may even cause unintended shell injection. Single quotes can be escaped using '"'"'. So, client tools need to: • Replace \ by \\ • Replace ' by '"'"' inside the expressions. If the expression also contains strings, " in the strings needs to be replaced by \". Client tools written in Lisp could build the expression using lists. prin1 (in Common Lisp style Lisps) and write (in Scheme style Lisps) can translate the list into a string that can be directly used. For example, in EmacsLisp:
(let ((name "hi")) (prin1 `(eq? $name ,name))) => "(eq\\? $name "hi")"
The “?” is escaped, and readtags can handle it. Scheme style Lisps should do proper escaping so the expression readtags gets is just the expression passed into write. Common Lisp style Lisps may produce unrecognized escape sequences by readtags, like \#. Readtags provides some aliases for these Lisps: • Use true for #t. • Use false for #f. • Use nil or () for (). • Use (string->regexp "PATTERN") for #/PATTERN/. Use (string->regexp "PATTERN" :case-fold true) for #/PATTERN/i. Notice that string->regexp doesn’t require escaping “/” in the pattern. Notice that even when the client tool uses this method, ' still needs to be replaced by '"'"' to prevent broken expressions and shell injection. Another thing to notice is that missing fields are represented by #f, and applying string operators to them will produce an error. You should always check if a field is missing before applying string operators. See the “Filter- ing” section in readtags(1) to know how to do this. Run “readtags -H filter” to see which operators take string arguments.
Parse Readtags Output
In the output of readtags, tabs can appear in all field values (e.g., the tag name itself could contain tabs), which makes it hard to split the line into fields. Client tools should use the -E option, which keeps the escape sequences in the tags file, so the only field that could contain tabs is the pattern field. The pattern field could: • Use a line number. It will look like number;" (e.g. 10;"). • Use a search pattern. It will look like /pattern/;" or ?pattern?;". Notice that the search pattern could contain tabs. • Combine these two, like number;/pattern/;" or number;?pattern?;". These are true for tags files using extended format, which is the default one. The legacy format (i.e. --format=1) doesn’t include the semicolons. It’s old and barely used, so we won’t discuss it here. Client tools could split the line using the following steps:
2.4. ctags-client-tools 50 Universal Ctags Documentation, Release 0.3.0
• Find the first 2 tabs in the line, so we get the name and input field. • From the 2nd tab: – If a / follows, then the pattern delimiter is /. – If a ? follows, then the pattern delimiter is ?. – If a number follows, then:
* If a ;/ follows the number, then the delimiter is /. * If a ;? follows the number, then the delimiter is ?. * If a ;" follows the number, then the field uses only line number, and there’s no pattern delimiter (since there’s no regex pattern). In this case the pattern field ends at the 3rd tab. • After the opening delimiter, find the next unescaped pattern delimiter, and that’s the closing delimiter. It will be followed by ;" and then a tab. That’s the end of the pattern field. By “unescaped pattern delimiter”, we mean there’s an even number (including 0) of backslashes before it. • From here, split the rest of the line into fields by tabs. Then, the escape sequences in fields other than the pattern field should be translated. See “Proposal” in tags(5) to know about all the escape sequences.
Make Use of the Pattern Field
The pattern field specifies how to find a tag in its source file. The code generating this field seems to have a long history, so there are some pitfalls and it’s a bit hard to handle. A client tool could simply require the line: field and jump to the line it specifies, to avoid using the pattern field. But anyway, we’ll discuss how to make the best use of it here. You should take the words here merely as suggestions, and not standards. A client tool could definitely develop better (or simpler) ways to use the pattern field. From the last section, we know the pattern field could contain a line number and a search pattern. When it only contains the line number, handling it is easy: you simply go to that line. The search pattern resembles an EX command, but as we’ll see later, it’s actually not a valid one, so some manual work are required to process it. The search pattern could look like /pat/, called “forward search pattern”, or ?pat?, called “backward search pattern”. Using a search pattern means even if the source file is updated, as long as the part containing the tag doesn’t change, we could still locate the tag correctly by searching. When the pattern field only contains the search pattern, you just search for it. The search direction (for- ward/backward) doesn’t matter, as it’s decided solely by whether the -B option is enabled, and not the actual context. You could always start the search from say the beginning of the file. When both the search pattern and the line number are presented, you could make good use of the line number, by going to the line first, then searching for the nearest occurrence of the pattern. A way to do this is to search both forward and backward for the pattern, and when there is a occurrence on both sides, go to the nearer one. What’s good about this is when there are multiple identical lines in the source file (e.g. the COMMON block in Fortran), this could help us find the correct one, even after the source file is updated and the tag position is shifted by a few lines. Now let’s discuss how to search for the pattern. After you trim the / or ? around it, the pattern resembles a regex pattern. It should be a regex pattern, as required by being a valid EX command, but it’s actually not, as you’ll see below. It could begin with a ^, which means the pattern starts from the beginning of a line. It could also end with an unescaped $ which means the pattern ends at the end of a line. Let’s keep this information, and trim them too. Now the remaining part is the actual string containing the tag. Some characters are escaped: • \.
2.4. ctags-client-tools 51 Universal Ctags Documentation, Release 0.3.0
• $, but only at the end of the string. • /, but only in forward search patterns. • ?, but only in backward search patterns. You need to unescape these to get the literal string. Now you could convert this literal string to a regexp that matches it (by escaping, like re.escape in Python or regexp-quote in Elisp), and assemble it with ^ or $ if the pattern originally has it, and finally search for the tag using this regexp.
Remark: About a Previous Format of the Pattern Field
In some earlier versions of Universal Ctags, the line number in the pattern field is the actual line number minus one, for forward search patterns; or plus one, for backward search patterns. The idea is to resemble an EX command: you go to the line, then search forward/backward for the pattern, and you can always find the correct one. But this denies the purpose of using a search pattern: to tolerate file updates. For example, the tag is at line 50, according to this scheme, the pattern field should be:
49;/pat/;"
Then let’s assume that some code above are removed, and the tag is now at line 45. Now you can’t find it if you search forward from line 49. Due to this reason, Universal Ctags turns to use the actual line number. A client tool could distinguish them by the TAG_OUTPUT_EXCMD pseudo tag, it’s “combine” for the old scheme, and “combineV2” for the present scheme. But probably there’s no need to treat them differently, since “search for the nearest occurrence from the line” gives good result on both schemes.
2.4.7 JSON OUTPUT
Universal Ctags supports JSON (strictly speaking JSON Lines) output format if the ctags executable is built with libjansson. JSON output goes to standard output by default.
Format
Each JSON line represents a tag.
$ ctags --extras=+p --output-format=json --fields=-s input.py {"_type": "ptag", "name": "JSON_OUTPUT_VERSION", "path": "0.0", "pattern": "in
˓→development"} {"_type": "ptag", "name": "TAG_FILE_SORTED", "path": "1", "pattern": "0=unsorted,
˓→1=sorted, 2=foldcase"} ... {"_type": "tag", "name": "Klass", "path": "/tmp/input.py", "pattern": "/^class
˓→Klass:$/", "language": "Python", "kind": "class"} {"_type": "tag", "name": "method", "path": "/tmp/input.py", "pattern": "/^ def
˓→method(self):$/", "language": "Python", "kind": "member", "scope": "Klass",
˓→"scopeKind": "class"} ...
A key not starting with _ is mapped to a field of ctags. “--output-format=json --list-fields” options list the fields. A key starting with _ represents meta information of the JSON line. Currently only _type key is used. If the value for the key is tag, the JSON line represents a normal tag. If the value is ptag, the line represents a pseudo-tag. The output format can be changed in the future. JSON_OUTPUT_VERSION pseudo-tag provides a change client- tools to handle the changes. Current version is “0.0”. A client-tool can extract the version with path key from the pseudo-tag.
2.4. ctags-client-tools 52 Universal Ctags Documentation, Release 0.3.0
The JSON output format is newly designed and has no limitation found in the default tags file format. • The values for kind key are represented in long-name flags. No one-letter is here. • Scope names and scope kinds have distinguished keys: scope and scopeKind. They are combined in the default tags file format.
Data type used in a field
Values for the most of all keys are represented in JSON string type. However, some of them are represented in string, integer, and/or boolean type. “--output-format=json --list-fields” options show What kind of data type used in a field of JSON.
$ ctags --output-format=json --list-fields #LETTER NAME ENABLED LANGUAGE JSTYPE FIXED DESCRIPTION F input yes NONE s-- no input file ... P pattern yes NONE s-b no pattern ... f file yes NONE --b no File-restricted
˓→scoping ... e end no NONE -i- no end lines of various
˓→items ...
JSTYPE column shows the data types. ‘s’ string ‘i’ integer ‘b’ boolean (true or false) For an example, the value for pattern field of ctags takes a string or a boolean value.
2.4.8 SEE ALSO ctags(1), ctags-lang-python(7), ctags-incompatibilities(7), tags(5), readtags(1)
2.5 ctags-incompatibilities
Incompatibilities between Universal Ctags and Exuberant Ctags Version 5.9.0 Manual group Universal Ctags Manual section 7
2.5.1 SYNOPSIS ctags [options] [file(s)] etags [options] [file(s)]
2.5.2 DESCRIPTION
This page describes major incompatible changes introduced to Universal Ctags forked from Exuberant Ctags.
2.5. ctags-incompatibilities 53 Universal Ctags Documentation, Release 0.3.0
Option files loading at starting up time (preload files)
Universal Ctags doesn’t load ~/.ctags at starting up time. File paths for preload files are changed. See “FILES” section of ctags(1).
Environment variables for arranging command lines
Universal Ctags doesn’t read CTAGS and/or ETAGS environment variables.
Incompatibilities in command line interface
Ordering in a command line
The command line format of Universal Ctags is “ctags [options] [source_file(s)]” following the standard POSIX convention. Exuberant Ctags accepts a option following a source file.
$ ctags -o - foo.c --list-kinds=Sh f functions
Universal Ctags warns and ignores the option --list-kinds=Sh as follows.
$ ctags -o - foo.c --list-kinds=Sh ctags: Warning: cannot open input file "--list-kinds=Sh" : No such file or
˓→directory a foo.c /^void a () {}$/;" f typeref:typename:void b foo.c /^void b () {}$/;" f typeref:typename:void
The order of application of patterns and extensions in --langmap
When applying mappings for a name of given source file, Exuberant Ctags tests file name patterns AFTER file extensions (e-map-order). Universal Ctags does this differently; it tests file name patterns BEFORE file extensions (u-map-order). This incompatible change is introduced to deal with the following situation: • build.xml as a source file, • The Ant parser declares it handles a file name pattern build.xml, and • The XML parser declares it handles a file extension .xml. Which parser should be used for parsing build.xml? The assumption of Universal Ctags is the user may want to use the Ant parser; the file name pattern it declares is more specific than the file extension that the XML parser declares. However, e-map-order chooses the XML parser. So Universal Ctags uses the u-map-order even though it introduces an incompatibility. --list-map-extensions=
Remove --file-tags and --file-scope options
Even in Exuberant Ctags, --file-tags is not documented in its man page. Instead of specifying --file-tags or --file-tags=yes, use --extras=+f or --extras=+{inputFile}. Instead of specifying --file-tags=no, use --extras=-f or --extras=-{inputFile}. Universal Ctags introduces F/fileScope extra as the replacement for --file-scope option.
2.5. ctags-incompatibilities 54 Universal Ctags Documentation, Release 0.3.0
Instead of specifying --file-tags or --file-tags=yes, use --extras=+F or --extras=+{fileScope}. Instead of specifying --file-tags=no, use --extras=-F or --extras=-{fileScope}.
Incompatibilities in language and kind definitions
Language name defined with --langdef=name option
The characters you can use are more restricted than Exuberant Ctags. For more details, see the description of --langdef=name in ctags-optlib(7).
Obsoleting --
Some options have
Disallowing to define a kind with file as name
The kind name file is reserved. Using it as part of kind spec in --regex-
Disallowing to define a kind with ‘F’ as letter
The kind letter ‘F’ is reserved. Using it as part of a kind spec in --regex-
Disallowing to use other than alphabetical character as kind letter
Exuberant Ctags accepts a character other than alphabetical character as kind letter in --regex-
Acceptable characters as parts of a kind name
Exuberant Ctags accepts any character as a part of a kind name defined with --regex-
--regex-Foo=/abstract +class +([a-z]+)/\1/a,abstract class/i
Universal Ctags rejects this because the kind name, abstract class, includes a whitespace character. This requirement is for making the output of Universal Ctags follow the tags file format.
2.5. ctags-incompatibilities 55 Universal Ctags Documentation, Release 0.3.0
A combination of a kind letter and a kind name
In Universal Ctags, the combination of a kind letter and a kind name must be unique in a language. You cannot define more than one kind reusing a kind letter with different kind names. You cannot define more than one kind reusing a kind name with different kind letters. An example:
--regex-Foo=/abstract +class +([a-z]+)/\1/a,abstractClass/i --regex-Foo=/attribute +([a-z]+)/\1/a,attribute/i
Universal Ctags rejects this because the kind letter, ‘a’, used twice for defining a kind abstractClass and attribute.
Incompatibilities in tags file format
Using numerical character in the name part of tag tagfield
The version 2 tags file format, the default output format of Exuberant Ctags, accepts only alphabetical characters in the name part of tag tagfield. Universal Ctags introduces an exception to this specification; it may use numerical characters in addition to alpha- betical characters as the letters other than initial letter of the name part. The kinds heading1, heading2, and heading3 in the HTML parser are the examples.
Truncating the pattern for long input lines
To prevent generating overly large tags files, a pattern field is truncated, by default, when its size exceeds 96 bytes. A different limit can be specified with --pattern-length-limit=N. Specifying 0 as N results no truncation as Exuberant Ctags does not.
Kind letters and names
A kind letter ‘F’ and a kind name file are reserved in the main part. A parser cannot have a kind conflicting with these reserved ones. Some incompatible changes are introduced to follow the above rule. • Cobol’s file kind is renamed to fileDesc because the kind name file is reserved. • Ruby’s ‘F’ (singletonMethod) is changed to ‘S’. • SQL’s ‘F’ (field) is changed to ‘E’.
2.5.3 SEE ALSO ctags(1), ctags-optlib(7), and tags(5).
2.6 ctags-faq
Universal Ctags FAQ Version 5.9.0 Manual group Universal Ctags Manual section 7
2.6. ctags-faq 56 Universal Ctags Documentation, Release 0.3.0
This is the Universal Ctags FAQ (Frequently-Asked Questions). It is based on Exuberant Ctags FAQ
Contents
• ctags-faq – DESCRIPTION
* What is the difference between Universal Ctags and Exuberant Ctags? * How can I avoid having to specify my favorite option every time? * What are these strange bits of text beginning with ;" which follow many of the lines in the tag file?
* Why can’t I jump to class::member? * Why do I end up on the wrong line when I jump to a tag? * How do I jump to the tag I want instead of the wrong one by the same name? * How can I locate all references to a specific function or variable? * Why does appending tags to a tag file tag so long? * How should I set up tag files for a multi-level directory hierarchy? * Does Universal Ctags support Unicode file names? * Why does zsh cause “zsh: no matches found” error? – SEE ALSO – AUTHOR
2.6.1 DESCRIPTION
What is the difference between Universal Ctags and Exuberant Ctags?
Universal Ctags is an unofficial fork of Exuberant Ctags. The differences are summarized in ctags- incompatibilities(7) man page. The most notable one is that Universal Ctags doesn’t read ~/.ctags file. Instead, it reads *.ctags under ~/.ctags.d directory.
How can I avoid having to specify my favorite option every time?
Either by setting the environment variable CTAGS to your custom options, or putting them into a ~/.ctags.d/ anyname.ctags file in your home directory.
What are these strange bits of text beginning with ;" which follow many of the lines in the tag file?
These are extension flags. They are added in order to provide extra information about the tag that may be utilized by the editor in order to more intelligently handle tags. They are appended to the EX command part of the tag line in a manner that provides backwards compatibility with existing implementations of the Vi editor. The semicolon is an EX command separator and the double quote begins an EX comment. Thus, the extension flags appear as an EX comment and should be ignored by the editor when it processes the EX command. Some non-vi editors, however, implement only the bare minimum of EX commands in order to process the search command or line number in the third field of the tag file. If you encounter this problem, use the option --format=1 to generate a tag file without these extensions (remember that you can set the CTAGS environment
2.6. ctags-faq 57 Universal Ctags Documentation, Release 0.3.0
variable to any default arguments you wish to supply). Then ask the supplier of your editor to implement handling of this feature of EX commands.
Why can’t I jump to class::member?
Because, by default, ctags only generates tags for the separate identifiers found in the source files. If you specify the --extra=+q option, then ctags will also generate a second, class-qualified tag for each class member (data and function/method) in the form class::member for C++, and in the form class.method for Eiffel and Java.
Why do I end up on the wrong line when I jump to a tag?
By default, ctags encodes the line number in the file where macro (#define) tags are found. This was done to remain compatible with the original UNIX version of ctags. If you change the file containing the tag without rebuilding the tag file, the location of tag in the tag file may no longer match the current location. In order to avoid this problem, you can specify the option --excmd=p, which causes ctags to use a search pattern to locate macro tags. I have never uncovered the reason why the original UNIX ctags used line numbers exclusively for macro tags, but have so far resisted changing the default behavior of Exuberant (and Universal) Ctags to behave differently.
How do I jump to the tag I want instead of the wrong one by the same name?
A tag file is simple a list of tag names and where to find them. If there are duplicate entries, you often end up going to the wrong one because the tag file is sorted and your editor locates the first one in the tag file. Standard Vi provides no facilities to alter this behavior. However, Vim has some nice features to minimize this problem, primarily by examining all matches and choosing the best one under the circumstances. Vim also pro- vides commands which allow for selection of the desired matching tag.
How can I locate all references to a specific function or variable?
There are several packages already available which provide this capability. Namely, these are: GLOBAL source code tag system, GNU id-utils, cscope, and cflow. As of this writing, they can be found in the following locations: • GLOBAL: http://www.gnu.org/software/global • id-utils: http://www.gnu.org/software/idutils/idutils.html • cscope: http://cscope.sourceforge.net • cflow: ftp://www.ibiblio.org/pub/Linux/devel/lang/c
Why does appending tags to a tag file tag so long?
Sometimes, in an attempt to build a global tag file for all source files in a large source tree of many directories, someone will make an attempt to run ctags in append (-a) mode on every directory in the hierarchy. Each time ctags is invoked, its default behavior is to sort the tag file once the tags for that execution have been added. As the cumulative tag file grows, the sort time increases arithmetically. The best way to avoid this problem (and the most efficient) is to make use of the --recurse (or -R) option of ctags by executing the following command in the root of the directory hierarchy (thus running ctags only once):
ctags -R
If you really insist on running ctags separately on each directory, you can avoid the sort pass each time by spec- ifying the option --sort=no. Once the tag file is completely built, use the sort command to manually sort the final tag file, or let the final invocation of ctags sort the file.
2.6. ctags-faq 58 Universal Ctags Documentation, Release 0.3.0
How should I set up tag files for a multi-level directory hierarchy?
There are a few ways of approaching this: 1. A local tag file in each directory containing only the tags for source files in that directory. 2. One single big, global tag file present in the root directory of your hierarchy, containing all tags present in all source files in the hierarchy. 3. A local tag file in each directory containing only the tags for source files in that directory, in addition to one single global tag file present in the root directory of your hierarchy, containing all non-static tags present in all source files in the hierarchy. 4. A local tag file in each directory of the hierarchy, each one containing all tags present in source files in that directory and all non-static tags in every directory below it (note that this implies also having one big tag file in the root directory of the hierarchy). Each of these approaches has its own set of advantages and disadvantages, depending upon your particular condi- tions. Which approach is deemed best depends upon the following factors: A. The ability of your editor to use multiple tag files. If your editor cannot make use of multiple tag files (original vi implementations could not), then one large tag file is the only way to go if you ever desire to jump to tags located in other directories. If you never need to jump to tags in another directory (i.e. the source in each directory is entirely self-contained), then a local tag file in each directory will fit your needs. B. The time is takes for your editor to look up a tag in the tag file. The significance of this factor depends upon the size of your source tree and on whether the source files are located on a local or remote file system. For source and tag files located on a local file system, looking up a tag is not as big a hit as one might first imagine, since vi implementations typically perform a binary search on a sorted tag file. This may or may not be true for the editor you use. For files located on a remote file system, reading a large file is an expensive operation. C. Whether or not you expect the source code to change and the time it takes to rebuild a tag file to account for changes to the source code. While Universal Ctags is particularly fast in scanning source code (around 1-2 MB/sec), a large project may still result in objectionable delays if one wishes to keep their tag file(s) up to date on a frequent basis, or if the files are located on a remote file system. D. The presence of duplicate tags in the source code and the ability to handle them. The impact of this factor is influenced by the following three issues: 1. How common are duplicate tags in your project? 2. Does your editor provide any facilities for dealing with duplicate tags? While standard vi does not, many modern vi implementations, such as Vim have good facilities for selecting the desired match from the list of duplicates. If your editor does not support duplicate tags, then it will typically send you to only one of them, whether or not that is the one you wanted (and not even notifying you that there are other potential matches). 3. What is the significance of duplicate tags? For example, if you have two tags of the same name from entirely isolated software components, jumping first to the match found in component B while working in component A may be entirely misleading, distracting or inconvenient (to keep having to choose which one if your editor provides you with a list of matches). However, if you have two tags of the same name for parallel builds (say two initialization routines for different hosts), you may always want to specify which one you want. Of the approaches listed above, I tend to favor Approach 3. My editor of choice is Vim, which provides a rich set of features for handling multiple tag files, which partly influences my choice. If you are working with source files on a remote file system, then I would recommend either Approach 3 or Approach 4, depending upon the hit when reading the global tag file.
2.6. ctags-faq 59 Universal Ctags Documentation, Release 0.3.0
The advantages of Approach 3 are many (assuming that your editor has the ability to support both multiple tag files and duplicate tags). All lookups of tag located in the current directory are fast and the local tag file can be quickly and easily regenerated in one second or less (I have even mapped a keystroke to do this easily). A lookup of a (necessarily non-static) tag found in another directory fails a lookup in the local tag file, but is found in the global tag file, which satisfies all cross-directory lookups. The global tag file can be automatically regenerated periodically with a cron job (and perhaps the local tag files also). Now I give an example of how you would implement Approach 3. Means of implementing the other approaches can be performed in a similar manner. Here is a visual representation of an example directory hierarchy: project `-----misccomp | `... `-----sysint `-----client | `-----hdrs | `-----lib | `-----src | `-----test `-----common | `-----hdrs | `-----lib | `-----src | `-----test `-----server `-----hdrs `-----lib `-----src `-----test
Here is a recommended solution (conceptually) to build the tag files: 1. Within each of the leaf nodes (i.e. hdrs, lib, src, test) build a tag file using “ctags *.[ch]”. This can be easily be done for the whole hierarchy by making a shell script, call it dirtags, containing the following lines:
#!/bin/sh cd $1 ctags *
Now execute the following command:
find * -type d -exec dirtags{} \;
These tag files are trivial (and extremely quick) to rebuild while making changes within a directory. The following Vim key mapping is quite useful to rebuild the tag file in the directory of the current source file:
:nmap ,t :!(cd %:p:h;ctags *.[ch])&
2. Build the global tag file:
cd ~/project ctags --file-scope=no -R
thus constructing a tag file containing only non-static tags for all source files in all descendent directories. 3. Configure your editor to read the local tag file first, then consult the global tag file when not found in the local tag file. In Vim, this is done as follows:
:set tags=./tags,tags,~/project/tags
If you wish to implement Approach 4, you would need to replace the dirtags script of step 1 with the following:
2.6. ctags-faq 60 Universal Ctags Documentation, Release 0.3.0
#!/bin/sh cd $1 ctags * # Now append the non-static tags from descendent directories find * -type d -prune -print | ctags -aR --file-scope=no -L-
And replace the configuration of step 3 with this:
:set tags=./tags;$HOME,tags
As a caveat, it should be noted that step 2 builds a global tag file whose file names will be relative to the directory in which the global tag file is being built. This takes advantage of the Vim tagrelative option, which causes the path to be interpreted a relative to the location of the tag file instead of the current directory. For standard vi, which always interprets the paths as relative to the current directory, we need to build the global tag file with absolute path names. This can be accomplished by replacing step 2 with the following:
cd ~/project ctags --file-scope=no -R`pwd`
Does Universal Ctags support Unicode file names?
Yes, Unicode file names are supported on unix-like platforms (Linux, macOS, Cygwin, etc.). However, on Windows, you need to use Windows 10 version 1903 or later to use Unicode file names. (This is an experimental feature, though.) On older versions on Windows, Universal Ctags only support file names represented in the current code page. If you still want to use Unicode file names on them, use Cygwin or MSYS2 version of Universal Ctags as a workaround.
Why does zsh cause “zsh: no matches found” error? zsh causes error on the following cases;
ctags --extra=+* ... ctags --exclude=foo/* ...
This is the 2nd most significant incompatibility feature of zsh. Cited from “Z-Shell Frequently-Asked Questions”, “2.1: Differences from sh and ksh”; . . . The next most classic difference is that unmatched glob patterns cause the command to abort; set NO_NOMATCH for those. You may add “setopt nonomatch” on your ~/.zshrc. Or you can escape glob patterns with backslash;
ctags --extra=+\* ... ctags --exclude=foo/\* ...
Or quote them;
ctags '--extra=+*' ... ctags '--exclude=foo/*' ...
2.6.2 SEE ALSO
The official Universal Ctags web site at: https://ctags.io/ ctags(1), tags(5)
2.6. ctags-faq 61 Universal Ctags Documentation, Release 0.3.0
2.6.3 AUTHOR
This FAQ is based on Exuberant Ctags FAQ by Darren Hiebert and [email protected] Universal Ctags project: https://ctags.io/
2.7 ctags-lang-inko
Version 5.9.0 Manual group Universal Ctags Manual section 7
2.7.1 SYNOPSIS ctags . . . –languages=+Inko . . . ctags . . . –language-force=Inko . . . ctags . . . –map-Inko=+.inko . . .
2.7.2 DESCRIPTION
This man page describes the Inko parser for Universal Ctags. The input file is expected to be valid Inko source code, otherwise the output of ctags is undefined. Tags are generated for objects, traits, methods, attributes, and constants. String literals are ignored.
2.7.3 SEE ALSO ctags(1), ctags-client-tools(7)
2.8 ctags-lang-iPythonCell
The man page of the iPythonCell parser for Universal Ctags Version 5.9.0 Manual group Universal Ctags Manual section 7
2.8.1 SYNOPSIS ctags . . . –extras={subparser} –languages=+iPythonCell,Python \ [–extras-IPythonCell=+{doubleSharps}] \ [–regex-IPythonCell=/
2.7. ctags-lang-inko 62 Universal Ctags Documentation, Release 0.3.0
2.8.2 DESCRIPTION
iPythonCell is a subparser stacked on top of the Python parser. It works when: • The Python parser is enabled, • the subparser extra is enabled, and • the iPythonCell parser itself is enabled. iPythonCell extracts cells explained as in vim-ipython-cell (https://github.com/hanschen/vim-ipython-cell/blob/ master/README.md).
2.8.3 KIND(S)
The iPythonCell parser defines only a cell kind.
2.8.4 EXTRA(S)
Tagging cells staring with ##... is disabled by default because the pattern is too generic; with that pattern unwanted tags can be extracted. Enable doubleSharps language specific extra for tagging cells staring with ##....
2.8.5 CUSTOMIZING
If your favorite cell pattern is not supported in the parser, you can define the pattern in your .ctagd.d/your. ctags or command lines. Here is an example how to support “# CTAGS: ...”: “input.py”
x=1 # CTAGS: DEFINE F def F(): # CTAGS: DO NOTING pass
“output.tags” with “–options=NONE –sort=no –extras=+{subparser} –regex-IPythonCell=/[ t]*# CTAGS:[ ]?(.*)$/1/c/ -o - input.py”
x input.py /^x=1$/;" v DEFINE F input.py /^# CTAGS: DEFINE F$/;" c F input.py /^def F():$/;" f DO NOTING input.py /^ # CTAGS: DO NOTING$/;" c
You can put “--regex-IPythonCell=/[ \t]*# CTAGS:[ ]?(.*)$/\1/c/” in your.ctags to avoid specifying the pattern repeatedly.
2.8.6 SEE ALSO
ctags(1), ctags-client-tools(7), ctags-lang-python(7)
2.9 ctags-lang-julia
Random notes about tagging Julia source code with Universal-ctags Version 5.9.0
2.9. ctags-lang-julia 63 Universal Ctags Documentation, Release 0.3.0
Manual group Universal-ctags Manual section 7
2.9.1 SYNOPSIS ctags . . . –languages=+Julia . . . ctags . . . –language-force=Julia . . . ctags . . . –map-Julia=+.jl . . .
2.9.2 DESCRIPTION
This man page gathers random notes about tagging Julia source code.
2.9.3 TAGGING import AND using EXPRESSIONS
Summary using X
name kind role other noticeable fields X module used N/A using X: a, b
name kind role other noticeable fields X module namespace N/A a, b unknown used scope:module:X import X
name kind role other noticeable fields X module imported N/A import X.a, Y.b
name kind role other noticeable fields X, Y module namespace N/A a unknown imported scope:module:X b unknown imported scope:module:Y import X: a, b
name kind role other noticeable fields X module namespace N/A a,b unknown imported scope:module:X
Examples
“input.jl”
2.9. ctags-lang-julia 64 Universal Ctags Documentation, Release 0.3.0
using X0
“output.tags” with “–options=NONE -o - –extras=+r –fields=+rzK input.jl”
X0 input.jl /^using X0$/;" kind:module roles:used
--extras=+r (or --extras=+{reference}) option is needed for this tag, since it’s a reference tag. This is because module X is not defined here. It is defined in another file. Enable roles: field with --fields=+r is for recording that the module is “used”, i.e., loaded by using. “input.jl”
import X1.a, X2.b, X3
“output.tags” with “–options=NONE -o - –extras=+r –fields=+rzKZ input.jl”
X1 input.jl /^import X1.a, X2.b, X3$/;" kind:module
˓→roles:namespace X2 input.jl /^import X1.a, X2.b, X3$/;" kind:module
˓→roles:namespace X3 input.jl /^import X1.a, X2.b, X3$/;" kind:module
˓→roles:imported a input.jl /^import X1.a, X2.b, X3$/;" kind:unknown
˓→scope:module:X1 roles:imported b input.jl /^import X1.a, X2.b, X3$/;" kind:unknown
˓→scope:module:X2 roles:imported
Why X1 and X2 have role “namespace”, while X3 have role “imported”? It’s because the symbol a in module X1, and b in module X2 are brought to the current scope, but X1 and X2 themselves are not. We use “namespace” role for such modules. X3 is different. The symbol X3, together with all exported symbols in X3, is brought to current scope. For such modules, we use “imported” or “used” role depending whether they are loaded by import or using. Also, notice that a and b have the “unknown” kind. This is because we cannot know whether it’s a function, constant, or macro, etc.
2.9.4 SEE ALSO ctags(1), ctags-client-tools(7)
2.10 ctags-lang-python
Random notes about tagging python source code with Universal Ctags Version 5.9.0 Manual group Universal Ctags Manual section 7
2.10.1 SYNOPSIS ctags . . . –languages=+Python . . . ctags . . . –language-force=Python . . . ctags . . . –map-Python=+.py . . .
2.10. ctags-lang-python 65 Universal Ctags Documentation, Release 0.3.0
2.10.2 DESCRIPTION
This man page gathers random notes about tagging python source code.
2.10.3 TAGGING import STATEMENTS
Summary import X
name kind role other noticeable fields X module imported N/A import X as Y
name kind role other noticeable fields X module indirectlyImported N/A Y namespace definition nameref:module:X from X import *
name kind role other noticeable fields X module namespace N/A from X import Y
name kind role other noticeable fields X module namespace N/A Y unknown imported scope:module:X from X import Y as Z
name kind role other noticeable fields X module namespace N/A Y unknown indirectlyImported scope:module:X Z unknown definition nameref:unknown:Y
Examples
“input.py” import X0
“output.tags” with “–options=NONE -o - –extras=+r –fields=+rzK input.py”
X0 input.py /^import X0$/;" kind:module roles:imported
A tag for an imported module has module kind with imported role. The module is not defined here; it is defined in another file. So the tag for the imported module is a reference tag; specify --extras=+r (or --extras=+{reference}) option for tagging it. “roles:” field enabled with --fields=+r is for recording the module is “imported” to the tag file. “input.py”
2.10. ctags-lang-python 66 Universal Ctags Documentation, Release 0.3.0
import X1 as Y1
“output.tags” with “–options=NONE -o - –extras=+r –fields=+rzK –fields-Python=+{nameref} input.py”
X1 input.py /^import X1 as Y1$/;" kind:module
˓→roles:indirectlyImported Y1 input.py /^import X1 as Y1$/;" kind:namespace roles:def
˓→nameref:module:X1
“Y1” introduces a new name and is defined here. So “Y1” is tagged as a definition tag. “X1” is imported in a way that its name cannot be used in this source file. For letting client tools know that the name cannot be used, indirectlyImported role is assigned for “X1”. “Y1” is the name for accessing objects defined in the module imported via “X1”. For recording this relationship, nameref: field is attached to the tag of “Y1”. Instead of module kind, namespace kind is assigned to “Y1” because “Y1” itself isn’t a module. “input.py”
from X2 import *
“output.tags” with “–options=NONE -o - –extras=+r –fields=+rzK input.py”
X2 input.py /^from X2 import *$/;" kind:module roles:namespace
The module is not defined here; it is defined in another file. So the tag for the imported module is a reference tag. Unlike “X0” in “import X0”, “X2” may not be used because the names defined in “X2” can be used in this source file. To represent the difference namespace role is attached to “X2” instead of imported. “input.py”
from X3 import Y3
“output.tags” with “–options=NONE -o - –extras=+r –fields=+rzKZ input.py”
X3 input.py /^from X3 import Y3$/;" kind:module roles:namespace Y3 input.py /^from X3 import Y3$/;" kind:unknown scope:module:X3
˓→roles:imported
“Y3” is a name for a language object defined in “X3” module. “scope:module:X3” attached to “Y3” represents this relation between “Y3” and “X3”. ctags assigns unknown kind to “Y3” because ctags cannot know whether “Y3” is a class, a variable, or a function from the input file. “input.py”
from X4 import Y4 as Z4
“output.tags” with “–options=NONE -o - –extras=+r –fields=+rzKZ input.py”
X4 input.py /^from X4 import Y4 as Z4$/;" kind:module
˓→roles:namespace Y4 input.py /^from X4 import Y4 as Z4$/;" kind:unknown
˓→scope:module:X4 roles:indirectlyImported Z4 input.py /^from X4 import Y4 as Z4$/;" kind:unknown roles:def
˓→ nameref:unknown:Y4
“Y4” is similar to “Y3” of “from X3 import Y3” but the name cannot be used here. indirectlyImported role assigned to “Y4” representing this. “Z4” is the name for accessing the language object named in “Y4” in “X4” module. “nameref:unknown:Y4” attached to “Z4” and “scope:module:X4” attached to “Y4” represent the relations.
2.10.4 LAMBDA EXPRESSION AND TYPE HINT
2.10. ctags-lang-python 67 Universal Ctags Documentation, Release 0.3.0
Summary id = lambda var0: var0
name kind role other noticeable fields id function definition signature:(var0) id_t: Callable[[int], int] = lambda var1: var1
name kind role other noticeable fields id_t vari- defini- typeref:typename:Callable[[int], int] able tion nameref:function:anonFuncN anon- func- defini- signature:(var1) FuncN tion tion
Examples
“input.py” from typing import Callable id= lambda var0: var0 id_t: Callable[[int], int]= lambda var1: var1
“output.tags” with “–options=NONE -o - –sort=no –fields=+KS –fields-Python=+{nameref} – extras=+{anonymous} input.py” id input.py /^id = lambda var0: var0$/;" function
˓→signature:(var0) id_t input.py /^id_t: Callable[[int], int] = lambda var1: var1$/;"\ variable typeref:typename:Callable[[int], int]
˓→nameref:function:anonFunc84011d2c0101 anonFunc84011d2c0101 input.py /^id_t: Callable[[int], int] = lambda
˓→var1: var1$/;"\ function signature:(var1)
If a variable (“id”) with no type hint is initialized with a lambda expression, ctags assigns function kind for the tag of “id”. If a variable (“id_t”) with a type hint is initialized with a lambda expression, ctags assigns variable kind for the tag of “id_t” with typeref: and nameref: fields. ctags fills typeref: field with the value of the type hint. The way of filling nameref: is a bit complicated. For the lambda expression used in initializing the type-hint’ed variable, ctags creates anonymous extra tag (“anonFunc84011d2c0101”). ctags fills the nameref: field of “id_t” with the name of anonymous extra tag: “nameref:function:anonFunc84011d2c0101”. You may think why ctags does so complicated, and why ctags doesn’t emit following tags output for the input: id input.py /^id = \\$/;" function signature:(var0) id_t input.py /^id_t: \\$/;" function
˓→typeref:typename:Callable[[int], int] signature:(var1)
There is a reason. The other languages of ctags obey the following rule: ctags fills typeref: field for a tag of a callable object (like function) with the type of its return value. If we consider “id_t” is a function, its typeref: field should have “typename:int”. However, for filling typeref: with “typename:int”, ctags has to analyze “Callable[[int], int]” deeper. We don’t want to do so.
2.10. ctags-lang-python 68 Universal Ctags Documentation, Release 0.3.0
2.10.5 SEE ALSO
ctags(1), ctags-client-tools(7), ctags-lang-iPythonCell(7)
2.11 ctags-lang-r
Random notes about tagging R source code with Universal Ctags Version 5.9.0 Manual group Universal Ctags Manual section 7
2.11.1 SYNOPSIS ctags . . . –languages=+R . . . ctags . . . –language-force=R . . . ctags . . . –map-Python=+.r . . .
2.11.2 DESCRIPTION
This man page gathers random notes about tagging R source code with Universal Ctags.
2.11.3 Kinds
If a variable gets a value returned from a well-known constructor and the variable appears for the first time in the current input file, the R parser makes a tag for the variable and attaches a kind associated with the constructor to the tag regardless of whether the variable appears in the top-level context or a function. Well-known constructor and kind mapping
Constructor kind function() function c() vector list() list data.frame() dataframe
If a variable doesn’t get a value returned from one of well-known constructors, the R parser attaches globalVar or functionVar kind to the tag for the variable depending on the context. Here is an example demonstrating the usage of the kinds: “input.r”
G <-1 v <-c(1,2) l <- list(3,4) d <- data.frame(n= v) f <- function(a) { g <- function (b) a+b w <-c(1,2) m <- list(3,4) e <- data.frame(n= w) L <-2 }
2.11. ctags-lang-r 69 Universal Ctags Documentation, Release 0.3.0
“output.tags” with “–options=NONE –sort=no –fields=+KZ -o - input.r”
G input.r /^G <- 1$/;" globalVar v input.r /^v <- c(1, 2)$/;" vector l input.r /^l <- list(3, 4)$/;" list d input.r /^d <- data.frame(n = v)$/;" dataframe n input.r /^d <- data.frame(n = v)$/;" nameattr scope:dataframe:d f input.r /^f <- function(a) {$/;" function g input.r /^ g <- function (b) a + b$/;" function
˓→scope:function:f w input.r /^ w <- c(1, 2)$/;" vector scope:function:f m input.r /^ m <- list (3, 4)$/;" list scope:function:f e input.r /^ e <- data.frame(n = w)$/;" dataframe
˓→scope:function:f n input.r /^ e <- data.frame(n = w)$/;" nameattr
˓→scope:dataframe:f.e L input.r /^ L <- 2$/;" functionVar scope:function:f
2.11.4 SEE ALSO
ctags(1)
2.12 ctags-lang-sql
The man page of the SQL parser for Universal Ctags Version 5.9.0 Manual group Universal Ctags Manual section 7
2.12.1 SYNOPSIS ctags . . . [–extras={guest}] –languages=+SQL . . .
2.12.2 DESCRIPTION
The SQL parser supports various SQL dialects. PostgreSQL is one of them. PostgreSQL allows user-defined functions to be written in other languages (procedural languages) besides SQL and C [PL]. The SQL parser makes tags for language objects in the user-defined functions written in the procedural languages if the guest extra is enabled. The SQL parser looks for a token coming after LANGUAGE keyword in the source code to choose a proper guest parser.
... LANGUAGE plpythonu AS '... user-defined function ' ...... AS $$ user-defined function $$ LANGUAGE plv8 ...
In the above examples, plpythonu and plv8 are the names of procedural languages. The SQL parser trims pl at the start and u at the end of the name before finding a ctags parser. For plpythonu and plv8, the SQL parser extracts python and v8 as the candidates of guest parsers. For plpythonu, ctags can run its Python parser. ctags doesn’t have a parser named v8. However, the JavaScript parser in ctags has v8 as an alias. So ctags can run the JavaScript parser as the guest parser for plv8.
2.12. ctags-lang-sql 70 Universal Ctags Documentation, Release 0.3.0
2.12.3 EXAMPLES tagging code including a user-defined function in a string literal [GH3006]: “input.sql”
CREATEOR REPLACE FUNCTION fun1() RETURNS VARCHAR AS ' DECLARE test1_var1 VARCHAR(64) := $$ABC$$; test1_var2 VARCHAR(64) := $xyz$XYZ$xyz$; test1_var3 INTEGER := 1; BEGIN RETURN TO_CHAR(test_var3, ''000'') || test1_var1 || test1_var2; END; ' LANGUAGE plpgsql;
“output.tags” with “–options=NONE -o - –sort=no –extras=+{guest} input.sql” fun1 input.sql /^CREATE OR REPLACE FUNCTION fun1() RETURNS VARCHAR AS '$/;
˓→" f test1_var1 input.sql /^ test1_var1 VARCHAR(64) := $$ABC$$;$/;" v test1_var2 input.sql /^ test1_var2 VARCHAR(64) := $xyz$XYZ$xyz$;$/;
˓→" v test1_var3 input.sql /^ test1_var3 INTEGER := 1;$/;" v tagging code including a user-defined function in a dollar quote [GH3006]: “input.sql”
CREATE OR REPLACE FUNCTION fun2() RETURNS VARCHAR LANGUAGE plpgsql AS $$ DECLARE test2_var1 VARCHAR(64) := 'ABC2'; test2_var2 VARCHAR(64) := 'XYZ2'; test2_var3 INTEGER := 2; BEGIN RETURN TO_CHAR(test2_var3, '000') || test2_var1 || test2_var2; END; $$;
“output.tags” with “–options=NONE -o - –sort=no –extras=+{guest} input.sql” fun2 input.sql /^CREATE OR REPLACE FUNCTION fun2() RETURNS VARCHAR
˓→LANGUAGE plpgsql AS $\$$/;" f test2_var1 input.sql /^ test2_var1 VARCHAR(64) := 'ABC2';$/;" v test2_var2 input.sql /^ test2_var2 VARCHAR(64) := 'XYZ2';$/;" v test2_var3 input.sql /^ test2_var3 INTEGER := 2;$/;" v tagging code including a user-defined written in JavaScript:
-- Derived from https://github.com/plv8/plv8/blob/r3.0alpha/sql/plv8.sql CREATE FUNCTION test(keys text[], vals text[]) RETURNS text AS $$ var o = {}; for (var i = 0; i < keys.length; i++) o[keys[i]] = vals[i]; return JSON.stringify(o); $$ LANGUAGE plv8 IMMUTABLE STRICT;
“output.tags” with “–options=NONE -o - –sort=no –extras=+{guest} input.sql”
2.12. ctags-lang-sql 71 Universal Ctags Documentation, Release 0.3.0
test input.sql /^CREATE FUNCTION test(keys text[], vals text[]) RETURNS
˓→text AS$/;" f o input.sql /^ var o = {};$/;" v
2.12.4 KNOWN BUGS
Escape sequences (‘’) in a string literal may make a guest parser confused.
2.12.5 SEE ALSO
ctags(1), ctags-client-tools(7)
2.12.6 REFERENCES
2.13 ctags-lang-tcl
Random notes about tagging tcl source code with Universal Ctags Version 5.9.0 Manual group Universal Ctags Manual section 7
2.13.1 SYNOPSIS ctags . . . –languages=+Tcl . . . ctags . . . –language-force=Tcl . . . ctags . . . –map-Tcl=+.tcl . . .
2.13.2 DESCRIPTION
This man page gathers random notes about tagging tcl source code.
2.13.3 TAGGING language objects of OO Extensions
TclOO parser and ITcl parser are subparsers running on the Tcl parser. As the names of parsers show, they are for tagging language objects of object oriented programming extensions for the Tcl language. A pattern, “namespace import oo” in an input file activates the TclOO parser. A pattern, “namespace import itcl” in an input file activates the ITcl parser. There are cases that one of the OO extensions is used though neither pattern are appeared in an input file. Consider the following input files: “main.tcl”
package require Itcl namespace import itcl::* source input.tcl
“input.tcl”
2.13. ctags-lang-tcl 72 Universal Ctags Documentation, Release 0.3.0
class MyClass { public method foo {}{ } }
The pattern for activating the ITcl parser is not appeared in “input.tcl” though “class” command is used. As a result, ctags cannot extract “MyClass”. The parameters TclOO.forceUse=true|[false] and ITcl.forceuse=true|[false] for handling this situation. With the parameter, you can force ctags to activate one of the subparsers. You can use the parameters like --param-ITcl.forceuse=true in a command-line. Note that you can enable only one of ITcl parser or TclOO parser. Enabling both parsers with specifying the parameters can cause unexpected results.
2.13.4 SEE ALSO ctags(1)
2.14 ctags-lang-verilog
The man page about SystemVerilog/Verilog parser for Universal Ctags Version 5.9.0 Manual group Universal Ctags Manual section 7
2.14.1 SYNOPSIS ctags . . . [–kinds-systemverilog=+Q] [–fields-SystemVerilog=+{parameter}] . . . ctags . . . [–fields-Verilog=+{parameter}] . . .
Language Language ID File Mapping SystemVerilog SystemVerilog .sv, .svh, svi Verilog Verilog .v
2.14.2 DESCRIPTION
This man page describes about the SystemVerilog/Verilog parser for Universal Ctags. SystemVerilog parser sup- ports IEEE Std 1800-2017 keywords. Verilog parser supports IEEE Std 1364-2005 keywords.
Supported Kinds
$ ctags --list-kinds-full=SystemVerilog #LETTER NAME ENABLED REFONLY NROLES MASTER DESCRIPTION A assert yes no 0 NONE assertions (assert, assume, cover,
˓→ restrict) C class yes no 0 NONE classes E enum yes no 0 NONE enumerators H checker yes no 0 NONE checkers I interface yes no 0 NONE interfaces (continues on next page)
2.14. ctags-lang-verilog 73 Universal Ctags Documentation, Release 0.3.0
(continued from previous page) K package yes no 0 NONE packages L clocking yes no 0 NONE clocking M modport yes no 0 NONE modports N nettype yes no 0 NONE nettype declarations O constraint yes no 0 NONE constraints P program yes no 0 NONE programs Q prototype no no 0 NONE prototypes (extern, pure) R property yes no 0 NONE properties S struct yes no 0 NONE structs and unions T typedef yes no 0 NONE type declarations V covergroup yes no 0 NONE covergroups b block yes no 0 NONE blocks (begin, fork) c constant yes no 0 NONE constants (define, parameter,
˓→specparam, enum values) e event yes no 0 NONE events f function yes no 0 NONE functions i instance yes no 0 NONE instances of module or interface l ifclass yes no 0 NONE interface class m module yes no 0 NONE modules n net yes no 0 NONE net data types p port yes no 0 NONE ports q sequence yes no 0 NONE sequences r register yes no 0 NONE variable data types t task yes no 0 NONE tasks w member yes no 0 NONE struct and union members
Note that prototype (Q) is disabled by default.
$ ctags --list-kinds-full=Verilog #LETTER NAME ENABLED REFONLY NROLES MASTER DESCRIPTION b block yes no 0 NONE blocks (begin, fork) c constant yes no 0 NONE constants (define, parameter,
˓→specparam) e event yes no 0 NONE events f function yes no 0 NONE functions i instance yes no 0 NONE instances of module m module yes no 0 NONE modules n net yes no 0 NONE net data types p port yes no 0 NONE ports r register yes no 0 NONE variable data types t task yes no 0 NONE tasks
Supported Language Specific Fields
$ ctags --list-fields=Verilog #LETTER NAME ENABLED LANGUAGE JSTYPE FIXED DESCRIPTION - parameter no Verilog --b no parameter whose value can be
˓→overridden. $ ctags --list-fields=SystemVerilog #LETTER NAME ENABLED LANGUAGE JSTYPE FIXED DESCRIPTION - parameter no SystemVerilog --b no parameter whose value can be
˓→overridden. parameter field
If the field parameter is enabled, a field parameter: is added on a parameter whose value can be overridden on an instantiated module, interface, or program. This is useful for a editor plugin or extension to enable auto- instantiation of modules with parameters which can be overridden.
2.14. ctags-lang-verilog 74 Universal Ctags Documentation, Release 0.3.0
$ ctags ... --fields-Verilog=+{parameter} ... $ ctags ... --fields-SystemVerilog=+{parameter} ...
On the following source code fields parameter: are added on parameters P*, not on ones L*. Note that L4 and L6 is declared by parameter statement, but fields parameter: are not added, because they cannot be overridden. “input.sv”
// compilation unit scope parameter L1="synonym for the localparam"; module with_parameter_port_list #( P1, localparam L2= P1+1, parameter P2) ( /*port list...*/ ); parameter L3="synonym for the localparam"; localparam L4="localparam"; // ... endmodule module with_empty_parameter_port_list #() ( /*port list...*/ ); parameter L5="synonym for the localparam"; localparam L6="localparam"; // ... endmodule module no_parameter_port_list ( /*port list...*/ ); parameter P3="parameter"; localparam L7="localparam"; // ... endmodule
$ ctags -uo - --fields-SystemVerilog=+{parameter} input.sv L1 input.sv /^parameter L1 = "synonym for the localparam";$/;" c
˓→ parameter: with_parameter_port_list input.sv /^module with_parameter_port_list
˓→#($/;" m P1 input.sv /^ P1,$/;" c module:with_parameter_port_list
˓→parameter: L2 input.sv /^ localparam L2 = P1+1,$/;" c
˓→module:with_parameter_port_list P2 input.sv /^ parameter P2)$/;" c module:with_
˓→parameter_port_list parameter: L3 input.sv /^ parameter L3 = "synonym for the localparam";$/;"
˓→ c module:with_parameter_port_list L4 input.sv /^ localparam L4 = "localparam";$/;" c
˓→module:with_parameter_port_list with_empty_parameter_port_list input.sv /^module with_empty_parameter_port_
˓→list #()$/;" m L5 input.sv /^ parameter L5 = "synonym for the localparam";$/;"
˓→ c module:with_empty_parameter_port_list L6 input.sv /^ localparam L6 = "localparam";$/;" c
˓→module:with_empty_parameter_port_list no_parameter_port_list input.sv /^module no_parameter_port_list$/;" m P3 input.sv /^ parameter P3 = "parameter";$/;" c
˓→module:no_parameter_port_list parameter: L7 input.sv /^ localparam L7 = "localparam";$/;" c
˓→module:no_parameter_port_list
2.14. ctags-lang-verilog 75 Universal Ctags Documentation, Release 0.3.0
TIPS
If you want to map files *.v to SystemVerilog, add --langmap=SystemVerilog:.v option.
2.14.3 KNOWN ISSUES
See https://github.com/universal-ctags/ctags/issues/2674 for more information.
2.14.4 SEE ALSO
• ctags(1) • ctags-client-tools(7) • Language Reference Manuals (LRM) – IEEE Standard for SystemVerilog — Unified Hardware Design, Specification, and Verification Language, IEEE Std 1800-2017, https://ieeexplore.ieee.org/document/8299595 – IEEE Standard for Verilog Hardware Description Language, IEEE Std 1364-2005, https:// ieeexplore.ieee.org/document/1620780
2.15 readtags
Find tag file entries matching specified names Version 5.9.0 Manual group Universal Ctags Manual section 1
2.15.1 SYNOPSIS readtags -h | –help readtags (-H | –help-expression) (filter|sorter) readtags [OPTION]. . . ACTION
2.15.2 DESCRIPTION
The readtags program filters, sorts and prints tag entries in a tags file. The basic filtering is done using actions, by which you can list all regular tags, pseudo tags or regular tags matching specific name. Then, further filtering and sorting can be done using post processors, namely filter expressions and sorter expressions.
2.15.3 ACTIONS
-l, --list List regular tags. [-] NAME List regular tags matching NAME. “-” as NAME indicates arguments after this as NAME even if they start with -. -D, --list-pseudo-tags Equivalent to --list-pseudo-tags.
2.15. readtags 76 Universal Ctags Documentation, Release 0.3.0
2.15.4 OPTIONS
Controlling the Tags Reading Behavior
The behavior of reading tags can be controlled using these options: -t TAGFILE, --tag-file TAGFILE Use specified tag file (default: “tags”). -s[0|1|2], --override-sort-detection METHOD Override sort detection of tag file. METHOD: un- sorted|sorted|foldcase The NAME action will perform binary search on sorted (including “foldcase”) tags files, which is much faster then on unsorted tags files.
Controlling the NAME Action Behavior
The behavior of the NAME action can be controlled using these options: -i, --icase-match Perform case-insensitive matching in the NAME action. -p, --prefix-match Perform prefix matching in the NAME action.
Controlling the Output
By default, the output of readtags contains only the name, input and pattern field. The Output can be tweaked using these options: -d, --debug Turn on debugging output. -E, --escape-output Escape characters like tabs in output as described in tags(5). -e, --extension-fields Include extension fields in output. -n, --line-number Also include the line number field when -e option is give. About the -E option: certain characters are escaped in a tags file, to make it machine-readable. e.g., ensuring no tabs character appear in fields other than the pattern field. By default, readtags translates them to make it human-readable, but when utilizing readtags output in a script or a client tool, -E option should be used. See ctags-client-tools(7) for more discussion on this.
Filtering and Sorting
Further filtering and sorting on the tags listed by actions are performed using: -Q EXP, --filter EXP Filter the tags listed by ACTION with EXP before printing. -S EXP, --sorter EXP Sort the tags listed by ACTION with EXP before printing. These are discussed in the EXPRESSION section.
Examples
• List all tags in “/path/to/tags”:
$ readtags -t /path/to/tags -l
• List all tags in “tags” that start with “mymethod”:
$ readtags -p - mymethod
• List all tags matching “mymethod”, case insensitively:
2.15. readtags 77 Universal Ctags Documentation, Release 0.3.0
$ readtags -i - mymethod
• List all tags start with “myvar”, and printing all fields (i.e., the whole line):
$ readtags -p -ne - myvar
2.15.5 EXPRESSION
Scheme-style expressions are used for the -Q and -S options. For those who doesn’t know Scheme or Lisp, just remember: • A function call is wrapped in a pair of parenthesis. The first item in it is the function/operator name, the others are arguments. • Function calls can be nested. • Missing values and boolean false are represented by #f. #t and all other values are considered to be true. So, (+ 1 (+ 2 3)) means add 2 and 3 first, then add the result with 1. (and "string" 1 #t) means logical AND on "string", 1 and #t, and the result is true since there is no #f.
Filtering
The tag entries that make the filter expression produces true value are printed by readtags. The basic operators for filtering are eq?, prefix?, suffix?, substr?, and #/PATTERN/. Language com- mon fields can be accessed using variables starting with $, e.g., $language represents the language field. For example: • List all tags start with “myfunc” in Python code files:
$ readtags -p -Q '(eq? $language "Python")' - myfunc
downcase or upcase operators can be used to perform case-insensitive matching: • List all tags containing “my”, case insensitively:
$ readtags -Q '(substr? (downcase $name) "my")' -l
We have logical operators like and, or and not. The value of a missing field is #f, so we could deal with missing fields: • List all tags containing “impl” in Python code files, but allow the language: field to be missing:
$ readtags -Q '(and (substr? $name "impl")\ (or (not $language)\ (eq? $language "Python")))' -l
#/PATTERN/ is for the case when string predicates (prefix?, suffix?, and substr?) are not enough. You can use “Posix extended regular expression” as PATTERN. • List all tags inherits from the class “A”:
$ readtags -Q '(#/(^|,) ?A(,|$)/ $inherits)' -l
Here $inherits is a comma-separated class list like “A,B,C”, “P, A, Q”, or just “A”. Notice that this filter works on both situations where there’s a space after each comma or there’s not. Case-insensitive matching can be performed by #/PATTERN/i: • List all tags inherits from the class “A” or “a”:
2.15. readtags 78 Universal Ctags Documentation, Release 0.3.0
$ readtags -Q '(#/(^|,) ?A(,|$)/i $inherits)' -l
To include “/” in a pattern, prefix \ to the “/”. NOTE: The above regular expression pattern for inspecting inheritances is just an example to show how to use #/ PATTERN/ expression. Tags file generators have no consensus about the format of inherits:, e.g., whether there should be a space after a comma. Even parsers in ctags have no consensus. Noticing the format of the inherits: field of specific languages is needed for such queries. The expressions #/PATTERN/ and #/PATTERN/i are for interactive use. Readtags also offers an alias string->regexp, so #/PATTERN/ is equal to (string->regexp "PATTERN"), and #/PATTERN/ i is equal to (string->regexp "PATTERN" :case-fold #t). string->regexp doesn’t need to prefix \ for including “/” in a pattern. string->regexp may simplify a client tool building an expression. See also ctags-client-tools(7) for building expressions in your tool. Let’s now consider missing fields. The tags file may have tag entries that has no inherits: field. In that case $inherits is #f, and the regular expression matching raises an error, since string operators only work for strings. To avoid this problem: • Safely list all tags inherits from the class “A”:
$ readtags -Q '(and $inherits (#/(^|,) ?A(,|$)/ $inherits))' -l
This makes sure $inherits is not missing first, then match it by regexp. Sometimes you want to keep tags where the field is missing. For example, your want to exclude reference tags, which is marked by the extras: field, then you want to keep tags who doesn’t have extras: field since they are also not reference tags. Here’s how to do it: • List all tags but the reference tags:
$ readtags -Q '(or (not $extras) (#/(^|,) ?reference(,|$)/ $extras))' -l
Notice that (not $extras) produces #t when $extras is missing, so the whole or expression produces #t. Run “readtags -H filter” to know about all valid functions and variables.
Sorting
When sorting, the sorter expression is evaluated on two tag entries to decide which should sort before the other one, until the order of all tag entries is decided. In a sorter expression, $ and & are used to access the fields in the two tag entries, and let’s call them $-entry and &-entry. The sorter expression should have a value of -1, 0 or 1. The value -1 means the $-entry should be put above the &-entry, 1 means the contrary, and 0 makes their order in the output uncertain. The core operator of sorting is <>. It’s used to compare two strings or two numbers (numbers are for the line: or end: fields). In (<> a b), if a < b, the result is -1; a > b produces 1, and a = b produces 0. Strings are compared using the strcmp function, see strcmp(3). For example, sort by names, and make those shorter or alphabetically smaller ones appear before the others:
$ readtags -S '(<> $name &name)' -l
This reads “If the tag name in the $-entry is smaller, it goes before the &-entry”. The
$ readtags -S '(
2.15. readtags 79 Universal Ctags Documentation, Release 0.3.0
The *- operator is used to flip the compare result. i.e., (*- (<> a b)) is the same as (<> b a). Filter expressions can be used in sorter expressions. The technique is use if to produce integers that can be compared based on the filter, like:
(<>( if filter-expr-on-$-entry -11) (if filter-expr-on-&-entry -11))
So if $-entry satisfies the filter, while &-entry doesn’t, it’s the same as (<> -1 1), which produces -1. For example, we want to put tags with “file” kind below other tags, then the sorter would look like:
(<>( if (eq? $kind "file")1 -1) (if (eq? &kind "file")1 -1))
A quick read tells us: If $-entry has “file” kind, and &-entry doesn’t, the sorter becomes (<> 1 -1), which produces 1, so the $-entry is put below the &-entry, exactly what we want.
Inspecting the Behavior of Expressions
The print operator can be used to print the value of an expression. For example:
$ readtags -Q '(print $name)' -l
prints the name of each tag entry before it. Since the return value of print is not #f, all the tag entries are printed. We could control this using the begin or begin0 operator. begin returns the value of its last argument, and begin0 returns the value of its first argument. For example:
$ readtags -Q '(begin0 #f (print (prefix? "ctags" "ct")))' -l
prints a bunch of “#t” (depending on how many lines are in the tags file), and the actual tag entries are not printed.
2.15.6 SEE ALSO
See tags(5) for the details of tags file format. See ctags-client-tools(7) for the tips writing a tool utilizing tags file. The official Universal Ctags web site at: https://ctags.io/ The git repository for the library used in readtags command: https://github.com/universal-ctags/libreadtags
2.15.7 CREDITS
Universal Ctags project https://ctags.io/ Darren Hiebert
2.15. readtags 80 CHAPTER 3
Parsers
This section deals with individual parser topics.
3.1 Asm parser
Maintainer Masatake YAMATO
$ cat input.S #define S1
$ e-ctags --fields=+l -o - input.S S input.S /^#define S 1$/;" d language:Asm
$ u-ctags --fields=+l -o - input.S S input.S /^#define S /;" d language:CPreProcessor file:
3.2 CMake parser
The CMake parser is used for .cmake and CMakeLists.txt files. It generates tags for the following items: • User-defined functions • User-defined macros • User-defined options created by option() • Variables defined by set() • Targets created by add_custom_target(), add_executable() and add_library() The parser uses the experimental multi-table regex ctags options to perform the parsing and tag generation. Caveats:
81 Universal Ctags Documentation, Release 0.3.0
Names that are ${} references to variables are not tagged. For example, given the following:
set(PROJECT_NAME_STR ${PROJECT_NAME}) add_executable( ${PROJECT_NAME_STR} ... ) add_custom_target( ${PROJECT_NAME_STR}_tests ... ) add_library( sharedlib ... )
. . . the variable PROJECT_NAME_STR and target sharedlib will both be tagged, but the other targets will not be. References: https://cmake.org/cmake/help/latest/manual/cmake-language.7.html
3.3 The new C/C++ parser
Maintainer Szymon Tomasz Stefanek
3.3.1 Introduction
The C++ language has strongly evolved since the old C/C++ parser was written. The old parser was struggling with some of the new features of the language and has shown signs of reaching its limits. For this reason in February/March 2016 the C/C++ parser was rewritten from scratch. In the first release several outstanding bugs were fixed and some new features were added. Among them: • Tagging of “using namespace” declarations • Tagging of function parameters • Extraction of function parameter types • Tagging of anonymous structures/unions/classes/enums • Support for C++11 lambdas (as anonymous functions) • Support for function-level scopes (for local variables and parameters) • Extraction of local variables which include calls to constructors • Extraction of local variables from within the for(), while(), if() and switch() parentheses. • Support for function prototypes/declarations with trailing return type At the time of writing (March 2016) more features are planned.
3.3.2 Notable New Features
Some of the notable new features are described below.
Properties
Several properties of functions and variables can be extracted and placed in a new field called properties. The syntax to enable it is:
$ ctags ... --fields-c++=+{properties} ...
At the time of writing the following properties are reported: • virtual: a function is marked as virtual
3.3. The new C/C++ parser 82 Universal Ctags Documentation, Release 0.3.0
• static: a function/variable is marked as static • inline: a function implementation is marked as inline • explicit: a function is marked as explicit • extern: a function/variable is marked as extern • const: a function is marked as const • pure: a virtual function is pure (i.e = 0) • override: a function is marked as override • default: a function is marked as default • final: a function is marked as final • delete: a function is marked as delete • mutable: a variable is marked as mutable • volatile: a function is marked as volatile • specialization: a function is a template specialization • scopespecialization: template specialization of scope a
Preprocessor macros
Defining a macro from command line
The new parser supports the definition of real preprocessor macros via the -D option. All types of macros are supported, including the ones with parameters and variable arguments. Stringification, token pasting and recursive macro expansion are also supported. Option -I is now simply a backward-compatible syntax to define a macro with no replacement. The syntax is similar to the corresponding gcc -D option. Some examples follow.
$ ctags ... -D IGNORE_THIS ...
With this commandline the following C/C++ input
int IGNORE_THIS a;
will be processed as if it was
int a;
Defining a macro with parameters uses the following syntax:
$ ctags ... -D "foreach(arg)=for(arg;;)" ...
This example defines for(arg;;) as the replacement foreach(arg). So the following C/C++ input
foreach(char * p,pointers) {
}
is processed in new C/C++ parser as:
3.3. The new C/C++ parser 83 Universal Ctags Documentation, Release 0.3.0
for(char * p;;) {
} and the p local variable can be extracted. The previous commandline includes quotes since the macros generally contain characters that are treated specially by the shells. You may need some escaping. Token pasting is performed by the ## operator, just like in the normal C preprocessor.
$ ctags ... -D "DECLARE_FUNCTION(prefix)=int prefix ## Call();"
So the following code
DECLARE_FUNCTION(a) DECLARE_FUNCTION(b) will be processed as int aCall(); int bCall();
Macros with variable arguments use the gcc __VA_ARGS__ syntax.
$ ctags ... -D "DECLARE_FUNCTION(name,...)=int name(__VA_ARGS__);"
So the following code
DECLARE_FUNCTION(x,int a,int b) will be processed as int x(int a,int b);
Automatically expanding macros defined in the same input file (HIGHLY EXPERIMENTAL)
If a CPreProcessor macro defined in a C/C++/CUDA file, the macro invocation in the SAME file can be expanded with following options:
--param-CPreProcessor._expand=1 --fields-C=+{macrodef} --fields-C++=+{macrodef} --fields-CUDA=+{macrodef} --fields=+{signature}
Let’s see an example. input.c: .. code-block:: C #define DEFUN(NAME) int NAME (int x, int y) #define BEGIN { #define END } DEFUN(myfunc) BEGIN return -1 END The output without options: .. code-block:
$ ctags -o - input.c BEGIN input.c /^#define BEGIN /;" d language:C file: DEFUN input.c /^#define DEFUN(/;" d language:C file: END input.c /^#define END /;" d language:C file:
3.3. The new C/C++ parser 84 Universal Ctags Documentation, Release 0.3.0
The output with options: .. code-block:
$ ctags --param-CPreProcessor._expand=1 --fields-C=+'{macrodef}' --fields=+'
˓→{signature}' -o - input.c BEGIN input.c /^#define BEGIN /;" d language:C file:
˓→macrodef:{ DEFUN input.c /^#define DEFUN(/;" d language:C file:
˓→signature:(NAME) macrodef:int NAME (int x, int y) END input.c /^#define END /;" d language:C file: macrodef:} myfunc input.c /^DEFUN(myfunc)$/;" f language:C
˓→typeref:typename:int signature:(int x,int y) myfunc coded by DEFUN macro is captured well. This feature is highly experimental. At least three limitations are known. • This feature doesn’t understand #undef yet. Once a macro is defined, its invocation is always expanded even after the parser sees #undef for the macro in the same input file. • Macros are expanded incorrectly if the result of macro expansion includes the macro invocation again. • Currently, ctags can expand a macro invocation only if its definitions are in the same input file. ctags cannot expand a macro defined in the header file included from the current input file. Enabling this macro expansion feature makes the parsing speed about two times slower.
3.3.3 Incompatible Changes
The parser is mostly compatible with the old one. There are some minor incompatible changes which are described below.
Anonymous structure names
The old parser produced structure names in the form __anonN where N was a number starting at 1 in each file and increasing at each new structure. This caused collisions in symbol names when ctags was run on multiple files. In the new parser the anonymous structure names depend on the file name being processed and on the type of the structure itself. Collisions are far less likely (though not impossible as hash functions are unavoidably imperfect). Pitfall: the file name used for hashing includes the path as passed to the ctags executable. So the same file “seen” from different paths will produce different structure names. This is unavoidable and is up to the user to ensure that multiple ctags runs are started from a common directory root.
File scope
The file scope information is not 100% reliable. It never was. There are several cases in that compiler, linker or even source code tricks can “unhide” file scope symbols (for instance *.c files can be included into each other) and several other cases in that the limitation of the scope of a symbol to a single file simply cannot be determined with a single pass or without looking at a program as a whole. The new parser defines a simple policy for file scope association that tries to be as compatible as possible with the old parser and should reflect the most common usages. The policy is the following: • Namespaces are in file scope if declared inside a .c or .cpp file • Function prototypes are in file scope if declared inside a .c or .cpp file • K&R style function definitions are in file scope if declared static inside a .c file. • Function definitions appearing inside a namespace are in file scope only if declared static inside a .c or .cpp file. Note that this rule includes both global functions (global namespace) and class/struct/union members defined outside of the class/struct/union declaration.
3.3. The new C/C++ parser 85 Universal Ctags Documentation, Release 0.3.0
• Function definitions appearing inside a class/struct/union declaration are in file scope only if declared static inside a .cpp file • Function parameters are always in file scope • Local variables are always in file scope • Variables appearing inside a namespace are in file scope only if they are declared static inside a .c or .cpp file • Variables that are members of a class/struct/union are in file scope only if declared in a .c or .cpp file • Typedefs are in file scope if appearing inside a .c or .cpp file Most of these rules are debatable in one way or the other. Just keep in mind that this is not 100% reliable.
Inheritance information
The new parser does not strip template names from base classes. For a declaration like template
Typeref
The syntax of the typeref field (typeref:A:B) was designed with only struct/class/union/enum types in mind. Generic types don’t have A information and the keywords became entirely optional in C++: you just can’t tell. Furthermore, struct/class/union/enum types share the same namespace and their names can’t collide, so the A information is redundant for most purposes. To accommodate generic types and preserve some degree of backward compatibility the new parser uses struct/class/union/enum in place of A where such keyword can be inferred. Where the information is not available it uses the ‘typename’ keyword. Generally, you should ignore the information in field A and use only information in field B.
3.4 The new HTML parser
Maintainer Jiri Techet
3.4.1 Introduction
The old HTML parser was line-oriented based on regular expression matching. This brought several limitations like the inability of the parser to deal with tags spanning multiple lines and not respecting HTML comments. In addition, the speed of the parser depended on the number of regular expressions - the more tag types were ex- tracted, the more regular expressions were needed and the slower the parser became. Finally, parsing of embedded JavaScript was very limited, based on regular expressions and detecting only function declarations. The new parser is hand-written, using separated lexical analysis (dividing the input into tokens) and syntax anal- ysis. The parser has been profiled and optimized for speed so it is one of the fastest parsers in Universal Ctags. It handles HTML comments correctly and in addition to existing tags it extracts also