Managing Multiple TDS Trees Michael J Downes American Mathematical Society, Providence, Rhode Island 02904, USA [email protected] Abstract The TDS specification for TEX directory structure provides a good framework for organizing a single TEXMF tree such as the one at the heart of Thomas Esser’s teTEX distribution. In practice, however, because of the normal processes of local changes and ongoing upgrades it is often advantageous to set up a more elaborate system of multiple TDS trees. This permits organizing the pieces that have been added above and beyond the original base tree in a way that makes it easier to cope with continuing change. I could have more accurately indicated the scope of this article by using a longer title: Managing multiple TDS trees in a web2c-based TEX system. But it should be fairly clear how to apply the general principles discussed here to other kinds of TEX systems with similar file searching capabilities. Introduction example, this could map to a logical drive specification such as ‘T:’. T X Directory Structure After T X had been E E ... in use for some years people started to notice that ... the set of files needed by T X when processing E A site may choose to have more documents was growing rather large and unwieldy. than one TDS hierarchy installed (for The simplest way for T X to look for input files was E example, when installing an upgrade). to keep them all in a single directory. But this made maintenance difficult. In fact, my experience suggests that in most In response a volunteer committee analyzed the cases it is useful to set up quite a few trees, branch- categories of files involved and drew up a specifi- ing freely whenever you find a key difference in usage or upgrade policy, as long as your T X system can cation known as “TDS”: TEX Directory Structure. E (See http://tug.org/tds.) support it. For example, consider this quote from a The TDS spec gives a recommended hierarchical recent message on the teTEX mail list: structure for collections of TEX-related files in a (We’re even using one and the same generic form that is intended to be usable and com- nfs archive for three platforms, patible across the various kinds of computers in com- Solaris, Linux and W2k. Only trouble mon use nowadays: PC/Windows, Macintosh, Unix, is you cannot update executables and so forth. A TDS tree is a tree of directories for one platform without putting with each major subcategory of files residing in its the *@:format files in special local own subtree — fonts, macros, and documentation, to places.) name a few. A solution for such a problem would be to make But a TDS tree has everything branching from a separate, very sparse TDS tree that only holds a single root. In practice it may often be desirable format files (and any other files that are tied to to set up more than one TDS tree, as mentioned in specific binaries). I would probably put the binaries the TDS documentation: in the same tree, if that could be done without ... we shall designate the root causing extra work elsewhere. TDS directory by ‘texmf’ (for "TeX TEX systems: web2c, teTEX, MiKTEX, etc. and METAFONT"). We recommend using Installing a TEX system The TEX Users Group that name where possible, but the has a good listing of TEX systems for various plat- actual name of the directory is up to forms at http://www.tug.org/interest.html.I the installer. On PC networks, for am going to place my discussion of TDS trees within the framework of a TEX system called teTEX, which 228 TUGboat, Volume 22 (2001), No. 3 — Proceedings of the 2001 Annual Meeting Managing Multiple TDS Trees is layered in turn on top of another system called to automatically find the .dvi file and start up a web2c. One of the most important elements in viewer (probably xdvi). web2c is a library called kpathsea, designed as a When I tried texdoc teTeX-FAQ, however, it tool for other programs to use when they need to didn’t work. You may have better luck. My attempt efficiently search a tree containing a large collection was with teTEX 1.0. of files, and in particular to provide ways for users to Recursive directory searching For programs specify search rules that reflect the needs of a given whose file searching is based on the kpathsea library, application. Thus, the versions of TEX, METAFONT, dvips, bibtex, and other programs included in the • A ‘path’ is the name of a directory (aka folder), web2c distribution are all adapted to use the kpath- which may have several components in its sea library. name, one for each level. The use of the kpathsea library is the critical • In the texmf.cnf file the generic separator char factor that determines the exact syntax and configu- between components of a path is slash /. ration methods one uses when making arrangements • In a list of multiple paths, the generic separator to have TEX search multiple TDS trees. Other TEX char between paths is semicolon ;. (On Unix systems such as MiKTEX or TrueTEX that support systems you would normally see colon : here.) recursive searching through a TDS tree differ in • Trailing // on a directory name means to search various details, but many of the general principles subdirectories (recursively). will be the same. • A leading !! means to look only in the ls-R The web2c guys, notably Karl Berry & Olaf database, never on the disk. Weber have pulled the source code for the programs • A leading/trailing/doubled ; in the paths will TEX, METAFONT, dvips, xdvi, into a coherent col- be expanded into the compile-time default lection that compiles easily and plugs in the kpath- path. sea path searching library where applicable. (Thank you, Karl and Olaf, for all your hard work!) Use of filename DB to speed up file searches The teTEX guys, notably Thomas Esser, have The kpathsea library supports the use of a built a big standard collection that includes web2c “filename database” to speed up file searches. (This as a base, but (a) is designed to simplify installation database is the referent of texhash, mktexlsr, ls-R for ordinary users who don’t necessarily want to files and so on.) It is a simple lookup table that have all the sources and don’t plan to make a gives the location of each file in a TDS tree. practice of recompiling things; and (b) includes a In practice, using the DB seems to cause users rather comprehensive set of commonly needed run- a lot of trouble because it is easy to forget to time files, which makes the programs easier to use update the DB after adding files. Because most after they are installed. (Thank you, Thomas, for systems nowadays are fast enough that the search all your hard work!) time is negligible for even a fairly large collection of files, I would recommend in most cases to freeze Documentation the TEXMFMAIN tree and use the DB lookup for Much of the most useful documentation for web2c TEXMFMAIN but not for any other trees. Whether and kpathsea is in “Info” form, which can be read this recommendation is a good idea in your situation with Emacs or with the standalone info application. depends on the following general principles: I found the following docs especially useful: 1. Use the DB for TEXMF trees that don’t change. web2c: web2c.info 2. Use the DB for TEXMF trees where file access kpathsea: kpathsea.info is slow (e.g., across a slow network). dvips: dvips.info For example, consider the Usenet post shown in xdvi: man page for xdvik Figure 1. teTEX: See TETEXDOC.* and teTeX-FAQ: Using an alternate texmf.cnf file If you want to TEXMF/doc/tetex/teTeX-FAQ specify a different arrangement of TEXMF trees than TEXMF/doc/tetex/TETEXDOC.dvi the one in the out-of-box cnf file (and in practice this is almost always going to be the case), you have two If your T X system is teT X, you can use the E E choices: command 1. Edit the texmf.cnf file in the TEXMFMAIN tree texdoc TETEXDOC TUGboat, Volume 22 (2001), No. 3 — Proceedings of the 2001 Annual Meeting 229 Michael J Downes From: Peter Neergaard <[email protected]> Subject: Re: Why does miktex need to refresh data base Newsgroups: comp.text.tex Date: 16 Jul 2001 12:24:59 -0400 On July 16, 2001, Robin Fairbairns wrote: RF> consider the following example from my texmf.cnf file. RF> TEXMF = {$HOMETEXMF,!!$TEXMFLOCAL,!!$TEXMFMAIN,!!$TEXMFEXTRA} RF> the !! indicators mean that ls-R search is obligatory. [...] In some cases you might furthermore need to set TEXMFDBS which specifies where tetex looks for ls-R files. On my work, I am actually troubled by the fact that most of the file systems are NFS mounted. I did not get a decent speed of file searching before I started building ls-R files for my private hierachies. Figure 1: Some discussion of file searching speed 2. Set the environment variable TEXMFCNF to file-database lookup only for the MAIN tree, which point to a different location (just give the di- is kept frozen. The distinction between public and rectory name, you don’t need to give the full private is simply this: Put a file into the PRIVATE file name) tree if it is not publicly released somewhere on the Obviously, if you do not have privileges to change World-Wide Web for all to use.