The Comprehensive TEX Archve Network (CTAN)
George D. Greenwade Department of Economics and Business Analysis College of Business Administration Sam Houston State University Huntsville, TX, USA 77341-21 18 Voice: (409) 294-1265 FAX: (409) 294-3612 Internet: bed-gdg@SHSU. edu
Abstract Ths paper outlines the concept, development, and use of the Comprehensive TEX Archive Network (CTAN) - a network-accessible archve for files related to the TEX family of document processing. The CTAN is a coordinated effort among consenting well-known archve sites which provides quick identification and retrieval files in a consistent manner from hosts on different continents, thereby reducing overall network load and increasing speed of retrieval. Moreover, it provides users with a parallel archive structure between hosts with holdings whch are generally synchronized to with 30 hours of one another. This is achieved by routinely mirroring one another's holdings, as well as mirroring other archves to maintain an up-to-date collection of files.
Why a Comprehensive TEX Archive 1992, with the author as its Chair.' While a Network? variety of issues related to archving have been discussed (and broader guidelines, per se, will in all Since the inception of publicly-accessible network- likelihood be forthcoming), the concept of creating a based archves, TEX and its related packages, macros, systematically coordinated family of network-based and utilities have been available for retrieval by archve sites was tacitly agreed upon as a mechanism users via any number of techniques. The combina- for verifying that the ideas under discussion were tion of the growth of the Internet in recent years, the workable. Also, ths approach was viewed as a growth of publicly-accessible network-based archive mechanism for creating a more efficient design, sites, and the growth in the number of files asso- both theoretically and practically, to meet the needs ciated with TEX and it's affiliated packages and of the worldwide TEX community. programs, created a rather overwhelming number of files for users with network connections to sort General Consensus Notes From WG-TAG. Ger- through. In terms of "overwhelming," the number mane to the development of a set of archves in lieu of files available has been a significant boost for of a "hard and fast" set of guidelines for propagation users; however, in these same terms, the number to other hosts were the following concerns: of different versions available, their precise location on a given archive host, the user interface avdable to access these files, and the ability to efficiently identify the various pieces required to make the var- I would like to take this opportunity to formally ious iterations of TEX and its relatives work properly recognize the members of ths Working Group has evolved to be a non-trivial task. and publicly thank them for their efforts on any In recognition of these problems, the then- number of topics whch we have dealt with. These newly-created Technical Council of the TEX Users individuals are, in alphabetical order: Nelson Beebe, Group formed a Techcal Working Group on TEX Barbara Beeton, Karl Berry, Johannes L. Braams, Archive Guidelines (officially WG-92-05; informally David Carlisle, Michael J. Ferguson, Alan J. Hoenig, referred to as TWG-TAG) in the latter months of Don Hosek, David M. Jones, Pierre MacKay, David Osborne, Philip Taylor, Jon Radel, Sebastian Rahtz, Rainer Schoepf, Joachlm Schrod, and Elizabeth Tachkawa.
TUGboat, Volume 14 (1993), No. 3 -Proceedings of the 1993 Annual Meeting The Comprehensive TEX Archve Network (CTAN)
Existing archive hosts very likely have chosen Genesis of the Design and utilize a structure which is politically and Prior to defining the CTAN directory structure, ex- practically acceptable for their site. tensive discussion was undertaken by TWG-TAG While the concept of the Internet's File Trans- regarding the optimal directory hierarchy. Com- fer Protocol (ftp) for a user interface was on pared for use were the herarchical design used everyone's mind, access via alternate means on most ftp-accessible hosts now possessing a TEX (primarily electronic mail and hard media) had archive and the flat design used by a number of to be considered. mail-oriented hosts. For brevity's sake, allow it Very few hosts on the network possess a to be said that the hierarchcal design was cho- comprehensive archve of TEX-relatedmaterials; sen. Once this decision was made, comparisons thus, a design guideline whch includes all of existing archves were undertaken, primarily dimensions of TEX may not be proper for a focusing on the holdings of three major sites- specialized archive. Stuttgart University's ftp.uni -stuttgart. de, As- Even if a set of guidelines were developed, ton University's TeX. ac . u k, and Claremont Col- there is no way to ensure that every site lege's ymi r . cl aremont edu. The end result is which possesses some aspect of TEX in its . a hybrid of the three directory structures, focus- archve would follow them since archves are a ing on top-level directories whch are somewhat function of local resources, support, and needs mnemonically-based directory names at ths level more than "network" demands. which, at least in the author's view, is pleasing. If a workable demonstration of the guidelines The structure is adequately diverse so that dvi- existed, more sites may voluntarily elect to related files (such as device drivers, dvi to output follow the ultimate guidelines. programs, and dvi viewers) are distinctly different No single site on the network possessed a from macros, and that macros are adequately cate- canonical listing, much less collection, of the gorized into the appropriate flavor of TEX for which latest relevant files available; therefore, any they are intended. The top-level directory herarchy guidelines developed would be hypothetical is presented in Appendix A. more than worlung. Conceptually, every file in the archve fits into It is necessary to make the structure as flexible one branch of ths directory herarchy (albeit the as possible, while at the same time ensuring directory mutually agreed to by the maintainers that files may be easily located by users via of the hosts involved in this project). Where a some consistent and logical design. file conceptually fits into more than one directory, As much as foreseeably possible, the structure efforts are made to ensure that the file properly should be extensible into evolving network appears where it should. retrieval and browsing technologies, such as Gopher, Mosaic, and other developing utilities. Coincidences of Consequence. A critical dimen- It is essential that the archives support users sion of ths project was its timing. Two hosts were from the variety of platforms under which TEX new machines; therefore, they were easily designed is available; while a given platform may be into whatever structure was agreed to. The main used for the archives themselves, it should not coincidences whch aided the success of ths project impose problems on any other platform which were: a user may wish to ultimately use the files on. Sebastian Rahtz, who was just beginning to At least initially, the concern was to provide put together the Aston "daughter" archve at a reliable archving directory herarchy from ftp . tex. ac. uk, began utilizing the directory with the project, as opposed to a production design by following the very rough outline of system herar~hy.~ the preliminary structure. Malung the struc- ture operational was a significant factor in more than a few subsequent decisions as it illustrated, in a close-to-production environ- ment, the strengths and weaknesses of the Please note that the concerns above are those then-tentative hierarchy. perceived by the author as the majority opinion and Sam Houston State University was just in- are not necessarily those of the TWG-TAG, nor are stalling its first Unix-based host and learn- they necessarily those of any one given member of ing its ihosyncrasies; however, its use as an the TWG-TAG.
TUGboat, Volume 14 (1993),No. 3 -Proceedings of the 1993 Annual Meeting George D. Greenwade
archive host was established soon after its in- disjoint and unrelated projects coming to fruition stallation. This was a preferable choice for an at approximately the same time, this project would archive host over SHSU's more established ftp likely still be in its planning stages. host, Niord. SHSU .edu, as it ran a compara- ble software to the other two hosts, and was The CTAN Hosts configurable for use as a mirror. David M. Jones had just released the first ver- By now, you're very likely aslung where these hosts sion of his index of TEX and ETEX macros, are on the network. The present CTAN hosts and providing the TWG-T.4G with a relatively com- their root CTAN directories are: prehensive listing of files, their authoritative ftp. uni -stuttgart. de /pub/tex/ location, current version, and other critical ftp.tex.ac.uk /pub/archive/ information. ftp.shsu.edu /tex-archive/ Joachirn Schrod's mirroring software for Unix Based at Stuttgart University germ an^),^ Aston Uni- hosts (which the CTAN hosts utilize) was being versity (EnglandL4and Sam Houston State University upgraded. Joachim was able to quickly address (United state^),^ respectively. a few specific problems resulting from its Whde the designs of the consulted archives massive use. were largely congruent, there were some modifi- Nelson Beebe was developing a few new ex- cations when compared to all existing archives. tensions to the ftp server software which was Rainier Schoepf, TEX archve manager at Stuttgart's adopted for use. Significant contributions were ftp. uni -stuttgart . de, was fully agreeable to his definitions which allow users to retrieve modifying the directory structure there based on recursive ZIP and ZOO archves of directory the recommendations which were developed. While holdings. it was clear that the changes would impose short A bug was located in the adopted ftp server run problems for users of his collection and po- software. This led to the new release of the ftp tential problems in the management of his site's server software, resulting in a much nicer in- mirroring of packages, he was in agreement that terface, easier management, and more generic the long term benefits to the TEX community of a installation as compared to prior versions. In well-known and specified structure outweighed any essence, while each host is unique in archtec- short term impact. Also, as the other two hosts ture, the configurations are virtually parallel, as involved were new to the network, being able to are most features available to users. plan for the inclusion of an existing large and active Karl Berry further extended hssomewhat stan- archive from the start was a real benefit, especially dardized Unix TEX directories and installation. since it had a mirroring structure already in place WMe this has yet to play a significant role in and the site was mirrorable by other sites. design, it provides significant future extensions to the services of the CTAN. The University of Minnesota upgraded the facil- In addition to the administration of the insti- ities and functions of its Gopher software. This tution, thanks are extended to DANTE, the German interface now serves as an alternative access speaking TEX Users Group, for their archival support method to the CTAN at Aston and SHSU. and leadership role in maintaining this archve. Significant enhancements to the ZIP utilities by In addition to the administration of the institu- the Info-ZIP team were released. Ths is a major tion, thanks are extended to the UK TEX Users Group item of concern as it allows for a platform- for their archival support and leadership role in the independent mechanism for the archval of development of their original archive, as well as in files held within the CTAN collection. In the creation and development of ths new archve essence, when combined with Nelson Beebe's host. extensions, this feature allows users on virtu- In addition to the administration of the insti- ally any platform to retrieve recursive directory tution, thanks are extended to SHSU's Computer holdings, then convert them to the operating Services Division, which has been more than forth- system specific needs which they confront. coming in support and access as this archive has By no means is this a complete listing of evolved. Additionally (and especially), thanks are all coincidences of consequence to the project - extended to SHSU's College of Business Administra- suffice it to say that without a number of apparently tion for its continued interest in and support of ths project.
TUGboat, Volume 14 (1993),No. 3 -Proceedings of the 1993 Annual Meeting The Comprehensive TEX Archve Network (CTAN)
What's Available ftp. The ftp interface utilized by each CTW host is the "wuarchve ftpd" program.' In addition to As noted in the name selected for these hosts, the the normal ftp commands a user would expect, Comprehensive TEX Archive Network, the collections these hosts support a variety of additional useful available are about as comprehensive and timely functions, such as locating a file, creating ZIP, as humanly possible. Do these hosts have "every- ZOO, and Unix tar archives on-the-fly, automatic thing"? The truthful answer is that they probably compression of files on-the-fly, and a few other do not and probably never will- but they are and features. As these features may be modified, the will be about as comprehensive a collection as can best policy is to consult any files named README in be arranged. As sites possessing relevant files and the default directory at login. packages are identified, one of the CTAN hosts will include them in its mirroring passes; from there, Gopher. Two of the CTAN hosts, ftp. tex. ac. uk the files will then propagate to the other CTAN and ftp. shsu. edu are configured for Gopher ac- hosts. Presently, there are about 790 megabytes of cess. The Internet Gopher, developed by the Univer- files available in the /tex-archive/ directory tree sity of Minnesota, provides users with a menu-driven on ftp. shsu. edu. It would be expected that the interface for transactions. Users may view and re- collections in the TEX archive area on the other two trieve files in a fashion somewhat more friendly hosts would be virtually identical. than ftp. Also, the Gopher menus presented do not necessarily represent files actually available at a How It Is Achieved. The synchronization between site; instead, the client/server relationshp utilized hosts is handled within a "mirroring" program allows for servers to point to information stored whereby files on one host are propagated to the elsewhere, whch users are automatically connected other hosts of the CTAN. The variant of mirroring to by simply selecting a menu entry.8 used by the CTAN hosts is a modification of a family Pointing the Gopher client directly to the Go- of per1 scripts written by Joachim Schr~d.~ pher server on ftp. tex .ac . uk is the preferred Conceptually, the process entails three major method for access to the Aston CTAN archve; Go- points. First, hosts where files are authoritatively phering to SHSU's "front door" Gopher server on available for retrieval are mirrored inbound by a Ni ord. SHSU . edu is the preferred method to access selected CTAN host. This CTAN host locates these SHSU's CTAN archive. In addition to the archves files in a pre-specified area of the collection. Second, themselves, a variety of TEX-related files and ser- users may contribute files to a given CTAN host by vices are available on each host. Sample menus uploading their files into an "incoming" area on from each are provided in Appendix B. any given CTAN host. From there, the local CTAN administrator moves these contributions into the Mail. At the moment, no agreed-to mail interface CTAN hierarchy. Finally, the CTAN hosts mirror one has been installed on all hosts. The ftpmail another on a routine basis to collect all new files program will be used and ths will be documented whch have entered the collection at the mirrored better in a future article scheduled for TEX and TUG host. In all, about one gigabit of files are referenced News discussing user interfaces to the CTAN hosts.g daily by the mirroring processes in order to maintain Hard Copy. Sebastian Rahtz has already made ar- accuracy, timeliness, and correctness of holdings. rangements and compiled a hard copy of the CTAN archves for CD-ROM distribution. The distribution Existing User Interfaces As noted throughout this paper, the Internet's File ' Ths program is available in the directory Transfer Protocol (ftp) is supported at the moment archive-tool s/ftpd/mi rror/ on the CTAN hosts. for a user interface. There are three other interfaces It is the classic ftp server/daemon utilized by Wash- which should be recognized - Gopher, mail, and ington University in St. Louis, Missouri (USA) for hard copy - as these are important to users which their massive collection of files. prefer to utilize them. Gopher clients are available for a wide variety of platforms. Sources for most platforms are available in the directory archive-tools/gopher/ on the CTAN hosts. Sources for the ftpmail service are available The program itself is available in the archive- in the directory archive-tool s/ftpmai 1/ on the tool s/mi rror/ directory of the CTAN hosts. CTAN hosts.
TUGboat, Volume 14 (1993),No. 3 - Proceedings of the 1993 Annual Meeting George D. Greenwade
is available from Prime Time Freeware. The first Directory Suggestions. Although it is well-specified, version of thls file set is already available and it is no official designation of the CTAN directory struc- to be routinely updated to reflect changes made to ture now exists. However, it is advisable for the archives. maintainers of authoritative files and packages to consider the utilization of the CTAN hierarchy, if at Aspects for Authors all possible. In this manner, the author will be able to envision how the archive is laid out for the end While the process of mirroring and uploading pro- user. Also, if this structure is used, especially on vides a rough and ready mechanism for acheving Unix-based hosts, proper hkage to related files may the reliability of the CTAN, authors should be aware be used. At the moment, one of the most wasteful of a few dimensions. At the moment, no specific uses of the CTAN directory contents are the mul- guidelines exist on these topics, so please view the tiple copies of files whch authors elect to include. following as personal views which will, in time, be For example, as of 1 July 1993, approximately 27 discussed. copies of the LATg macros exist in the archve- File Headers and Checksurns. Nelson Beebe has only one of whch (in macros/l atex/di stribs/) is developed a package written as an Emacs Lisp file authoritative. With linkmg, the mirror simply calls which creates "standardized" file headers.1° These in the link and the latest versions of related files headers, while clearly consuming space with the (from their well-specified directory) dlbe delivered archve, provide valuable information which users to users. and archive maintainers may refer to quickly. More- over, these file headers provide valuable indexing Getting a File into the CTAN Without information which may be blended into existing Mirroring indices, such as David Jones' heroic effort. An example file header taken from a contributed LATEX If mirroring is not practical (either because of style option is provided in Appendix C. inability to mirror a host, lack of a public ftp area Optionally included in these headers is a in which to place a file, lack of ftp altogether, "checksum" option, whch is of significant bene- the contribution is a small number of files, or fit whenever files are transferred electronically to any other reason), authors can still easily get files verify correctness. The preferred is Robert Solovay's included into the CTAN with electronic mail or via CWEB-based program which, with the assistance of anonymous ftp. Nelson Beebe, has been ported to a number of Electronic Mail. To submit a contribution via elec- operating systems.ll tronic mail, use the address: Location of Files. It would behoove authors who [email protected] maintain authoritative file set(s) on network hosts or which are ftp-accessible to contact me so that [email protected] arrangements may be made to mirror these files into the CTAN collection. Without a doubt, the including the file in whatever manner is feasible ability to mirror files into the CTAN hierarchy is the (encoding of executables, splitting of files, indicating least painful and most efficient method available as within a single post with multiple files where to human intervention arises mainly in the review of "cut", etc.). In these cases, checksums are ideal the logs of the mirroring session. If an authoritative as it provides verification that the file received for host is identified, its holdings will automatically archving purposes is indeed the same file whch be included when the site is referenced. In this the author intended for inclusion. A brief note way, additional copying to additional hosts for describing the contribution is very appreciated. As propagation wdl not be needed. Moreover, you soon as the contribution has been processed into wdl ensure that the latest versions are available for the CTAN, the submitting author will be notified via public consumption. return electronic mail. Anonymous ftp. To submit a contribution via ftp, connect to one of the CTAN hosts with an This package is available in the archive- ftp client. When prompted for a username, type tool s/fi 1ehdr/ directory of the CTAN hosts. "anonymous" (all lowercase; without quotes) and l1 This package is available in the archive- use your complete electronic mail address as your tool s/checksum/ directory of the CTAN hosts.
346 TUGboat, Volume 14 (1993),No. 3 -Proceedings of the 1993 Annual Meeting The Comprehensive TEX Archve Network (CTAN) password. The Aston and SHSU CTAN hosts sup- 2. The use of a non-ftp mirroring mechanism. At port the /i ncomi ng/ directory for contributions present, the mirroring mechanism used by the CTAN (uploads) of files; the Stuttgart CTAN host supports hosts suffice for its use. However, ftp is not an the /soft/i ncomi ng/tex/ directory for contribu- efficient mechanism for moving the quantity of files tions of TEX-related files. Once connected, a typical involved in this project. A much preferable solution upload session would look like: would be the use of some alternate protocol whch cd
TUGboat, Volume 14 (1993),No. 3 -Proceedings of the 1993 Annual Meeting George D. Greenwade
Appendix A
The Top-Level Directory Hierarchy Once into the CTAN host's root area, the following directory herarchy is presented: archi ve-tool s/ bibliography/ digests/ documentati on/ dvi ware/ fonts/ graphi cs/ he1p/ indexing/ 1anguages/ 1ocal/ macros/ misc/ support/ systems/ web/
In brief, the contents of these directories include: archive-tool s/ contains the various archiving tools which users may find useful. bibl i ography/ contains bibliography-related files, such as B~BTEX. digests/ contains back issues of TEX-relatedperiodicals. documentation/ contains files and tutorials which document various aspect of TEX. dviware/ contains the various dvi-to-whatever filters and drivers. fonts/ contains a collection of fonts, both sources and pre-compiled. graphics/ contains utilities and macros related to graphics. he1 p/ contains files which provide an overview to the archwe and the TEX system. indexi ng/ contains utilities and related files for indexing documents. 1anguages/ contains non-English related implementations of TEX. 1ocal / contains local site-specific files -not of general interest. macros/ contains macros for TEX and its derivatives in unique subdirectories. misc/ contains files and programs which cannot otherwise be catalogued. support/ contains files and programs which can be used in support of TEX. systems/ contains complete system setups, organized by operating system. web/ contains WEB-related files and utilities.
TUGboat, Volume 14 (1993), No. 3 -Proceedings of the 1993 Annual Meeting The Comprehensive TEX Archve Network (CTAN)
Appendix B
Gopher Menus from Aston and SHSU The menus for the CTAN archives via Gopher as of the time of ths paper are provided below. For ftp. tex. ac. uk the Gopher menu structure appears as:
The menu's for the CTAN archives via Gopher as of the time of this paper Root gopher server: ftp.tex.ac.uk
1. UK TeX Archive/ 2. Aston University/ 3. Minnesota Gopher root/ 4. Archaeology/ 5. Archie 6. Font Samples/ 7. General WAIS databases/ 8. Looking things at AMS
1. We1 come. 2. Archive di rectory/ 3. Indexed FTP (Directories names only) of UK TeX Archive. 4. Indexed FTP of UK TeX Archive. 5. Indexed access to archive hierarchy 6. Other Archives/ 7. TeX Font Samples/ 8. UK-TeX. 9. WAIS database: TeX --- list of FTP sites with TeX material 10. WAIS database: TeX Frequent1y Asked Questions (UK specifi c) 11. WAIS database: TeX index of styles and macros (by David Jones) 12. WAIS database: The TeX Book 13. WAIS database: back issues of TeXhax, 1986 -- 14. WAIS database: back issues of UKTeX newsletter
Selecting item 2. from this menu yields the directory hierarchy described in Appenduc A.
TUGboat, Volume 14 (1993), No. 3 -Proceedings of the 1993 Annual Meeting George D. Greenwade
For Niord .SHSU . edu the Gopher menu structure appears as: Root gopher server: Ni ord. SHSU. edu
1. About the Sam Houston State University Gopher. 2. Customizing the Print command at SHSU. 3. All the Gopher Servers in the World/ 4. Chronicle of Higher Education "EVENTS in ACADEMEm/ 5. Economics (SHSU Network Access Initiative Project)/ 6. Internet Information/ 7. LaTeX3 Publ ic Document Library/ 8. Libraries, Periodicals, References, etc./ 9. Minnesota Gopher (Mama Gopher; get "your ownq clients here!)/ 10. SAMINFO -- Sam Houston State University Information System
1. Comprehensive TeX Archive Network (CTAN) at SHSU/ 2. UK TeX Archive/ 3. Archives of INFO-TeX/ctt-Di gest (comp. text. tex)/ 4. EconBi b (LaTeX/Bi bTeX sty1es for economi cs)/ 5. ;-c . * . FAQs , REFERENCE and PRIMERS . ?:. ;':. 6. FAQ for comp. text. tex (text and WAIS indexed)/ 7. FAQ Supplement for comp.text. tex (text and WAIS indexed)/ 8. TeX-Index (text and WAIS indexed)/ 9. FAQ for comp.fonts (text and WAIS indexed)/ 10. The Canonical list of MetaFont fonts. 11. A Gentle Introduction to TeX 12. Components of TeX/ 13. Essential LaTeX/ 14. MetaFont for Beginners. 15. NFSS in the Context of LaTeX 16. ?:. *. RELATED FILES .*.;':. 17. LaTeX3 Publ ic Document Library/ 18. Literate Programming Library/
Selecting item 1. from this menu yields the directory herarchy described in Appendix A.
TUGboat, Volume 14 (1993), No. 3 -Proceedings of the 1993 Annual Meeting The Comprehensive TEX Archve Network (CTAN)
Appendix C
000XXX QLaTeX-style-fi 1e{ 000XhX fi1 ename = "showkeys.sty", %%% version = "1.01", %%% date = "25 August 1992", %%% time = "11:32:08 BST", 00 0 XXX author = "David Carl isl en, %%% address = "Computer Science Department %%% Manchester University 000XX /6 Oxford Road 0XXX0 0 Manchester 0XXX0 0 England %%% M13 9PL", %%% telephone = "+44 61 275 6139", %%% FAX = "+44 61 275 6236", %%% checksum = "61501 431 1786 14304", 0 00 XXX emai 1 = "[email protected] (Internet)", 0XXX0 0 codetabl e = "ISO/ASCII", 0hXX0 0 keywords = "LaTeX, label, ref, citation, keys", %%% supported = "yes", %%% docstri ng - %%% %%% 0XXX0 0 0XXX0 0 A LaTeX style option which causes 0XXX0 0 \label , \ref, \pageref, \cite and \bi bitem %%% to print their argument for proof reading purposes. The main %%% feature of this style is that these labels are printed in such a 000XXX way as to minimise the changes caused to the formatting of the 000XhX rest of the document text. %%% 0XXX 00 Documentation requi res Mitrel bach's doc. sty. %%% %%% The checksum field above was produced by 0XXX 00 Robert Solovay's checksum utility.", 000XA 3
TUGboat, Volume 14 (1993),No. 3 -Proceedings of the 1993 Annual Meeting