TUGboat, Volume 34 (2013), No. 3 263
How to make a product catalogue that to produce in this way. So in 2004 I set out to create doesn’t look like a dissertation a more robust system to produce it. My goals were to create a system that would Jason Lewis produce a product catalogue automatically from our Abstract database, wouldn’t require long and detailed proof reading to ensure pricing was correct, be something I Needing a robust way to produce a catalogue from could delegate to staff for catalogue production, and a product database, Adobe InDesign, DocBook and that the staff need not have any special technical LAT X were evaluated. LAT X was favoured due to its E E knowledge to use the system. layout flexibility and non-proprietary nature. The challenge was to query the database and produce 2.1 Tools I looked at A suitable LTEX for the catalogue First, InDesign: Native LATEX was unable to query the database, so templating tools were investigated. Template • InDesign at the time had limited scripting ability. Toolkit (TT) was chosen over PHP, favouring its Perl This has improved now and you can write scripts roots and broader applicability. Using TT and DBI for in Microsoft Visual Basic or JavaScript. • no proper database connectivity at the time. querying the database a dynamically generated LATEX document can be quickly constructed. TT filters are There are now numerous commercial tools that developed to check and correct user supplied content allow you to link InDesign to a database. • proprietary software (I prefer to use free tools if for constructs that would cause the LATEX compiler to fail. Filters are also used to sanitise Windows file possible). paths for use in LATEX. Second, DocBook [1]: Styling the document to look like a product cat- • limited formatting and layout capabilities alogue was achieved using sans serif fonts, flowfram • recommends using LATEX for advanced layouts thumb tabs, colourful chapter and section headings • no database connectivity developed using tikz, long tables that allow page • plain text, easy to script using a template tool breaks, alternating row colours, wrapping of para- Third, LATEX: graph text around images and the highlighting of • flexible layout capabilities new products. Full page PDFs are included in the • free and open source document for the covers, front matter and adverts. • no database connectivity The result is system to generate a product cata- • plain text, easy to script using a template tool logue, quickly and easily directly from our database using free and open source tools. This has reduced 3 The challenge the workload in producing a catalogue and increased 3.1 How to script LAT X? staff productivity and efficiency. E I could have written the system in native LATEX but 1 Introduction LATEX has no way to retrieve data from a database. The other option was to choose a tool like PHP or I produce an 80 page full colour product catalogue Template Toolkit (TT). in LATEX. This paper outlines the tools I used to implement the system, link it to our database and 3.2 PHP style the catalogue for printing. Pros: PHP is widely used. We print a new catalogue every six months and it has approximately 1000 products, 16 full page Cons: geared towards HTML/web; it’s PHP; it’s colour adverts and takes less than two minutes to not Perl. build from the command line. 3.3 Template Toolkit 2 Why did I write this? Pros: I am part owner of a small wholesale distribution • Generalised to templating anything business in Australia. When we established the busi- • not PHP ness in 2001, we had approximately 40 products in • written in Perl our catalogue, for which a one page price list and • Template::Plugin::Latex can output PDF order form in Microsoft Excel sufficed. directly; can build manually for demonstration. As we grew the business and added more prod- Astute readers might notice a slight bias towards ucts to our portfolio, the catalogue became unwieldy Perl. This was mainly due to having prior knowl-
How to make a product catalogue that doesn’t look like a dissertation 264 TUGboat, Volume 34 (2013), No. 3
edge of Perl and meant I didn’t have to learn a new delimiter above strips the final newline from that line programming language. upon output; similarly, a minus sign after an opening delimiter (as below) removes a leading (preceding) 3.4 Database driven documents newline. Template Toolkit uses Perl’s DBI for database access, 4.3 Read a CSV file which means it can retrieve data from just about anything. Our data is stored in Microsoft SQL and Let’s suppose we have this table of data: Microsoft Access. I use DBI Proxy [2] to connect to the MS Access database. Table 1: simple-example.csv FirstName LastName FavouriteNumber 3.5 Compiling reliably Building the catalogue from source is a two step pro- Jason Lewis 34 cess: running Template Toolkit, and then passing its Joe Blogs 2 output through pdflatex. I created a Makefile to do Frank Sinatra 88 this, but often the document would require multiple runs to ensure all the cross-references were correct. A Here is TT code to parse it, using DBI to access tool such as latexmk [3] or rubber [4] helps with this the file as a database in the current directory: by providing a single command that will repeatedly [%- USE db = DBI( run pdflatex until all cross-references are stable. database => "DBI:CSV:f_dir=." ) -%] [%- FOREACH item = db.query( 3.6 User interface for entering data "SELECT * from simple-example.csv") -%] to the catalogue I needed a user interface for data entry. I chose MS FirstName: [% item.firstname %] \ Access as being easy to script, it was a familiar user LastName: [% item.lastname %] \ Favourite: [% item.favouritenumber %] interface for the staff, and I already owned it. The [% END %] downside is that of course it is a proprietary tool. Note that the column headings of table1 were sani- 4 Template Toolkit Primer tised to lower case by DBI::CSV. Here’s how I scripted LATEX using Template Toolkit. 4.4 Write your own parser 4.1 Install Template Toolkit Well not quite, but Template Toolkit makes it easy TT Cpanminus [5] is a great tool to retrieve, unpack, to implement a parser: build and install Perl modules from CPAN: #!/usr/bin/env perl use Template; cpanm Template die "no TT filename given" if (@ARGV != 1); cpanm DBD::CSV my $tt = Template->new({ cpanm DBI INCLUDE_PATH => ’.’, cpanm Template::Plugin::DBI INTERPOLATE => 1, Now we have Template Toolkit, DBI, and some ancil- }) || die "$Template::ERROR\n"; lary modules installed, and can try writing a simple Hello World. my $input = $ARGV[0];
4.2 Hello World # process input template, substituting variables: tpage is a simple script supplied by TT that parses $tt->process($input, $vars) || die $template->error(); a template file supplied on the command line and outputs it to standard output. Given this two-line This creates a TT object and parses the file whose template file: name is supplied on the command line. So far, this just provides the same functionality as tpage. [% str = ’Hello TUG2013’ -%] [% str %] 4.5 Build your LATEX document We can parse and output it like this: Here we write our LATEX document, including the $ tpage helloworld.tt TT lines to read from the database. Hello TUG2013 [%- USE db = DBI( database => Anything within [% %] will be treated as TT code "DBI:CSV:f_dir=." ) -%] and executed. The minus sign before the closing \documentclass{article}
Jason Lewis TUGboat, Volume 34 (2013), No. 3 265
\usepackage[utf8]{inputenc} my $tt = Template->new({ FILTERS => {latex_filter => \&latex_filter}, \title{build-document example} INCLUDE_PATH => ’.’, \author{Jason Lewis} INTERPOLATE => 1, \date{October 2013} }) || die "$Template::ERROR\n";
\begin{document} 5.2.1 More things that need to be filtered [%- FOREACH item = db.query( Then adding more filters is simply a matter of adding "SELECT * from simple-example.csv") -%] a regular expression to search for it and replace it First Name: [% item.firstname %] with whatever is appropriate. Some examples. \\ Last Name: [% item.lastname %] \\ Favourite Number: [% item.favouritenumber %] Degree symbol: \\ \newline $return =~ s/°/\\degree /g; [% END #FOREACH -%] Accented characters: \’{e} \end{document} Transform é into : $return =~ s/é/\\’{e}/g; Output (where mytpage is the script we just saw): Incorrect quotes: Convert beginning-of-line right $ ./mytpage build-document.tt or double quotes to left quotes, and use ASCII apos- \documentclass{article} \usepackage[utf8]{inputenc} trophes. $return =~ s/(^|\s)’(.*?)/$1‘$2/g; \title{build-document example} $return =~ s/(^|\s)"(.*?)/$1‘‘$2/g; \author{Jason Lewis} # replace Windows quote (octal 0222) with ASCII ’ \date{October 2013} $return =~ s/\222/’/g; En dash between numbers: Convert minus sign \begin{document} to an en-dash by searching for single minus sign First Name: Jason \\ Last Name: Lewis \\ Favourite Number: 34 \\ \newline between numbers and replacing it with two minus First Name: Joe \\ Last Name: Blogs signs: 1-10 vs. 1--10. \\ Favourite Number: 2 \\ \newline $return =~ s/(\d+)-(\d+)/$1--$2/g; First Name: Frank \\ Last Name: Sinatra A \\ Favourite Number: 88 \\ \newline 5.3 Pass through LTEX commands \end{document} From time to time I needed a way to pass LATEX commands through the filter. I chose an escape of 5 Technical problems
# escape $ with a backslash for LaTeX 5.4.1 LATEX does not like Windows paths $return =~ s/(\$)/\\$1/g; On our network, product images are stored on a return $return; server drive accessed via a Windows share w:\. In } MS Access, users select an image to go with a product
How to make a product catalogue that doesn’t look like a dissertation 266 TUGboat, Volume 34 (2013), No. 3
range and the path to the image is stored in the something that would shield the users from having database as a Windows path: to know LATEX in order to create and edit content w:\some\path to\an image.jpg for the catalogue. Users often use spaces, commas, apostrophes and 6 Make LATEX output look like a other abominable characters in the image paths, product catalogue and \includegraphics does not handle paths with Clearly I had to style the document so it would spaces in them. look more like a product catalogue and less like a The solution was to create a LAT X-friendly sym- E dissertation. bolic link to the file and include that instead. This was easy to achieve by making another TT filter. 6.1 Sans serif fonts 5.4.2 Convert Windows path to Unix The first thing I did was use a sans serif font. I chose Helvetica. The goal is to convert a Windows path such as \usepackage{helvet} w:\Alive & Radiant\2013-July-product.jpg % set the font to helvetica for body text to: \renewcommand{\familydefault}{\sfdefault} _mnt_Alive_&_Radiant_2013-July-product.jpg This provides URW Nimbus Sans which is a free clone [6] of Helvetica. 5.4.3 Build a filter Here is my Perl code: 6.2 Thumb tabs # convert all \ to / e.g. c:\ to c:/ Thumb-tabs are a nice feature for any catalogue. $return =~ s/([\\])/\//g; I created them using Nicola Talbot’s flowfram [7] # strip drive letter c:/path/file to path/file package, like this: $return =~ s/^\w:\///g; \setthumbtab{1}{backcolor=[rgb]{0.15,0.15,1}} # strip extension: /filename.jpg to /filename \setthumbtab{2}{backcolor=[rgb]{0.2,0.2,1}} #$return =~ s/\.\w\w\w$//g; \makethumbtabs[50mm]{30mm} # clean up the path $return = File::Spec::Unix->canonpath($return); \begin{document} 5.4.4 Make the symbolic link \tableofcontents \thumbtabindex Find spaces, / or % in filenames and replace with \enablethumbtabs underscores. Then use the resulting string as the \chapter{1ABC} name for a symbolic link to the original file. Use the \Blindtext symbolic link name in the LATEX document. \chapter{2ABC} \Blindtext # replace spaces, slashes, percents with _ \end{document} $safe_filename =~ s/[\s\/%]/_/g; # make a symbolic link to the file 6.3 Colourful chapter and section headings symlink($return, $safe_filename) || die; Colourful chapter and section headings were made 5.4.5 Use new filter on image paths using the tikz package. Use the new filter on image paths as they are re- \newcommand\colorchapter[1] trieved from the database: {\def\chapterbg{#1}\chapter} % begin CHAPTER format [% ImagePath \newcommand\boxedchapter[1]{{% = item.CategoryImagePath | path_filter %] \begin{tikzpicture}[inner sep=0pt, [% IF ImagePath != "" %] inner ysep=1.3ex] \includegraphics[width=12cm, % left position of text height=\imageheight, % right hand edge chapter title text keepaspectratio=true] \node[anchor=base west] {[%ImagePath%]} at (3,0) (counter) {}; [% END # if image exists %] \path let \p1 = (counter.base east) 5.5 User interface in node[anchor=base west, text width={\textwidth-\x1+26.33em}] (content) I needed to develop a user interface for staff to man- at ($(counter.base east)+(0.33em,0)$) age data in the catalogues. I chose MS Access as it {\textcolor{white} was easy to create and modify. I wanted to create {\Huge\sffamily\textsc{\thechapter \ \ #1}}};
Jason Lewis TUGboat, Volume 34 (2013), No. 3 267
\begin{pgfonlayer}{background} \includegraphics[width=12cm,\ \shade[left color=\chapterbg, height=\imageheight,\ right color=\chapterbg] keepaspectratio=true]\ let \p1=(counter.north),\p2=(content.north) {[%ImagePath%]} in (0,100 + \maxof{\y1}{\y2}) \end{wrapfigure} rectangle (content.south east); [% END # if image exists %] \end{pgfonlayer} \end{tikzpicture}% 6.7 Highlight new products }} New products are highlighted by yellow text with a red background. \titleformat{@@html:\@@chapter}% {}% [%- IF item.NewProduct -%] {}% \scriptsize\colorbox{red} {0pt}% {\textcolor{yellow}{NEW}} {\boxedchapter}% \sffamily\footnotesize{~ \ \titlespacing*{\chapter}{-100pt}{*-20}{*-1} [%-item.description | latex_filter -%]} & % end CHAPTER format \footnotesize{[%-item.description | latex_filter -%]} & 6.4 Long tables [%- END -%] I needed a way to create tables that could break 6.8 Including full page PDFs across page boundaries, but also be able to span We include full page PDFs in the catalogue for the more than one page, and possibly have a PDF (for an front matter, rear matter and adverts. All are sup- advert) embedded at a split. The xtab [8] package plied as PDFs and we just have to include them. produced the best results for me; however, there The trick is to turn off scaling so the bleed and trim are very many packages for tables. I found a good marks appear in the correct place. summary of table package features at http://tex. stackexchange.com/a/12673. [% IF String.length > 0 %] \includepdf[noautoscale=true]{[%CoverPage%]} 6.5 Alternating row colours in xtab [% ELSE # warn that file cannot be found %] Cover file [% CatInfo.CatalogueFrontPage %] TT This was easy to do in : while looping through appears to be missing : [% error.info %] query results, simply change the row colour every [% END %] two lines. \begin{xtabular}[l] 7 Conclusion [% i = 1 -%] I set out to create a tool for generating a catalogue [% FOREACH item = db.query( from our database. I was able to use free tools such "select * from $CatInfo.query") %] as LATEX and Template Toolkit to achieve this. The [%- IF (i mod 4 == 3) overall goal was achieved, saving time and money || (i mod 4 == 0) -%] and allowing staff to be more productive. \rowcolor{[%- item.SectionRow2BGColour -%]} References [% ELSE -%] \rowcolor{[%- [1] http://www.docbook.org/docbook item.SectionRow1BGColour -%]} [2] http://search.cpan.org/dist/DBI/ [%- END #iF -%] dbiproxy.PL [%- i = i+1 -%] [3] http://ctan.org/pkg/latexmk [% item.col1 %] & [4] http://launchpad.net/rubber [% item.col2 %] & [5] http://search.cpan.org/dist/App-cpanminus/ [% item.col3 %] \\ %row data bin/cpanm [% END; #FOREACH %] [6] http://www.tug.dk/FontCatalogue/helvetica \end{xtabular} [7] http://ctan.org/pkg/flowfram 6.6 Wrap description around an image [8] http://ctan.org/pkg/xtab http://ctan.org/pkg/wrapfig I used wrapfig [9] to put text around an image. [9] [% IF ImagePath != "" %] Jason Lewis \begin{wrapfigure}{r}{0pt} http://organictrader.com.au
How to make a product catalogue that doesn’t look like a dissertation