Connecting LuaTEX to MongoDB

Roberto Giacomelli

Abstract primary and foreign key constraint, the not null clause and so on. This paper describes in details a way to connect Secondly, the valuable advantage is that avoiding LuaT X to a MongoDB database server. As a con- E intermediate layer between data and T X means sequence, the Lua-powered typesetting engine be- E speeding up modifications. comes a candidate, along with other tools, to gen- I have frequently adapted project entity- erate beautiful reports. relationship diagrams whenever improvements From local-wide project to large web ap- were (e.g., the addition of a complex set of eco- plications, MongoDB—the popular NoSQL nomic information). So, I learned a lot about how database—creates a new point of view on datasets to translate real world entities into conceptual SQL also adopted by LuaT X. This could extend the E models, trying to keep minimalism in mind as much development perspectives. as possible. Changes were easy to apply but, after six years Sommario of development, things are no longer so easy to Questo articolo descrive in dettaglio un modo per keep up-to-date despite the success of the project. connettere LuaTEX a un database server Mon- After all, the world is constantly changing. And goDB, dimostrando come il motore di composi- I consider more important finding out database zione dotato dell’interprete Lua possa candidarsi, systems to allow a better data representation than insieme ad altri strumenti, a generare bellissimi incrementing data bandwidth as fast as hardware report. potentially could do. In other words, I’m looking Dal piccolo progetto alle grandi applicazioni web, for more natural criteria to model a reality in rapid MongoDB — il noto database di tipo NoSQL — and unpredictable evolution. crea un nuovo punto di vista sui dati che si ritro- When I talk about project I mean a production va anche in LuaTEX. Ciò potrebbe incrementare business where data are shared over the IT network ulteriormente le possibilità di sviluppo. infrastructure and automatically processed to get reports. Just think of a freelance engineer or a little 1 Reasons design team, dealing with invoices of technical documents as part of their customer service, or I’m currently working on a project based on Lua- a company delivering financial statements to the 1 jitLATEX, and SQLite3 as a set of databases han- clients. dler, to typeset a lot of high quality reports. I exposed about that in my article (GIACOMELLI, 1.1 What I’ll talk about 2017). This paper is covering a brief description of the LuajitTEX includes the LuaJIT FFI Foreign Func- NoSQL unconventional way of thinking in the tion Interface, a powerful technology to bind C section 2, where I also introduce the MongoDB executable modules such as the dynamic linking database system presenting fundamental concepts SQLite3 library. Not only LuajitTEX becomes capa- like the document. ble of collecting data by running SQL queries, but Section 3 and the following two are dedicated to it also makes life easier in configuring the system. a step-by-step detailed tutorial valid for Ubuntu Why did I discard simpler data management soft- and Windows operating systems—section 4 and ware such as spreadsheets or plain text files, and section 5 respectively. They provide instructions to why didn’t I ensure a full separation between TEX set up a MongoDB local server for experimenting and data layer, for example through an agnostic and a suitable MongoDB connector for LuaTEX. file format? Finally, from section 7 on I will discuss two demo First of all, the main goal of this architecture projects simple and meaningful, useful to practice is to ensure data safety: that is why I chose a the projects development. RDBMS system2. Every record saved in the archive is checked by the database engine itself that con- 1.2 Verbatim conventions trols the integrity of the relational model, in con- When a shell command doesn’t fit in the column sideration of a handful of concepts such as the width, an ending backslash \ is added to split text up into more lines. 1. https://www.sqlite.org. 2. The acronym RDBMS stands for Relational Database While experimenting, you can improve readabil- Management System. ity too, splitting the lines up in your command

62 ArsTEXnica Nº 26, Ottobre 2018 Connecting LuaTEX to MongoDB window according to your operating system syn- documents. Thus documents literally represent tax. For instance, Windows PowerShell supports themselves. Shift+Enter keys combination for multiline editing In the document above—delimited by or even a backtick3 ‘ at the end of breaking lines. braces—there are two fields separated by a Beware: PowerShell inserts a space for each newline comma. Each field has a key and a value separated you enter, so you won’t be able to even split up a by a colon. Keys are strings while values are, file path, while Linux or Mac OSX user can safely respectively, a string and an integer. Values use backslash \ to break up to command names. are not limited to atomic types like those just considered: they can also represent compound 1.3 Requirements types such as arrays—a list of comma separated You need a desktop computer running a 64 bit Op- values, enclosed in square brackets—or even other erating System in order to execute demo projects documents. For example, we can add geospatial programs illustrated from section 7 on: running information as an array of coordinates localizing locally database operations and typesetting docu- the river source: ments for reports. A basic Lua understanding is { also recommended to knowingly run such source ”name” : ”Missouri River”, files with LuaTEX. ”length km” : 3768, ”source coords” : [45.9275, -111.508056] 2 NoSQL, Not only SQL } That notation—pretty much the same of the ta- NoSQL philosophy denies the SQL model: no fixed ble constructor in Lua—corresponds exactly to the schemas and no relational constraints are imposed JavaScript object type and it highlights the strong on data. Leaving out that essential checks, web inclination of MongoDB for web development. applications can take advantage of NoSQL in two main areas: a great capability to scale out—that is 2.2 Collection and database partitioning data across several servers, optimizing A MongoDB collection is a set of documents and, costs and performances—and a flexible design of in turn, a database is a set of collections. data models. Shortly, a powerful and easy to use Collections are referenced by name as a single storage engine. UTF-8 string or as a sequence of strings concate- The heart of the SQL system is the table: a nated with the dot character. It’s the recommended tabular dataset organized in rows and columns that way to organize document groups with namespaces. describes and represents entities. Improvements For instance, the previous document regarding require to reformulate all the dataset archived in essential data about rivers could be part of the the table. This is what we call a fixed-schema: it’s info.basic collection. Please pay attention to the just not possible to write data that don’t suit the fact that in this case there is no collection belonging current schema. to another one but it’s only an optional naming Quite the opposite for the NoSQL databases: rule to identify a single collection. the information models can be eventually changed, There are no constraints defined on the doc- whatever and wherever. ument structure except on the field _id. If the Google, Amazon and other Internet big compa- document does not have an _id field, MongoDB nies are developing their own NoSQL data stor- will attach one to it with a default value of type age system. In this article, I’m going to consider ObjectId, otherwise it will check if the _id value the open source project MongoDB located at of any type is unique within the collection. www.mongodb.com. 2.3 The JSON and BSON formats 2.1 The MongoDB document JSON (JavaScript Object Notation) is a text-based A MongoDB document is a key/value ordered list. data-interchange open standard format. It is built As a matter of fact, not only the database can upon a key/value structure called object and a list archive data in binary format, but can also under- called array. Values types belong to a list of seven of stand the structure of the key/value data type for them like string or number, but even object and array. querying. The JSON specification takes only a single A document looks like this: web page, see https://www.json.org/, to be com- pletely defined. As the website says, JSON iseasy {”name”:”Missouri River”, ”length km”:3768} for humans to read and write, and it is also easy for machines to parse and generate. In absence of a fixed and predefined SQL-like JSON is not alone in the arena of structured schema, MongoDB developers use to visualize text formats; competitors are TOML4, YAML5, how information is structured, simply printing 4. Tom’s Obvious, Minimal Language https://github.com/ 3. Backtick is issued typing ALT+0096 on Windows ma- toml-lang/toml. chines. 5. YAML Ain’t Markup Language http://yaml.org/.

63 Roberto Giacomelli ArsTEXnica Nº 26, Ottobre 2018

XML6, RON7 and many others, but JSON is very • an independent execution of CLI tools like popular on the modern web. mongo shell itself, using files as communication The data transfer over the network is based on channel with LuaTEX (see section 10). the serialization/deserialization process where text is encoded into an efficient binary format. BSON Depending on the state of the art of open source (Binary JSON) is the network transfer format used projects, I chose the low level bindings. Building by MongoDB. them requires two steps regardless of the target OS: compiling the MongoDB C driver and binding Lua 2.4 Working with mongo to it via the LuaRocks package manager. LuaRocks Once MongoDB is installed, a JavaScript-based must use the same Lua version of the LuaTEX CLI client called mongo is available alongside the interpreter, currently 5.2 for a standard TEX Live other executables. 2018 distribution. mongo is a way for accomplishing administrative To determine your Lua version you can read 8 tasks on databases and it is a useful tool for get- the LuaTEX manual shipped with TEX Live or ting started with CRUD operations and querying. compile the following file with LuaTEX and check The acronym CRUD stands for the four basic op- the resulting PDF: erations on databases: Create, Read, Update and % !TeX program = LuaTeX Delete. \directlua{tex.print(lua.version)} 2.5 MongoDB learning resource \bye For more information on MongoDB visit the https: 3.1 Installing a MongoDB driver for //docs.mongodb.com website. It’s a clear, complete LuaTEX and well organized documentation site. A good The first step is to compile the MongoDB C printed reference is the book (CHODOROW, 2013). driver and its companion. The project home is Furthermore, online courses can be attended at at http://mongoc.org/, where we can find both https://university.mongodb.com/ website: ba- source code and documentation for the library sic, intermediate and advanced levels can suit ev- libmongoc and the companion libbson. The lat- erybody’s needs. est library deals with the BSON format, that is the binary format used by MongoDB to store doc- 3 Setting up a stand-alone system uments in the database and to perform network The objective is to get started with a locally- communication, as mentioned in the section 2.3. running MongoDB instance. With few adjust- The second step is to compile a Lua binding ments you should be able to run the database to MongoDB C driver. The project I take into server within a Local Area Network. Of course, a account is lua-mongo hosted on GitHub at https: production-ready installation of a MongoDB server //github.com/neoxic/lua-mongo. requires special treatments for security issues, an ef- In this paper I will only give detailed installation fort that goes beyond the objectives of the present instructions for Ubuntu Linux and Windows. work. Anyway, a 64 bit operating system is required 4 Installing on Ubuntu to run recent versions of MongoDB. No such lim- First of all it’s important to notice that the Mon- itation involves the client running from within a goDB binary for Ubuntu has been compiled with LuaT X process. E SSL enabled and dynamically linked. This requires A Lua-MongoDB connector can be build in dif- the SSL libraries to be seperately installed on your ferent ways: system with the following shell command: • a low level binding to the MongoDB C driver $ sudo apt-get install libcurl4 openssl http://mongoc.org/; • a pure Lua connector that understands the Adding to the repository list the official Mon- MongoDB server binary protocol over TCP goDB software archive, you can also install the apt network protocol; binary via the standard package manager. Do- ing so the package will be updated automatically. • an intermediate node connected to LuaTEX However, the manual procedure allows you to keep via TCP using the library luasocket, that is things under a complete control. statically linked in the Lua-powered typeset- ting engine. This proxy server is connected 4.1 Installing MongoDB to MongoDB by means of every suitable pro- MongoDB installation is very simple because you gramming languages; only have to download and unzip a file: select

6. Extensible Markup Language https://www.w3.org/XML/. 8. To access your TEX-related documentation just type 7. Rusty Object Notation https://github.com/ron-rs/ texdoc package in a terminal session. For instance, texdoc ron. luatex. h i

64 ArsTEXnica Nº 26, Ottobre 2018 Connecting LuaTEX to MongoDB

TABLE 1: Direct links to the downloadable files of the MongoDB Community Server package available at the time of writing for the main 64 bit desktop OS.

OS identifier/Direct download link Ubuntu 18.04 LTS 64 bit https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1804-4.0.1.tgz Windows 64 bit installer https://fastdl.mongodb.org/win32/mongodb-win32-x86_64-2008plus-ssl-4.0.1-signed.msi OSX 64 bit https://fastdl.mongodb.org/osx/mongodb-osx-ssl-x86_64-4.0.1.tgz the Community Server tab from the official Mon- /mongo-c-driver-1.12.0.tar.gz goDB download web page https://www.mongodb. $ tar xzf mongo-c-driver-1.12.0.tar.gz com/download-center and pick your choice select- $ cd mongo-c-driver-1.12.0 ing your preferred Operating System. You can also $ mkdir cmake-build grab your installation package from the direct link $ cd cmake-build $ cmake \ shown in table 1. -DENABLE_AUTOMATIC_INIT_AND_CLEANUP=OFF .. Placing executables in a system directory is safer due to the superuser rights protection, but for If everything is OK, you may go on and install testing operations I prefer to temporarily keep the the driver—make sure that the working current binary as well as the database files in my home directory is cmake-build—: directory: $ make $ tar -zxvf \ $ sudo make install mongodb-linux-x86_64-ubuntu1804-4.0.1.tgz $ cp -r \ 4.3 Installing LuaRocks ./mongodb-linux-x86_64-ubuntu1804-4.0.1 \ LuaRocks instruction from its website https:// ~/mongodb luarocks.org/ are quite clear: $ mkdir ~/mongodb/data $ wget https://luarocks.org/releases\ The following commands make the server start /luarocks-3.0.0.tar.gz on the localhost network address with no user $ tar zxpf luarocks-3.0.0.tar.gz authentication—in production environments a safe $ cd luarocks-3.0.0 access to the server must be configured: $ ./configure; sudo make bootstrap $ cd ~/mongodb/bin 4.4 Installing lua-mongo bindings $ ./mongod --smallfiles --dbpath ~/mongodb/data The very last shell command is: Carefully look at the output produced by mongod: $ sudo luarocks install lua-mongo you can check if the system is up and correctly running. To safely stop the server press CTRL+C. 4.5 Post-install adjustment It’s also recommended to install a copy of the To provide visibility to the Lua MongoDB bind- Compass Community Edition GUI client, looking ing from within LuaTEX, the library file could for related tab form in the download MongoDB web be copied in the bin directory of the main site. In figure 1 is shown a screenshot of Compass. TeX Live tree, accordingly to the system variable $CLUAINPUTS 4.2 Compiling MongoDB C Driver . In fact, it contains paths where the typesetting engines looks for the C module. As a preliminary step, we need to install some Indeed the LuaTEX manual reports the variable packages from the Ubuntu repository. These shell value as the following pattern: commands do the job: CLUAINPUTS= .:\ $ sudo apt-get install build-essential cmake $SELFAUTOLOC/lib/{$progname,$engine,}/lua// $ sudo apt-get install liblua5.2-dev $ sudo apt-get install libssl-dev libsasl2-dev The first directory in the pattern is simply the folder containing the .tex source file itself, identi- As reported in the installation guide of the Mon- fied by the dot char ‘.’, but what about the next goDB C driver, download, unzip and prepare the $SELFAUTOLOC variable? To find out the answer we project with: can address LuaTEX itself, compiling the source 9 $ wget \ file below: https://github.com/mongodb/mongo-c-driver\ 9. We obtain the same answer if we invoke the command /releases/download/1.12.0\ kpsewhich -var-value= $SELFAUTOLOC from the terminal.

65 Roberto Giacomelli ArsTEXnica Nº 26, Ottobre 2018

FIGURE 1: A screenshot of Compass Community Edition GUI client running on Ubuntu 18.04 while connected to the local MongoDB server. It shows a document of the prize collection belonging to the nobelprize database, part of the project defined in section 9.

% !TeX program = LuaTeX $ cd /usr/local/texlive/texmf-local \directlua{ $ sudo mkdir -p ./scripts/mongo print(kpse.expand_var [[$SELFAUTOLOC]]) $ sudo cp /usr/local/lib/lua/5.2/mongo.so \ } ./scripts/mongo/ \bye that prints on the standard output the variable 5 Installing on Windows expansion as: Since this task is slightly long, you may find com- /usr/local/texlive/2018/bin/x86_64-linux fortable to help yourself some coffee or tea, cookies and so on, and relax. Let’s start installing Visual Essentially, C modules home can be only a Studio Community latest release, currently 2017, platform-specific directory in the main TDS tree10, from Microsoft website https://visualstudio. and this in far from the ideal solution because microsoft.com/en/downloads/. If necessary, regis- the main tree is something that users should leave ter your Visual Studio copy submitting an account untouched, giving out local or personal tree of dis- from the dedicated command you find in the Help tribution to settle what is not officially part of menu. TEX Live. Later, it will be useful Microsoft Build Tools A workaround may be directly using the loader 2015 too, so install it from Microsoft Download internally called by the Lua function require(), Center but only after installing Visual Studio. that is the standard way to load a module. You also need to install cmake from https:// require() has a default list of search paths in- cmake.org/download/. You should have a 64 bit cluded those derived from the pattern in the OS so choose win64-x64 architecture and not win32- $CLUAINPUTS variable. x86 one. While we will discuss this in the section 6, let’s now complete the installation: we only have to 5.1 Installing MongoDB copy the C module in the local tree of TEX Live The Windows installer—the .msi file reported in with the command: the table 1—gives you two different kinds of config- 10. For more information on TDS tree structure run the uration: running MongoDB as a service or not. The shell command texdoc tds. second option is recommended for testing and local

66 ArsTEXnica Nº 26, Ottobre 2018 Connecting LuaTEX to MongoDB project according to our experimental purpose. In 5.3 Installing Lua Libraries any case, you may wish to install the Compass Pursuing a Lua driver installation for MongoDB, utility too, a GUI client for MongoDB databases. make sure you have Lua itself installed with the If so, check on the related flag in the Setup Wizard same version of LuaTEX, as explained before. If and press the Next button to enrich the final step. not, it is very simple, thanks to the project Lua To locally start the server, run in a console win- Binaries, getting that files. dow the following commands: Browse the web page http://luabinaries. > mkdir C:\mongodb-data sourceforge.net/ and pick up Lua 5.2.4 release > cd ’C:\Program Files\MongoDB\Server\4.0\bin\’ pages hosted on SourceForge. Click in sequence on > .\mongod --smallfiles --dbpath C:\mongodb-data the folder Windows, hence Dynamic, and download Check the output to assess if all is OK, then stop the file lua-5.2.4_Win32_dll15_lib.zip. The file the server pressing CTRL+C to send a shutdown name tells whether the platform we have chosen signal to the process. is correct: Win32 shows that library is for 32 bit binary and dll15 is about the latest Visual Studio 5.2 Compiling MongoDB C Driver version. Download the source code of the latest version of Please make sure you have clicked on the the C driver from http://mongoc.org/#download, Dynamic and not on the Static link or you will current release is 1.12.0, unzip the file mongo-c- suffer soon or later the error Multiple Lua VM’s de- driver-1.12.0.tar.gz, then open a console on tected. As a comparison, in the file name you must the uncompressed folder. read the string dll, standing for dynamic linking According to the documentation, the commands library in the Microsoft ecosystem terminology. to create a Visual Studio 2017 project are the Unpack the files and copy all of them in the following (the double dot in the last line is impor- folder, say C:\Lua, as a name easy to trace and re- tant): member. That folder will be the home of LuaRocks > cd mongo-c-driver-1.12.0 too, the next main character of our “play”. > mkdir cmake-build From the same repository, download a > cd cmake-build Lua interpreter clicking on the link folder > cmake -G ”Visual Studio 15 2017” \ Tools Executables. The file should be lua- ”-DCMAKE_INSTALL_PREFIX=C:\mongo-c-driver” \ 5.2.4_Win32_bin.zip. Unzip it in the same di- ”-DCMAKE_PREFIX_PATH=C:\mongo-c-driver” \ rectory where you are going to place the libraries .. file. If you don’t know how to split the last command Add to the system PATH variable the folder into several lines inside the shell, simply type it in C:\Lua, then run the interpreter. a single line. 5.4 Installing LuaRocks Since the luatex executable is a 32 bit x86 bi- nary, make sure Visual Studio is set for Win32 The starting point for the Lua most famous https: platforms before compiling. To inspect which ar- package manager is the official website //luarocks.org/ chitecture luatex is compiled for, digit the shell . The latest release is 3.0.1 but at command: the moment it is unable to complete the installation process due to a dynamic linking failure. Fall back > luatex --version on 2.4.4 version downloading the file luarocks- If the answer is (or looks like) 2.4.4-win32.zip from http://luarocks.github. io/luarocks/releases/ This is LuaTeX, Version 1.07.0 \ . (TeX Live 2018/W32TeX) Unzip it and run this command: it’s evident that we have a Win32 compliant pro- > .\install.bat /P ’C:\Lua\Luarocks244’ \ gram. /CONFIG ’C:\Lua\Luarocks244’ Back to the source folder, open the generated file The option /P gives the location where LuaRocks mongo-c-driver.sln in Visual Studio, select into will be installed, while /CONFIG gives the location the explorer the solution named ALL_BUILD, right of the configuration file. click on it and run the command Compile. When As it should be clear now, add to the user vari- the process ends, repeat the same steps with the able the entries LUA_PATH and LUA_CPATH as follow solution named INSTALL. At this point you should (type the first, that has two paths, in one line): see in C:\mongo-c-driver directory the files you were waiting for. LUA_PATH = C:\Lua\systree\share\lua\5.2\?.lua; C:\Lua\systree\share\lua\5.2\?\init.lua Add to the system PATH variable the folder C:\mongo-c-driver\bin . This allows Windows to LUA_CPATH = C:\Lua\systree\lib\lua\5.2\?.dll find the binary. In order to easily open thedia- log, type in the search bar “system environment We have to append the path of the MongoDB variable” and select the proper icon. C driver binaries

67 Roberto Giacomelli ArsTEXnica Nº 26, Ottobre 2018 external_deps_dirs = { path, ”c:/mongo-c-driver”, ”luaopen_mongo” } ) assert(_mongo) in the LuaRocks configuration file config- local mongo = assert(_mongo()) 5.2.lua. % print the available functions lua-mongo local par = string.char(92)..[[par]] 5.5 Installing the binding for k, v in pairs(mongo) do Navigate to the folder C:\Program Files (x86) if type(v) == [[function]] then Microsoft Visual C++ Build Tools. Right click tex.print(k..[[()]]..par) on “Visual C++ 2015 x86 Native Build Tools Com- end mand Prompt”, and select “Run as administrator”, end then in the opened console window, run the follow- } \bye ing command: ”C:\Program Files (x86)\LuaRocks\luarocks.bat” \ install lua-mongo As a general test, that .tex file prints also in the PDF file the list of the functions available inthe In order to allow LuaTEX to load the driver connector lua-mongo. If the compilation process using the strategy explained in section 4.5, as the ends correctly, the minimal test can be considered very last commands run the following sequence: passed and the functions list appearing regardless > cd C:\texlive/texmf-local of the order should be the following: > mkdir scripts > cd scripts Double() > mkdir mongo Int64() > cd mongo Decimal128() > copy ”C:\Lua\systree\lib\lua\5.2\mongo.dll” Regex() BSON() 6 Testing Timestamp() Binary() In the end, the connector binary file is in the lo- DateTime() cal tree, more specifically, in the sub-directory ObjectID() $TEXMFLOCAL/scripts/mongo. It can’t be loaded Int32() by the usual Lua function due to the restricted list ReadPrefs() of search path set for require(), as explained in type() the section 4.5. Client() The Lua function package.loadlib fits our case. Javascript() It returns the initialization function of a C module written according to the Lua C API, that returns That list can be compared with the API the actual library. The expected arguments are the documentation of the project at the URL path to the file and the name of the initialization https://github.com/neoxic/lua-mongo/blob/ luaopen_ function beginning with . master/doc/main.md. Instead of hard-coding the library path, a more general option is to call the LuaTEX specific func- tion expand_var() from the kpathsea utility mod- 7 LuaTEX application ule; in this way we get the value of the $TEXMFLOCAL variable and take into account the file extension In the next two sections I present example projects depending on the Operating System—.dll for involving operations on a MongoDB database and .so Windows, for other OS—calling the function queries execution from within LuaTEX in order to os.type(). typeset reports. The code that implements such solution is the Be sure you have a mongod server instance locally following: up and running while you compile with LuaTEX. % !TeX program = LuaTeX So, along with a modern TEX distribution, you have \directlua{ installed several components in order to compile local path = kpse.expand_var [[$TEXMFLOCAL]] the source code examples. In case you didn’t do assert(path) that yet, please head to section 3 for details. if os.type == [[windows]] then path = path..[[/scripts/mongo/mongo.dll]] All of the project files are downloadable from else ArsTEXnica home page at https://www.guitex. path = path..[[/scripts/mongo/mongo.so]] org/home/it/arstexnica. Browse issues online end page to find out the link pointing to the related local _mongo = package.loadlib( compressed archive named mongodb-project.zip.

68 ArsTEXnica Nº 26, Ottobre 2018 Connecting LuaTEX to MongoDB

8 Rivers { ”state”:”Nebraska”, The first example models a very simple dataset ”code” :”NE”, 11 regarding the longest rivers in the United States , ”shortname”:”Nebr.” including a one-to-many relationship: alongside }, basic information like river name and length, there { is the list of the crossed states too. ”state”:”Iowa”, The dataset would be represented in a SQL ”code” :”IA”, schema with three different tables: ”shortname”:”Iowa” }, • table river with the basic information; { ”state”:”Kansas”, • table regions defining states names; ”code” :”KS”, ”shortname”:”Kans.” • a pure relational table crossing connecting }, rivers with states through a pair of foreign key { identifiers. ”state”:”Missouri, ”code” :”MO”, On the contrary, thanks to the recursive nature ”shortname”:”Mo.” and compound data types of MongoDB documents, } we can represent rivers and crossed regions by them ] with only one document, within a single collection. } A document may contain the list of the rivers regions as an array assigned to the field regions In MongoDB we may decide to embed or to refer- as the following: ence data. While the first technique is represented by the latest river model, the second consists in { just including the reference to the region docu- ”name” : ”Missouri River”, ments instead of the documents themselves. Those ”length_km” : 3768, references have the form of _id values of the corre- ”regions” : [ sponding documents in a different and unspecified ”Montana”, collection. ”North Dakota”, ”South Dakota”, Deciding which is the best solution depends on ”Nebraska”, the performance of the application context and not ”Iowa”, on the normalization criteria that leads the design ”Kansas”, of SQL schemas. ”Missouri” 8.1 Making the database ] } In order to create the database, the simplest method is to write a JavaScript file river.js to We can even enrich the previous model because be executed in the mongo shell with the command: the regions may be represented as an array of em- bedded documents as showed in the listing below: $ mongo --host localhost river.js { // embedded document version The code must define names, collection, and ”name” : ”Missouri River”, rivers documents, structured as formerly seen in ”length_km” : 3768, the first model for the longest river in the USA, ”regions” : [ where regions are a simple list of strings. The script { river.js can be illustrated by the listing below ”state”:”Montana”, where a three dots sequence shortens lines but not ”code”:”MT”, the essence: ”shortname”:”Mont.” }, // get/define database object { db = db.getSiblingDB(”river”); ”state”:”North Dakota”, ”code” :”ND”, // data insertion ”shortname”:”N. Dak.” db.generalinfo.insertMany([ }, {...}, {...}, {...}, ... { ]) ”state”:”South Dakota”, ”code” :”SD”, The pre-instantiated db variable refers to the cur- ”shortname”:”S. Dak.” rent database. The method getSiblingDB() of the }, db type returns a pointer the requested database 11. Data source https://en.wikipedia.org/wiki/List_of_ which will be created if it does not exist. The next longest_rivers_of_the_United_States_(by_main_stem). expression is a chain where in the first stage the

69 Roberto Giacomelli ArsTEXnica Nº 26, Ottobre 2018 collection generalinfo of the database pointed mechanism of the binding library is explained in by db is returned—once again, if the collection section 6. doesn’t exist it will be created—then its method No further functions but the method FindIn() insertMany() will write an array of documents. are required. In the previous task list, it performs Too many words to explain such a few lines of the 2nd and 3rd tasks and the Lua array with all smart and practical code. mongo shell is inspired by the documents selected by the query is returned. SQL CLI clients like psql in PostgreSQL, but its The libmongo.lua code is the following: JavaScript library is designed with the following motto in mind: learn and try then configure. local lib = {} lib.__index = lib 8.2 Querying with mongo shell -- constructor It is a very common way in MongoDB that meth- function lib:Connect(dbname) ods performing operations on a database take as assert(type(dbname)==”string”) an argument a document object. For instance, in local path = kpse.expand_var [[$TEXMFLOCAL]] assert(path) mongo shell you can retrieve rivers that are exactly if os.type == [[windows]] then 2000 km long by the following query: path = path..[[/scripts/mongo/mongo.dll]] > use river else switched to db river path = path..[[/scripts/mongo/mongo.so]] > db.generalinfo.find({”length_km”: 2000}) end local _mongo = package.loadlib( Querying that on our dataset we obtain the follow- path,”luaopen_mongo” ing document: ) assert(_mongo) { local mongo = assert(_mongo()) ”_id” : ObjectId(”5b7af5a589923a94e7ce4cb0”), local client = assert( ”name” : ”Columbia River”, mongo.Client(”mongodb://127.0.0.1”) ”length_km” : 2000, ) ”regions” : [ local o = { ”British Columbia”, _mongo = mongo, ”Washington”, _client = client, ”Oregon” _dbname = dbname, ] } } setmetatable(o, self) As expected, MongoDB has automatically added return o end the mandatory _id field to each document inserted -- query method river.js by the script. function lib:FindIn( collection, query, option, prefs 8.3 This looks like a job for LuaLATEX ) Our goal is to typeset the tabular shown in table 2. assert(type(collection) == ”string”) This is what we have to do: local client = self._client local coll = client:getCollection( 1. create a client that has established a connec- self._dbname, collection tion with the MongoDB server; ) 2. get the collection object generalinfo of the local data = {} river database; query = query or {} for doc in coll:find(query, option, prefs) 3. perform the query to retrieve all the rivers :iterator() do information; data[#data + 1] = doc end 4. typeset the table with a tabular environment. return data end All these steps are straightforward. To keep the .tex source code simple and a little bit more ab- return lib stract, I’m going to write an auxiliary Lua library in the external file libmongo.lua to be located in According to the object oriented paradigm in the same directory of the main source file. Lua, we have to use the colon notation and not The constructor Connect() takes as an argu- the dot operator to call methods. If these sen- ment the database name, then establishes a client tence sounds weird to you, please refer to the book connection to the MongoDB server on localhost. (IERUSALIMSCHY, 2016) and read the chapter ded- The real job is demanded to the Lua binding of icated to metamethods and metatables and the MongoDB C Driver, compiled as described in sec- chapter dedicated to Object Oriented Program- tions 4 (for Ubuntu)–5 (for Windows). The loading ming.

70 ArsTEXnica Nº 26, Ottobre 2018 Connecting LuaTEX to MongoDB

TABLE 2: The longest United States of America’s rivers as reported by Wikipedia. The table is typeset by LuaTEX directly connecting to a MongoDB database as explained step by step in the text.

River name Length (km) Regions Missouri River 3768 Montana, North Dakota, South Dakota, Nebraska, Iowa, Kansas, Missouri Mississippi River 3544 Minnesota, Wisconsin, Iowa, Illinois, Missouri, Kentucky, Tennessee, Arkansas, Mississippi, Louisiana Yukon River 3185 British Columbia, Yukon Territory, Alaska Rio Grande 2830 Colorado, New Mexico, Texas, Chihuahua, Coahuila, Nuevo León, Tamaulipas Colorado River 2330 Colorado, Utah, Arizona, Nevada, California, Sonora, Baja California Arkansas River 2322 Colorado, Kansas, Oklahoma, Arkansas Columbia River 2000 British Columbia, Washington, Oregon Red River 1811 Oklahoma, Texas, Arkansas, Louisiana Snake River 1674 Wyoming, Idaho, Oregon, Washington Ohio River 1575 Pennsylvania, Ohio, West Virginia, Indiana, Illinois, Kentucky

The last task—number 4 in the list—is reserved As the second interesting point, the query re- to the LuaLATEX source file itself: turns documents in the same order of the original insertion, which is the order of the Wikipedia table % !TeX program = LuaLaTeX at the time I wrote the river.js script. \documentclass{standalone} An optional parameter allows us to explicitly set \usepackage{fontspec} the order, sorting documents by value. Try to edit \defaultfontfeatures{Ligatures=TeX} the previous code as in: \setmainfont{CMU Serif} \usepackage{booktabs} \directlua{ local libmongo = require ”libmongo” \directlua{ local db = libmongo:Connect(”river”) local libmongo = require ”libmongo” river = db:FindIn( local db = libmongo:Connect(”river”) ”generalinfo”, river = db:FindIn(”generalinfo”) {}, } {sort = {length_km = 1}} \begin{document} ) \begin{tabular}{lcp{120mm}} } \toprule The integer 1 specifies ascending order while -1 River name & Length (km) & Regions\\ \midrule specifies descending order, in our case applied to \directlua{ the rivers length. local stop = string.char(92) Furthermore, special query document fields stop = stop..stop named operators act as conditional parameters for _, r in ipairs(river) do being part of the MongoDB query language. They tex.print(r.name) have a dollar sign as their first character, like in tex.print(”&”) $gte—that stands for greater or equal or . For tex.print(r.length_km) instance, in the previous code we can select≥ only tex.print(”&”) those rivers that have a length greater or equal to tex.print(table.concat(r.regions, ”, ”)) 2000 km, adding a specific query document as the tex.print(stop) second argument of the FindIn() method: end } river = db:FindIn(”generalinfo”, \bottomrule { length_km = { [”$gte”] = 2000 }} \end{tabular} ) \end{document} or even by a length range like in: Reading this code, two elements worth a com- river = db:FindIn(”generalinfo”, ment: first of all, the field regions are an array of length_km = {[”$gte”] = 2000, [”$lte”]=3000}} strings as in the documents, so the Lua binding ) uses the Lua table type as the direct incarnation We can execute the same query in the mongo of the MongoDB document: in fact, the variable shell as in the following session: r in every cycle has the keys name and length_km as well as the key regions that refers to a further > use river switched to db river Lua table in the array form. The Lua standard > db.generalinfo.find({”length_km”: table.concat() function provides the string corre- ... {”$gte”:2000, ”$lte”:3000} sponding to the comma separated values of States ... }) and provinces crossed by river.

71 Roberto Giacomelli ArsTEXnica Nº 26, Ottobre 2018

Length of the longest rivers in the USA

Missouri River 3,768

Mississippi River 3,544

Yukon River 3,185

Rio Grande 2,830

Colorado River 2,330

Arkansas River 2,322

Columbia River 2,000

Red River 1,811

Snake River 1,674

Ohio River 1,575 1,400 1,600 1,800 2,000 2,200 2,400 2,600 2,800 3,000 3,200 3,400 3,600 3,800 km

FIGURE 2: The longest United States of America’s rivers as reported by Wikipedia. The histogram plot is typeset by LuaTEX performing a direct connection to a MongoDB database as explained step by step in the text.

8.4 Histogram tex.print(table.concat(t, ”,”)) }} Once data is retrieved from MongoDB and stored in Lua tables, LuaTEX can typesets information in \begin{document} whatever form the user requires, e.g., a histogram \begin{tikzpicture} plot. The figure 2 is the PDF output of the follow- \begin{axis}[ ing source file where pgfplots package plots rivers xbar, length: bar width=13pt, width=16cm, % !TeX program = LuaLaTeX height=9.5cm, enlarge y limits=0.060, \documentclass[margin=2pt]{standalone} xmajorgrids, \usepackage{fontspec} title={% \defaultfontfeatures{Ligatures=TeX} Length of the longest rivers in the USA}, \setmainfont{CMU Serif} xlabel={km}, symbolic y coords/.expanded={\riverlist}, \usepackage{pgfplots} ytick=data, \pgfplotsset{compat=1.16} nodes near coords, ] \directlua{ \addplot[draw=black,fill=blue!35!yellow] local libmongo = require ”libmongo” coordinates {\coords}; local db = libmongo:Connect(”river”) river = db:FindIn(”generalinfo”, {}, \end{axis} {sort={length_km = 1}} \end{tikzpicture} ) \end{document} } \newcommand\coords{\directlua{ for _, riv in ipairs(river) do 9 Nobel Prize tex.print( Appending to the URL http://api.nobelprize. ”(”..riv.length_km..”,”..riv.name..”)” org/v1/ one of the strings laureate.json, ) prize.json or country.json, everyone can down- end load data regarding Nobel Prizes in JSON format. }} \newcommand\riverlist{\directlua{ For more information about Nobel Prize API local t = {} please visit the website https://www.nobelprize. for i, riv in ipairs(river) do org/about/developer-zone-2/. t[i] = riv.name This dataset, freely available from the Nobel end Foundation, contains many-to-many relationships,

72 ArsTEXnica Nº 26, Ottobre 2018 Connecting LuaTEX to MongoDB such as people who won more than one Nobel more time—is an array because scientists can have Prize and single Nobel Prizes won by more than more than one affiliation, for instance when they one person. are part of world-wide research program in addition That problem is complex enough to deserve an to their Academic Institution. interesting project to be developped in MongoDB: But why affiliations fields aren’t in the laureate how should we define data models for such an model? After all Nobel Prize is unrelated to the admirable dataset? institution where laureates carry out their studies. This is right but what about the case of a laure- 9.1 Modelling documents ate who won more than one prize, each one being Instead of embedding data in a single structured afferent to different institutes or organizations? It document, we will consider a pair of distinct models is a very uncommon event, but we can’t forget the representing laureate and prize. case of Marie Curie. A listing is worth a thousand words, so let start In fact, affiliation is something that may change with a possible laureate document: during life, so we have to trace such a piece of information while the Nobel Prize is awarded. { // basic model for laureate ”_id”: 1, The introduced basic model fulfills that require- ”firstname”: ”Wilhelm Conrad”, mentin a practical way in spite of coherence. A dif- ”surname”: ”Röntgen”, ferent possible way is to reference a third referenced ”born”: new Date(”1845-03-27”), document affiliation that references laureate ”died”: new Date(”1923-02-10”), such as: ”bornCountry”: ”Prussia (now Germany)”, { // alternative prize model ”bornCountryCode”: ”DE”, ”year”: 1901, ”bornCity”: ”Lennep (now Remscheid)”, ”category”: ”physics”, ”diedCountry”: ”Germany”, ”laureates”: [ ”diedCountryCode”: ”DE”, { ”diedCity”: ”Munich”, ”id_affiliation”: 100, // reference ”gender”: ”male” ”motivation”: ”in recognition of ...”, } ”share”: 1, The fields born and died have type Date—they } ] are instantiated objects—and the remaining fields, } but the document identifier _id which is an integer, have type string. and A very simple model after all, and also flexible: { // alternative affiliation model for instance, if the laureate is alive, the field died ”_id”: 100, does not exist as well as diedCountry. Furthermore, ”id_laureate”: 1, laureate can be an organization without any first ”affiliation”: [ name. { A more structured model could represent the ”name”: ”Munich University”, prize: ”city”: ”Munich”, ”country”: ”Germany” { // basic model for prize } ”year”: 1901, ] ”category”: ”physics”, } ”laureates”: [ { while the laureate document remains the same. ”id_laureate”: 1, These alternative models require at least three ”motivation”: ”in recognition of ...”, queries to know who won a specific prize: the first ”share”: 1, one to retrieve the affiliation identifier, another ”affiliation”: [ one to query the affiliation document to get the { laureate identifier, and finally to query the laureate ”name”: ”Munich University”, document. Vice versa the same task with the basic ”city”: ”Munich”, model takes only one query for the prize document ”country”: ”Germany” and one for laureate document. } Talking about the alternative three-parted ] model, it is rather interesting the document refer- } ] ence scheme for people who won the prize twice: } there will be two prizes referencing two different affiliations, each one referencing the same laureate: The field laureates is an array of documents, a link figure similar to an upper case V. each one representing a winner referenced through From now on I will adopt the basic model count- the field id_laureate. The field affiliation—one ing the collections laureate and prize within the

73 Roberto Giacomelli ArsTEXnica Nº 26, Ottobre 2018 nobelprize database. It is out of the scope of this enum: [”female”, ”male”, ”org”], work an in-deep exploration of design pattern for description: ”enum values only” MongoDB applications. Nevertheless, it is impor- }, tant to show an example of document referencing } as counterpart of document embedding. Further } } information on design pattern application in Mon- }) goDB can be found in the book (COPELAND, 2013) from O’Reilly Media. Validator may be also a query filter expression, MongoDB doesn’t offer fast join functions be- generally less expressive than a $jsonSchema oper- tween tables as in SQL. From the server point of ator. view referencing documents means a larger number Server-side validation is only a condition, that of queries when data are required, while embed- can be true or false, about acceptability of docu- ding document means a larger number of updating ments whereas client can even edit a non-compliant when data are changed. Pros and cons assessment document according to specific rules, instead of is focused on server workload rather than data rejecting such objects. coherence, as in the case of high volume website In my opinion, in most cases it is desirable to add or big data archive. a server-side validator to every collection. Anyway 9.2 Data validation in the next section there will be an example of client-side validation, mainly to provide a very What I didn’t know about the Nobel Prize was the helpful method on how to execute JavaScript code. meaning of the field share in the prize document. If the Nobel Prize is won by more than one laureate, 9.3 A Method for client-side data they share the prize in proportion and not always validation in equal parts. As an example, if three laureates The basic model defined in the previous section re- share the prize respectively with 2, 4, 4 quote, the quires to apply several adjustments to the original first one deserves one half of the prize while the JSON file spread by Nobel Prize Foundation. With others deserve a quarter each. a mix of regular expressions and JavaScript code, I I’m not digressing. It should be true that the have translated the original files into the structure sum of the inverse of each prize share values is required by the basic model. The final result con- exactly equal to 1. Prior MongoDB 3.2 version, sists of two JavaScript scripts called laureates.js client-side validation was the only option to find and prizes.js. out errors that MongoDB wouldn’t have discovered Each script contains only one assignment: a vari- in absence of any server-side constraints usually able takes a literal array of objects. I’m going to active in a SQL schema. show how to load these files very soon. In a com- Recent versions of MongoDB are able to vali- pact syntax the laureates script may be written date documents before update or insert operations as: concerning a single collection. For instance, we can validate every values types, the existence of a // laureates.js mandatory field and the binding of a value toan var laureate = [{...}, {...}, ...]; enumeration. As an example on such server-side validation, we Now suppose such variables—laureate and can explicitly create the laureate collection adding prize—are available in a JavaScript environment: a validator as an object with the $jsonSchema we could check everything about dataset integrity operator. The following JavaScript code shows an and coherence by means of the full featured example: JavaScript programming language. For instance, to ensure that gender fields values are only limited db.createCollection(”laureate”, { to the strings male, female or org, we can iterate validator: { over laureate array and check if every gender value $jsonSchema: { is in a hash map as in the code below: bsonType: ”object”, required: [ ”surname”, ”gender” ], var gender = Object.create({ properties: { ”male” : true, // boolean values are unused firstname: { ”female”: true, bsonType: ”string”, ”org” : true description: ”must be a string” }); }, surname: { for (l of laureate) { bsonType: ”string”, if (!(l.gender in gender)) { description: ”must be a string” print(”’”+l.gender+”’ not allowed”) }, } gender: { }

74 ArsTEXnica Nº 26, Ottobre 2018 Connecting LuaTEX to MongoDB

While it will be MongoDB to check uniqueness of Passed ’laureate’ gender fields: 916/916 the field _id within the laureate collection, we are Passed ’prize’ laureate reference: 585/585 allowed to check dataset references: prize document Passed ’prize’ share fields: 585/585 must have valid laureates identifier: and finally load the dataset into database with the // checking laureate reference command: var idx = Object.create({}); $ mongo laureates.js prizes.js populate.js for (l of laureate) { MongoDB shell version v4.0.1 idx[l._id.toString()] = true; connecting to: mongodb://127.0.0.1:27017 } MongoDB server version: 4.0.1 for (p of prize) { loading file: laureates.js for (l of p.laureates) { loading file: prizes.js var id = l.id_laureate.toString(); loading file: populate.js if (!(id in idx)) { print(”wrong laureate id”) where the code of the populate.js JavaScript file } is the following: } } // run as // mongo laureates.js prizes.js populate.js and it is also not hard to check the sharing quotes of prizes: // get database object db = db.getSiblingDB(”nobelprize”); function okShare(share) { var num = 0; // numerator // insert laureates var den = 1; // denominator db.laureate.insertMany(laureate) var i; // insert Nobel Prize for (i = 0; i < share.length; i++) { db.prize.insertMany(prize) den *= share[i]; var k; 9.5 Asking LuaT X about laureates var part = 1; E for (k = 0; k < share.length; k++) { In our libmongo.lua auxiliary library introduced if ( i != k ) { in section 8 we can add a new function named part *= share[k]; FindOne() especially for treating the case of a } query returning only one document: } num += part; function } lib:FindOne(collection, query, option, prefs) return num == den assert(type(collection) == ”string”) } local client = self._client local coll = client:getCollection( // checking share fields self._dbname, collection for (p of prize) { ) var vshare = []; query = query or {} for (l of p.laureates) { return coll:findOne(query, option, prefs) vshare.push(l.share) :value() } end if (!okShare(vshare)) { A print(”wrong share group”) As a proof of concept, we can try LuaLTEX to } typeset basic information about laureates such as } the birth date and place. The Lua table that cor- responds to the query-generated document defines 9.4 Populating the database criteria with both firstname and surname fields: Functions similar to those presented in the previous local query = { section helps to keep documents reliable. Assum- firstname = ”Enrico”, ing that all of that checking code is saved in the surname = ”Fermi” file check.js, mongo shell is able to execute such } JavaScript files with the following command: The complete listing below is pretty straightfor- $ mongo --nodb laureates.js prizes.js check.js ward: once defined the Lua table fermi as a global MongoDB shell version v4.0.1 variable, we can index it with the field key without loading file: laureates.js any restriction in order to print the corresponding loading file: prizes.js value everywhere in the report: loading file: check.js ’laureate’ document to control: 916 % !TeX program = LuaLaTeX ’prize’ document to control: 585 \documentclass{article}

75 Roberto Giacomelli ArsTEXnica Nº 26, Ottobre 2018

\usepackage{fontspec} format. Subsequently, in the macro \printdate \defaultfontfeatures{Ligatures=TeX} defined in the code, we call the method unpack() to \setmainfont{CMU Serif} get the date as the number of milliseconds since the Epoch12, the moment in time fixed to the midnight \directlua{ of 1 January 1970. local libmongo = require ”libmongo” local db = libmongo:Connect(”nobelprize”) The Lua function of the standard library local query = { os.date() is able to determine date values starting firstname = ”Enrico”, from the number of seconds since the Epoch, and surname = ”Fermi” this is the reason why in the code a factor 1/1000 } multiply the unpacked value of the BSON date fermi = db:FindOne(”laureate”, query) object. } Unfortunately, the Windows system library that deals with dates returns a null pointer if the value \newcommand\print[1]{\directlua{ in time units is negative, and this is what happens local expr = tostring(#1); in the case of Enrico Fermi, who was born be- if expr then fore 1 January 1970. As a consequence, os.date() tex.print(expr) nil end returns . }} The best solution is to transform BSON date object in Lua date object. In recent time I have \newcommand\printdate[1]{\directlua{ aimed at the LuaDate project located at http: local d0 = #1; //tieske.github.io/date/, but I’m aware, time local d1 = d0:unpack()/1000 counting is not so easy, and this project could miss local d2 = os.date([[*t]], d1) the goal depending on your need. local m = { ”January”, 9.6 Asking LuaTEX about Nobel Prizes ”February”, The table 3 represents the next target. It shows the ”March”, ”April”, list of Nobel Prizes awarded in 2017, with Category, ”May”, Laureates and Motivation of the single Prize. ”June”, Our basic model is divided into two collections: ”July”, prize and laureate. In the first collection a prize ”August”, document defines, along with other pieces ofin- ”September”, formation, the category and, for each laureate, an ”October”, identifier that refers to the corresponding docu- ”November”, ment in the second collection, a share value and a ”December ”, motivation. } To summarise, the query process consists in three tex.print( tostring(d2.day), m[d2.month], d2.year different steps: ) }} • query prizes awarded in 2017;

\begin{document} • for each selected prize, query each laureate for \print{fermi.firstname} \print{fermi.surname} their complete names and share values; was born in \print{fermi.bornCity} on \printdate{fermi.born}. • then determine the main motivation as the \end{document} motivation of the laureate that has the most relevant share value or, two or more laureates The result is: have the same share, that comes first in the ordered array. Enrico Fermi was born in Rome, Italy, on 29 September 1901. MongoDB has a powerful query language with Dates are saved as objects and not as strings. It aggregate operations, projections and so on. It’s was our initial choice. After all objects are more probably capable to return the desired result with clever than strings because they have specific meth- only one query actually composed by a chain of ods. When dealing with dates, we can print only operations. However, I will implement the job in the year or even the days between two dates. Lua: this paper focus isn’t on learning advanced In spite of that, why the source code of our MongoDB. It is also important to notice that the working minimal example compiles under Linux ideal query takes advantage of what the server-side but not under Windows? has to offer. It’s because dates are not yet a Date object. In 12. For more information please visit the webpage https: fact, the lua-mongo driver returns dates in BSON //en.wikipedia.org/wiki/Unix_time.

76 ArsTEXnica Nº 26, Ottobre 2018 Connecting LuaTEX to MongoDB

TABLE 3: The Nobel Prizes awarded in 2017. The table is typeset by LuaTEX performing a direct connection to a MongoDB database as explained in the paper.

2017 Category Laureates (share)/Main motivation physics Rainer Weiss (2), Barry C. Barish (4), Kip S. Thorne (4) for decisive contributions to the LIGO detector and the observation of gravitational waves chemistry Jacques Dubochet (3), Joachim Frank (3), Richard Henderson (3) for developing cryo-electron microscopy for the high-resolution structure determination of biomolecules in solution medicine Jeffrey C. Hall (3), Michael Rosbash (3), Michael W. Young (3) for their discoveries of molecular mechanisms controlling the circadian rhythm literature Kazuo Ishiguro (1) who, in novels of great emotional force, has uncovered the abyss beneath our illusory sense of connection with the world peace International Campaign to Abolish Nuclear Weapons (ICAN) (1) for its work to draw attention to the catastrophic humanitarian consequences of any use of nuclear weapons and for its ground-breaking efforts to achieve a treaty-based prohibition of such weapons economics Richard H. Thaler (1) for his contributions to behavioural economics

Finally, the complete code generating the table 3 ” (”..share..”)” is showed in the listing below. Improvements are end left to the reader, such as to eliminate the redun- p.laureatelist = table.concat(tl, ”, ”) dant \midrule in the last row, to capitalize the cat- p.mainmotivation = motivation or ”” egory name, or to eliminate the share value when end } unnecessary, that is when there is no sharing at all: \newcommand\print[1]{\directlua{ % !TeX program = LuaLaTeX local expr = tostring(#1); \documentclass{standalone} if expr then \usepackage{fontspec} tex.print(expr) \defaultfontfeatures{Ligatures=TeX} end \setmainfont{CMU Serif} }} \usepackage{booktabs} \begin{document} \newcommand\YEAR{2017} \begin{tabular}{lp{140mm}} \toprule \directlua{ \textbf{\YEAR}\\ local libmongo = require ”libmongo” \midrule local db = libmongo:Connect(”nobelprize”) Category & Laureates (share)/Main motivation\\ local query = {year = \YEAR} \midrule prize = db:FindIn(”prize”, query) \directlua{ local bs = string.char(92) for _, p in ipairs(prize) do stop = bs..bs local tl = {} for _, p in ipairs(prize) do local share, motivation tex.print(p.category) for i, l in ipairs(p.laureates) do tex.print(”&”) if share then tex.print(p.laureatelist) if share < l.share then tex.print(stop) share = l.share tex.print(”&”) motivation = l.motivation tex.print(bs..”small ”) end tex.print(p.mainmotivation) else tex.print(stop) share = l.share tex.print(bs..”midrule”) motivation = l.motivation end end } local ql = {_id = l.id_laureate} \bottomrule linfo = db:FindOne(”laureate”, ql) \end{tabular} tl[i] = linfo.firstname .. ” ” .. \end{document} (linfo.surname or ””) ..

77 Roberto Giacomelli ArsTEXnica Nº 26, Ottobre 2018

10 Alternative ways The second part of this paper (sections 8–9) has shown two demo projects from the very first step Some alternative ways are suitable for experiment- in designing data models and creating database ing or for self-contained project, and can be part via JavaScript to the working example in LuaT X of a step-by-step development strategy toward an E for generating reports. HTTP or TCP MongoDB proxy server. MongoDB documents are expressive, flexible and Different ways to query MongoDB databases make it easy to improve data models. Thanks to the from LuaT X are alternative in the sense that E similarity to the Lua tables, high-quality reports they don’t use neither a Lua low-level binding can be typeset by LuaT X with less code than nor an intermediate network service. They rely E that required when the storage engine is a SQL on independent tools and an elementary form of relational database. communication between LuaT X and data, that is E Further work is required to fully use MongoDB essentially a file. query language capabilities, especially for building I’m referring to a file as a communication layer project in real context, and to make a final decision in two different ways: about how to safely connect LuaTEX to MongoDB. static: the file is saved in the file system and contains data that are a query result, encoded 12 Acknowledgments in a declared format like JSON; I would like to thank all of the ArsTEXnica team dynamic: textual result of a query is printed to members who have supported this work with great the standard output and caught by Lua via effort. My gratitude goes also to my family. the standard function io.popen(). If the channel is static, then data files are file References system objects, and the unique limitation is the CHODOROW, K. (2013). MongoDB: The Defini- precise knowledge of the exchange data format. If tive Guide: Powerful and Scalable Data Storage. the channel is dynamic, it is even required that O’Reilly Media, 2ª edizione. URL http://shop. tools generating data are a CLI program. oreilly.com/product/0636920028031.do. For instance, the external tool can be the mongo shell or Node.js. Both execute a JavaScript file or COPELAND, R. (2013). MongoDB Applied even a Rust program taking advantage of drivers Design Patterns. O’Reilly Media, 1ª edizione. like the open source project mongo-rust-driver- URL http://shop.oreilly.com/product/ prototype written in pure Rust and providing a 0636920027041.do. native interface to MongoDB. GIACOMELLI, R. (2017). «A database experi- 11 Conclusion ment with LuajitLATEX». ArsTEXnica, (23), pp. 12–34. URL http://www.guitex.org/home/ The first part of this paper (sections 3–5) has numero-23. shown how to set up a project infrastructure relying on MongoDB as database storage and LuaTEX IERUSALIMSCHY, R. (2016). Programming in Lua. as reporting tool. The guide is detailed both for Lua.org, 4ª edizione. Ubuntu and Windows, and leads you to a step- THE LUAT X DEVELOPMENT TEAM (2018). by-step source code compilation in order to build E LuaT X Reference Manual. Version 1.0.7. a database connector suitable for the typesetting E engine. Low-level Lua binding to the C module isn’t the . Roberto Giacomelli only communication way with MongoDB. Interme- Carrara diate network services via TCP connection proto- giaconet dot mailbox at gmail col can be implemented for LuaT X in order to E dot com simplify the components and to ensure reliability.

78