WikiGateway: a library for interoperability and accelerated wiki development Bayle Shanks http://communitywiki.org/BayleShanks Computational Neurobiology University of California San Diego, La Jolla, CA 92093 bshanks at ucsd.edu ABSTRACT Related resources WikiGateway is an open-source suite of tools for automated Related resources may be found at http://purl.net/bshanks/ interaction with wikis: work/papers/wikigateway_wikisym05/. These may include talk slides, amended or hypertext versions of this paper, and pointers to related work. It is suggested that the reader • Python and Perl modules with functions like getPage, check there for updates before reading this paper. putPage, getRecentChanges, and more. • A mechanism to add DAV, Atom, or XMLRPC capa- bilities to any supported wiki server. 1. INTRODUCTION • A command-line tool with functionality similar to the WikiGateway is a suite of tools that allows developers to Perl and Python modules. write programs that act as clients to wiki servers. Wiki- Gateway provides a unified interface to different types of • Demo applications built on top of these tools include a wiki engines. “Wiki engine” refers to a type of wiki server wiki copy command, a spam-cleaning bot, and a tool to software; for example, MoinMoin is a wiki engine. recursively upload text files inside a directory structure as wiki pages. There are a few different ways of communicating with wiki servers using WikiGateway tools. These options can be grouped in two categories; either the client software uses All WikiGateway tools are compatible with a number of WikiGateway components installed on the local machine, or different wiki engines. Developers can use WikiGateway to it communicates with a WikiGateway web service installed hide the differences between wiki engines and build applica- elsewhere. tions which interoperate with many different wiki engines. The simplest way to interact with wikis is for software run- Categories and Subject Descriptors ning on the client machine to call a WikiGateway library D.2 [Software engineering]: Interoperability function or tool. WikiGateway provides modules for Python ; H.5.3 [Information Interfaces and Presentation]: Group and Perl, and also a command-line client. Appendix A gives and Organization Interfaces; H.4 [Information systems a demonstration of various WikiGateway Python module applications]: Communications Applications functions. Appendix B shows example invocations of the WikiGateway command-line client. General Terms Standardization, Design However, WikiGateway also allows people to run web servers that act as gateways; these intermediaries accept requests Keywords from the client using a standardized protocol, and translate them into requests to the actual wiki server (which may be wiki, interwiki, interoperability, WikiGateway, client-side running on yet another machine). WikiGateway provides wiki, WikiClient, middleware, Atom, WebDAV, WikiRPCIn- tools to run servers that understand the WebDAV, Atom, terface, wiki XMLRPC or WikiRPCInterface protocols [37, 11, 21, 2, 13, 23]. Wiki- Gateway also provides a server that exposes an XML-RPC interface to the Python WikiGateway module. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are WikiGateway includes a few “demo applications” to show not made or distributed for profit or commercial advantage and that copies how it can be used. These include a despamming bot, a bear this notice and the full citation on the first page. To copy otherwise, or script to copy an entire wiki onto another wiki, and a script republish, to post on servers or to redistribute to lists, requires prior specific to upload a hierarchial directory structure of text files to permission and/or a fee. WikiSym ’05, October 16-18, San Diego, CA, U.S.A. a wiki (treating each text file as wiki markup source for a Copyright 2005 ACM 1-59593-111-2/05/0010 ...$5.00 single wiki page). This paper will begin by describing the reasons for building 1. The existence of software that can read and write to a WikiGateway, including why WikiGateway will help wiki variety of different types of wiki engines. technology to develop faster, and what kinds of software WikiGateway will allow others to build. Next, potential 2. The existence of software that knows how to convert disadvantages of building something like WikiGateway are the wiki markup between a variety of different types explored. The design and implementation of WikiGateway of wiki engine markup styles. are described, followed by a description of demo applications which were written as examples of WikiGateway usage. Fi- nally, future plans for WikiGateway are discussed, including WikiGateway currently can do (1), and the plan is for it also a list of additional demo applications to be developed. to do (2) in the future (see Section 6, “Future Work”). 2. MOTIVATIONS 2.4 Client-side tool development There are quite a few reasons that a project like Wiki- Gateway is needed. First, developers are already writing Problem: fragmentation of developer effort “screen-scraping”-style code to interact with wiki servers; it There are over 250 different wiki engines1. There are even would be more efficient for the community to write this code quite a few different “popular” wiki engines23. Even though “once and only once” in a single library. Second, there are there are many wiki software developers, their combined de- standardized protocols that wiki servers could support, but velopment effort is split into hundreds of small projects (see often don’t; WikiGateway provides gateway servers to al- Figure 1)[8]. low clients which speak standard protocols to communicate with wiki servers. Third, making wikis more interoperable One way to solve this problem would be to convince devel- will aid communications not only with other types of tools, opers to pool their efforts and focus development on a small but also between different wiki sites. For example, giving number of wiki engines. This is difficult because it would programs the ability to automatically talk to wiki servers involve a lot of individual developers and small teams giving is a prerequisite for automated wiki page interchange. Fi- up creative control and joining large organizations. nally, by allowing new ideas to be implemented as “uni- versal” client-side tools rather than as server-side features, Another answer is to allow developers who are working on a WikiGateway will help with the problem of the division of new feature to write a single implementation of that feature developer effort among different wiki engines. These moti- which will be compatible with multiple wiki engines. For vations are elaborated upon below. many features, this could be done by implementing features client-side rather than server-side. 2.1 A central collection of screen-scraping code Developers are already spending time implementing inter- Solution: implement new features client-side faces to wiki engines [22, 35, 20, 31, 19, 33, 32]. As long With the ability to download and upload the page source as developers are spending their time writing what is essen- from remote wiki servers, it is now possible to implement tially “screen scraping” code to talk to specific wiki engines, wiki user interfaces (UIs) partially or completely on the why not collect all of these efforts into a single library? client. Advanced features such as a revert button, Sub- scribedPages, “filtered Recent changes”, a collaborative rat- In addition to making it easy for developers to know where ing system for wiki pages, or “unified recent changes”4, could to go to find wiki interface code, WikiGateway has the added all be written without altering the software running at the advantage of providing clients with a single, unified API to server. use, regardless of which specific type of software the wiki server is running. There are also some features that seem to “naturally” be- long in the client. Examples include a browsing interface In the future, perhaps developers who want to write screen- based on an animated, clickable, graphical/spatial “map” of scraping code for a particular wiki engine will spend their nearby wiki pages; or a Refactoring Browser, that is, a pro- time writing a driver for WikiGateway. gram specifically designed for refactoring text in wiki pages, with special tools for moving blocks of text around between 2.2 Interoperability with standard protocols and within pages [28]. Many useful software clients use protocols such as WebDAV 1 or Atom. However, there are few wiki engines which support 252 was the result of a very superficial count of the number these protocols. WikiGateway provides a way for a DAV or of entries on http://c2.com/cgi/wiki?WikiEngines, on April 21, 2005, in the “sorted by language of implementation” Atom client to access a wiki server, even if the wiki engine section. running on the server does not know anything about DAV 2This is hard to quantify, but [38] lists 9 wiki engines with or Atom. over 100,000 Google hits; [41] lists 5 popular wiki engines; [15] lists 5 popular wiki engines; but these two 5-engine lists have only 2 engines in common. 2.3 Wiki page interchange 3 One much-discussed feature of the future is the ability to I don’t mean to claim that these lists are authoritative or exhaustive, but they do show that it is not the case that programmatically move or copy a page from one wiki to aggregate developer effort is being concentrated on two or another one, even if the source and destination wiki use three wiki engines different wiki software [17, 39, 27, 1, 16, 29, 7, 36]. This 4 “Unified recent changes” is a single list of the Re- would require two things: centChanges of multiple wikis. Client-side wiki software has not yet been widely developed wiki administrator because, before WikiGateway, it was difficult for client-side must programs to communicate with the wiki server.