Zope: Content Management Framework

An internship report submitted in partial satisfaction of the requirements for the degree of

MASTER OF SCIENCE in APPLIED COMPUTER SCIENCE

By

Ramakrishna Reddy Enugala Matriculation # 1338250 MS in Applied Computer Science [email protected]

Internship done at Chair of Computer Architecture Institute of Computer Science, Albert Ludwigs University, Georges Köhler Allee, Gebäude 51 D-79110 Freiburg im Breisgau Germany

Introduction I have done my internship in The Chair of Computer Architecture. The Duration of my internship was 10 weeks, beginning on 5th of August and lasted up to 12th October, 2002. During this period I was working under the guidance of Prof. Dr. Bernd Becker and Dipl.-Inf. Marc Herbstritt. My task during this period was to get familiar with ZOPE and to verify whether or not the features provided by ZOPE satisfy the needs of the Chair of Computer Architecture to setup a Content management System (CMS). In order to accomplish this task I developed a beta site( on the local server) using ZOPE to test whether or not it meets our requirements. Desirable changes in CMS(Implementing the dynamic menu structure, etc.) were done using PYTHON programming language.

Enterprises today have lot of content to deal with. This includes information generated by different departments, divisions and research facilities, corporate policies and procedures, product overviews and presentations, press releases, etc.

A good Content Management System must make it possible for teams or individuals in varied roles to collaboratively add, edit and delete content. Some of the basic functionalities that cover this are - role based security schemes, content syndication, catalogs, indexing, searching, and workflow.

Content Management System

Content Management System is to manage the content of the website, either in the form of text, image, animation, file etc. What it means by managing is that it enables the non-technical people to manage the content of the website and they should not have to know anything beyond viewing, typing and clicking. It offers more than just a tool to manage content. It delivers a comprehensive set of solution that will enable the non-technical person to control website easily and efficiently.

Zope

Zope is a open source web application server, written in Python. It is a scalable, stable, powerful system that includes an object database, a web server and several templating languages. Zope is developed and supported primarily by Zope Corporation, but also by many developers worldwide

Python

Python is a powerful, interpreted, interactive, object-oriented programming language. Python is open source and can run on almost any platform or system. Zope is written primarily in Python, with some optimisations in C.

Content Management Framework (CMF)

The Content Management Framework (CMF) is an application that contains a series of tools for Zope. These tools form a framework that provide many of the key services a Content Management System would need. The CMF can be used as a standalone product, or in Plone's case, built on top of. The CMF provides core tools like Workflow, Personalisation and Cataloguing. The CMF development is lead by Zope Corporation and is an open source product that benefits from the input and hard work of many developers around the world.

Plone

Plone is a free, open source Content Management System. The focus of Plone is to provide value at every level of an organization. It comes with a workflow engine, pre-configured security and roles, a set of content types and multi-lingual support. There are many developers, writers and testers from all over the world, contributing to Plone everyday. Plone is based on the Content Management Framework.

Following in this document are the detailed features present in ZOPE which I have gone through during the internship period. Zope: Content Management Framework

Zope is often labelled a web application server, but its main strength and organizing principle stems directly from its name - Z Object Publishing Environment. Centred around the Python scripting language, Zope’s main reason for existence is to publish "objects" - which can be plain pages, structured content or whatever else we can put on the Web.

Zope didn't seem robust enough to compete with Java or Vignette for enterprise applications, and wasn't simple enough to take the place of PHP for throw-away mini-applications. The core product, was built by talented programmers working with consistent web design philosophies, including:

• everything should be an object

• objects should be manageable through the web

• objects should be able to easily inherit properties and methods from each other

A large and dedicated user community has sprung up around Zope, providing support, guidance, and extensions in the form of "products" (essentially a collection of Zope classes that work together). This is generally a benefit, but it can be difficult to penetrate the morass as a beginner. As we might expect, there is variation in the extent to which the original design principles are understood and upheld, both in documentation and development. There’s good stuff in there, but it takes digging and community support to find. Zope is neither strictly commercial nor open source; the broader Zope code base exists in several states at once:

• An open source code release that is a publicly available server we can download from www.zope.org and install freely on our own machine.

• Open source extensions ("products") contributed by developers, both employees of the Zope Corporation and third parties.

• Commercial extensions (both "products" and other plugins) developed by the Zope Corporation, such as the ZEO clustering plug in, which enables several distributed ZServers to act as one. Zope the Product has been released into open source by Zope the Corporation (a US-based consulting firm formerly known as Digital Creations). the core product is free, Zope Corp. services are commercial, and there are a large number of privately developed object classes that can come with a commercial consulting contract.

On the one hand, since Zope is open source, we can address problems ourselves. On the other hand, Python developers are still scarce in the marketplace, so we may have to rely on public Product releases (which generally seem to be well supported by their authors).

The Python development experience has its own ups and downs. Python is not a hard language to pick up - it’s elegant, not particularly hung up on syntax, and decent documentation abounds. However, hardcore Python programmers tend to be among the most zealous -- topping even Perl gurus for platform dedication. Therefore, Python tends to be a very self-contained universe.

The lack of good Zope documentation is a problem. In the online docs, and even in the ZWACK (Zope Web Application Construction Kit) book, the people explaining Zope can't seem to decide if it's about hardcore development or rapid prototyping without programming. This indecision makes the documentation situation troublesome, since there is no concise, centralized explanation of many of the features.

Most of the documentation that's missing is that critical next level where we understand the basics, but aren't sure where to go next, because most of what exists is of one of two flavours: 1) "I’m new to Zope but here’s what I love about it" and 2) "I’m a Zope expert; here’s a highly detailed analysis of one complicated part that we won’t understand unless we’re also an expert." the problem is that there are very straightforward questions, the answers should be easy to find (particularly in the FAQs, which are woefully inadequate), and they often weren't.

Installation and Configuration

Installing the server can be trivial or complicated, depending on what configuration we want and what else we need to integrate with it. If we only want to use the ZServer (a python-based free webserver bundled with Zope), installation is as simple as unpacking the distribution, running the install script or Windows Installer, and starting the server. Installing Zope with an existing webserver is trickier, and requires some wrangling to get everything working. It also requires some up-front planning about where we want to do authentication - in the webserver or in Zope. there is no common authentication/authorization mechanism for the webserver authentication configuration. That is, if we want to have users that have permissions in Zope, and we also want to have the webserver authenticate them, we need to maintain two separate lists. With that exception, once we get everything hooked up, it seems to work fine, with the caveat that we don’t get Zserver’s additional FTP and WebDAV functionality if we are fronting our system with another webserver.

In practice, we'll never use a Zope installation out of the box. There are several products which we'll want to install immediately, since they provide functionality that will probably eventually end up in the core product in future versions. Zope is a product continually under development by a dedicated and vocal community, so expect this to always be the case.

Zope is a mostly self-contained web publishing environment in which every item - document, image, binary file, folder, whatever - is a first-class object derived from a base object class. Instead of storing its basic documents in the file system or relational database (although developers can access both of these with freely available objects), Zope puts everything in its Zope Object Database (ZODB). This way, every entity in the system can have properties and methods, security settings, and managed transaction support (including rollback).

Objects and Zope

Many of the built-in objects are simple - they represent files or documents. The basic "page" object type is currently a "DTML" document. DTML is a superset of HTML with some custom tags, which is interpreted and rendered out to HTML at page-view time. However, it looks like the user community will gradually phase DTML out in favour of the superior "Zope Page Templates" (ZPT) model.

Although they don’t currently offer all of the functionality of DTML pages, ZPT objects have the advantage that all of the control structures (which dictate tag and/or content substitution) are embedded in the source document as XHTML attributes instead of custom tags. This means that it is possible to take a source HTML document and put some dynamic code in it without overwriting any of the sample page data. For example, our designer can load the page in Dream weaver (or other WYSIWYG editor) using Zope’s WebDAV interface, make modifications to the layout (with all of their original sample text), bring it back into Zope, and it will still work in a dynamic way, assuming that the designer didn’t remove any of the tags containing special attributes.

This still isn’t bullet proof roundtrip editing, but it’s a few steps closer than most. There’s no reason to stick to DTML and ZPT, though - "pages" can take the form of just about any object we have installed, and can be extended in a number of different ways.

Some objects, like folders, can have children. While this may seem trivial, it is actually critical to the way objects referenced from within other objects are found, because Zope objects support inheritance - what Zope calls "acquisition". When we refer to an object, say, from within a DTML document, the system first looks for that object inside the object we're in. If it doesn't find it, it travels back up the hierarchy of objects until it does. Importantly, this means that local objects override more global objects, allowing developers easy flexibility and separation of site properties across multiple objects, which are all assembled at page view time.

Acquisition makes it easy to create objects that provide exactly the functionality they need and no more, having them inherit other functionality from objects higher up in the acquisition path. While developers used to functional or procedural systems might label this as a problem - "we can break an object by moving it to a different place in the hierarchy!", Zope developers instead view it as an advantage - "we can give an object an entirely new set of behaviours simply by moving it to a different place in the hierarchy!". Like any other feature, this can be good and bad, and the trick is in understanding how it works so we know how to fix it when it does break.

As standard features of OO systems, these concepts pervade Zope. The typical illustration of this is having a DTML document that represents the header element of our site. we can keep the global version in a top-level folder, but have local versions for each subsection which get automatically applied to all pages in that section.

Everything in Zope is an object, this same concept applies equally across the board - for SQL queries, images, etc… This approach is made even more powerful by the presence of Z Classes - the ability to use the ZMI to create new content types using only the web interface. Content and Zope

Since Zope includes a transactional object database, the platform encourages storage of data there. For a standalone Zope application, this is probably sufficient, and not a bad way to run - we get access to all of our data through Zope objects, the Zope Management Interface, and our Python code; we can move data from one server to another using the built-in import/export mechanism; and we get access control, transaction support, and display for free.

Nevertheless, if we have existing data, plan to integrate with other data sources, or need relational integrity, Zope can still accommodate us. Although much of this functionality isn’t in the core product, it is freely available. the Product archive contains Products for interfacing with a variety of servers, including Oracle, MySQL, PostgreSQL, ODBC, SAP DB, MS Access (Jet), CGI scripts, arbitrary URL pattern matching, and many others. XML support is similarly added through a number of add-on products.

Zope allows us to manage a variety of object types and publish them to the web. But publishing alone does not a CMS make.

In order to service the requirements of an organization, a platform also needs a methodology. It must support metadata constructs that make it possible for multiple people (or one person filling multiple roles) to collaboratively add, edit, and manage content. Most of these are mechanisms to classify information and control access to it in different ways - user and group management; permission schemes; syndication; cataloging, indexing, and searching; and workflow. Although a number of Zope “Products” provide these and other functions for working with content, the Zope Content Management Framework (CMF) is the “official” way to extend Zope for CMS, and it is the implementation that will eventually be rolled into the core product.

CMF: Content Management Framework

CMF is a Zope Product that extends the basic Zope server.The CMF began life as the PTK (Portal Toolkit), and its heritage is clear - the most straightforward path from installation to working site is the construction of an aggregation portal, where the site can be browsed anonymously, visitors can create a new account for themselves, then log in and submit content for approval for publication to the rest of the site. "Content" in this case, consists of images, binary files, news items, and general unstructured documents, such as a basic informational page or a note. Additional content types can be added, but doing so in any meaningful way requires programming. Examples of structured content types we would add in this way include FAQs, Articles, Author Information pages, CVs/Resumes, and so forth.

Because the content types can be easily adjusted, and because Zope’s permissioning scheme is adjustable and granular, the CMF can be suited to most kinds of CMS-driven sites, with varying levels of effort.

The easiest place to see a running CMF implementation is the CMF website itself - it's run as a "Dogbowl", where the developers "eat their own dog food". However, if we want to make customisations, we'll of course need to install CMF on our own gear.

The Product installation couldn't be much simpler - we copy the product files to the “Products” directory under our Zope installation and restart the server. For the record, this is almost as easy as it gets, and upgrading from 1.2 to 1.3 was similarly easy. Although the server restart is a drawback in a high-traffic environment, such sites hopefully already have contingency plans for doing round-robin updates, and in any case, the process of upgrading most commercial products requires restarting, so this drawback is not unique to Zope.

Installing the CMF makes some new object types available in our Zope Management Interface, and we can immediately add a CMF subsection to our basic Zope site. The CMF implementation exists as a CMF Site object which grants the CMF functionality to child objects through the normal Zope acquisition mechanism. Any object underneath a CMF Site object can then take advantage of the CMF services, if the CMF is supported by the object’s type. If we want to use the CMF services in the entire site (e.g.: if we want the home page for our site to be a CMF front page), we’ll need to use Zope's virtual hosting capability to map the host root to our CMF Site object.

Like Zope, the CMF immediately offers some features OOTB (Out Of The Box/Out Of TarBall), and we can make changes from there, in the form of extending/copying the existing objects, setting parameters, or writing python/Zope code. As with some other parts of Zope, the documentation lacks an illustrative discussion of what we might want to change and why. As with most of Zope, there is no centralized explanation of the core CMF functionality, and no overview of why the product works the way it does. Nonetheless, after a few days of experimenting, it was possible to gain some appreciation for why people speak of the product with such affection - the product has a very low “install to running site” time, there are already available a wealth of products to extend the CMF, and the developer community is relatively active and responsive.

CMF Services

In addition to making Zope easier to develop for by making strides towards better componentizations in the core product, the CMF provides a number of framework services for us to take advantage of:

• membership services, including allowing users to create their own accounts • extending the available content types, either through Zclasses or python programming • registering content for indexing and cataloguing • making all content types user-discussable (the built-in functionality for this is clunky, but it works).

Workflow is extremely important to any collaborative editing environment, and CMF offers but one choice by default - the simple publish/review cycle of the portal. We’ll immediately want to obtain and install the open-source DCWorkflow product (in addition to being useful in itself, several other products depend on it), which allows web-based modification of workflow states (but not action triggers - some form of scripting is still required). If we have more complex workflow needs, other products, such as OpenFlow, are available as alternatives to building our own, which is of course always an option, but which requires python programming.

The first step in configuring a CMF site is to create the master site object.When we do that, a lot of work is done under the covers. Our site is now configured with an empty user folder, some basic skins, and defaults for registrations, cataloging/search, and workflow. From here, users can create their own accounts (we'll need to modify the security settings to prevent that if we want more controlled user management), browse the site, and log in to start creating content. CMF supports built-in handling for: • “Structured Text” (the Zope alternative to the most basic markup)/DTML/ZPT documents • Images • Files • News Items • Favourites • Links • Folders

Additional, more structured content types are fairly easy to add (there's a detailed description in the ZWACK book, with either Python programming or through Z Classes.

CMF Skinning

On the presentation side, the choice of “skin” affects the look and feel of the site. Under the default configuration, each end user can select their own skin, but of course, the site developers can modify that if desired. Skins can differ widely, both in appearance and in the data displayed.

To understand the presentation level, though, we need to digress a bit into Zope’s object-oriented model. Philosophically, hierarchical acquisition is very elegant and clean, but it makes life difficult if we want to present a different acquisition hierarchy for each user, because objects have to live somewhere in the object hierarchy.

Although not the subject of heavy focus in the docs, CMF skins (managed through the “portal_skins” tool) are essential to the way the CMF presents its content to end users, because they allow developers to break the acquisition hierarchy for inheritance. More than simple UI configurations, CMF skins allow multiple preset configurations for all parts of the site -- in short, multiple object models.

As a rudimentary example, say we have two skins - red and blue, and we want them to each have their own navbar images (in their respective colours), but share common data elements. We would configure an “images” folder for each skin, plus a “common” folder for the data elements, such as a news item box. Then the skin path definitions would be “red, common” for the red skin and “blue, common” for the blue skin. Because we’re using skinning, these folders need not be placed in the object hierarchy above the pages using the skin, as they’d have to be using plain Zope. The interface for managing skins is poorly designed, and I hope it will change in future releases. Ideally, the acquisition/skin path should be navigable as a directed graph, rather than a strict tree.

In addition to the basic skins offered with the core product, some “3rd party” skins are also available. The most polished one so far is “Plone” [http://www.plone.org], which easily converts the fairly unattractive basic CMF site into a prettier portal layout.

The Plone installation process needs some work, and some users have noted performance problems, but the skin is admittedly much nicer than the defaults and worth a look. Like most of Zope, it is an implementation of the CMF. Things we learned from Plone development. Designer and Developer have a ying/yang relationship. For Plone to be successful targeted at a non- programmer audience it is important to understand the differences. • Files should not be stored in the ZODB • DTML development does not scale between Designer and Programmer • CVS/Perforce/anything is better code versioning mechanism than ZODB (some people think ZODB is versioning mechanism!) • Programmers do not care about how things looks, they just want it to look decent. Designers do care about how things look. As long as its minimal fuss programmers will markup HTML in a way Designers can leverage the markup. • Usability is difficult work. On a whole your first impression is probably the best. Focusing on usability is not in the backs of the mind of a programmer. Programmer wants to roll out as much working functionality in as little time as possible. • CMF/ZOPE are extremely complex entities. Insulating people from these complexities is key to end user adoption. ids vs. titles. skinning mechanism. WEBDAV via Web Folders on windows is unreliable. Functionality Most Liked: • Aesthetics - the CSS has been refined continuously and the XHTML has ungone quite a bit of work as well. Plone looks nice. Its very simple to make your own look and feel from simply customizing the stylesheet-proeprties and uploading new icons. • Easy to change - almost everything is a macro. the left column is a macro expansion of main_left_column.pt To customize the left column for the entire website all you do is need to customize this Page Template. • Familiar Paradigm - support a flat heirarchy, where we navigate using breadcrumbs and drilling down into the folder structure. The familiar tree view of a folder structure has been often requested (would like help on implementing this in ZPT). CMFWorkspaces gives us the third and final UI which is inside of one page you create all content types on the same screen. The workflow will shuffle it to the appropriate place. • Out-of-the-box - most people feel that after installing Plone they have very little to customize and (their primary focus shifts to) loading content. This could be one of the first ZOPE products that enjoys this sense of ease. And since Plone is built on top of higher level services than Squishdot. It will be easier to add functionality to Plone without having to learn the innards of ZOPE.

Plone and the CMF

When we say Plone is a implementation of the CMF, it mean CMFDefault is a default implementation of the CMF. It comes with a selection of skins, useful functionality like UrlTool, a specialized MembershipTool as well as a DefaultWorkflow. CMFDefault only required CMFCore to work. CMFDefault is really what we think of when we see the CMF out-of-the-box. Plone extends the CMFDefault in quite a few ways. The most noticeable is the modifications to the portal_skins tool. There will be about five new directories (Filesystem Directory Views) created. There should be 3 new skins added Plone Default, PloneXP and Plone Mozilla.

Plone's main goal is to focus on User Interface design. Along the way for Form validation mechanism there is portal_forms which exposes the Formulator product functionality. Also we have a workflow and assign all objects to it. (which is based off of DCWorkflow, a web configurable workflow product). Simple debugging mechanism, plone_debug which allows you to write info out to the debug log. All in all Plone attempts to make developing inside of CMF much less painful, as well as getting up and running as fast as possible.

Plone also demonstrates in a thorough way the best practices of development in CMF. Recently introduced the CMF Collective will demonstrate snap-in components for the CMF as well as demonstrate Best practices by: declaring Zope interfaces, use PageTemplates and Python Scripts, and have some sort of unit tests. The CMF Collective is interesting because although its starting off as a CMF-centric collective, to motivate people into using the plone as the standard UI. Showing that developing is not merely customizing existing PageTemplates. Customizing is writing our own Products! We need to clear up the perception of writing Products is a difficult or heavy handed thing to do.

Plone is used for the administrative interface but when you look at it from a public side, i.e. http://www.ira.de/ we want it to look like our brand not the default Plone look. • Done by configuring at the ZOPE level • Configuring this at the Folder level (in CMF) • works in conjunction usually with a proxy server, i.e. apache or SQUID

This is how we accomplish it. This would make a nice picture. • Proxy server rewrites url. If its public.xyz.com it goes to /cmf/public. If its admin.xyz.com it goes to /cmf. • /cmf/public has a Access Rule that checks to see if a VirtualRootPhysicalPathx was being used and if so was the folder traversed to public • if the request is inteded for the public folder we bind the Public Skinpath to the portal object • else we bind the Default Skinpath to the portal object. • if you were viewing the public side via public virtual host you get its skin else you get plone skin.

One of the most important places for Plone to shine is in its out of the box presentation. ZOPE for the masses is unacceptably complex and gives people too many options. CMF is unacceptably ugly and abstract. Plone is a nice entry point because it: • Does something immediately out-of-the-box • Provides a semblance of a framework to develop within, from filesystem Python to PageTemplates • Obsessed with User Experience

Other 3rd Party Products/Extending CMF

There is not very much built-in XML support in the core Zope product, and only a little in the CMF extensions. In practice, it seems likely that if we want to work with XML, we’ll be using the ParsedXML object, which remains under development and has not been updated in nearly 10 months. This illustrates a fundamental feature of open source development. if we choose Zope as our platform, we’re buying into a product that is free, but which may not immediately serve all of our needs and which may never do so but which we can coax into fulfilling our immediate requirements.

Zope has been described as a Swiss Army Knife for web development, and the metaphor is fairly apt. There are many useful tools built in, but extending the product beyond that generally means writing an extension in a proprietary environment In this case, the proprietary environment is an open source one, but it is still very different from most of the rest of the web development world.

Two things serve to differentiate Zope in this capacity - Products and ZClasses. Because any object can be designated a Product and easily exported with all of its children, it is relatively trivial to share pieces of code, with little or no configuration required, once a problem has been solved. The “normal” way to code Zope objects is with python classes on the filesystem, but there is also the ZClass extension system, which allows simple extension of existing objects through a web interface. The claim is made that new objects can be created without any programming, but for anything other than the simplest objects, some coding will be required to construct the customized user interfaces for the new object. However, ZClasses are still a good way to quickly prototype new objects.

The Future of CMF

The CMF is clearly still a work in progress, but it is just as clearly making that progress. The product is slated to be included in the core distribution of Zope 3, which is under development now. Zope 3 promises to incorporate a great number of features based on practical use-case analysis, as well as to address the most common complaint about learning Zope - that we have to learn much of the framework before we can make any useful changes. The CMF is a stepping stone to that end, and even if we don’t use all of the functionality, it contains a number of valuable tools that should substantially cut the development time of our CMS project. Conclusion

I am very much satisfied with this internship. For myself I found this internship as an excellent opportunity in enhancing my knowledge by learning many new things about Content Management. I also developed skills in programming with Python during this period. I also gained considerable amount of knowledge about ZOPE.

I would like to thank Prof. Dr. Bernd Becker for providing me this opportunity to do my internship under his guidance. Further I would also like to thank Mr. Marc Herbstritt for his beneficial guidance and support. Lastly I would like to thank the other members of the chair of Computer Architecture, who directly or indirectly provided their help in completion of my work successfully.

Bibliography

Zope Community http://www.zope.org/

Zope Corporation http://www.zope.com/

Python http://www.python.org/

Content Management Framework(CMF) http://cmf.zope.org/

Plone http://www.plone.org/