Notes on Liquibase

Notes on Liquibase Created 12/21/10 Updated 01/04/11, Updated 02/02/11, Updated 04/16/11, Updated 07/12/11, Updated 01/12/12, Updated 02/22/12

Introduction Liquibase is a database migration tool. Available from http://www.liquibase.org/

Liquibase is an open source (Apache 2.0 Licensed), database-independent library for tracking, managing and applying database changes. It is built on a simple premise: All database changes are stored in a human readable yet trackable form and checked into source control.

All changes to the database are stored in XML files and identified by a combination of an "id" and "author" tag as well as the name of the file itself. A list of all applied changes are stored in each database which is consulted on all database updates to determine what new changes need to be applied. As a result, there is no database version number but this approach allows it to work in environments with multiple developers and code branches.

A similar approach to database migration of provided by Flyway from GoogleCode. See our document “Notes on Flyway”.

Features  Over 30 built-in database refactorings  Extensibility to create custom changes  Update database to current version  Rollback last X changes to database  Rollback database changes to particular date/time  Rollback database to "tag"  SQL for Database Updates and Rollbacks can be saved for manual review  Stand-alone IDE and Eclipse plug-in  "Contexts" for including/excluding change sets to execute  Database diff report  Database diff changelog generation  Ability to create changelog to generate an existing database  Database change documentation generation  DBMS Check, user check, and SQL check preconditions  Ability to split change log into multiple files for easier management  Executable via command line, Ant, Maven, Servlet container, or Spring  Support for 10 database systems

Resources The current stable version of the Liquibase Core is 2.0.3 (Released Oct 10, 2011). The 2.0 revision was first released at the end of 2010, and featured major changes to codebase, focusing on extension/integration APIs.

The documentation is online at http://www.liquibase.org/manual/home. This is a well-organized wiki of content.

Concepts The basic approach is that all database schema changes are carried out under control of a file which is a part of your project source code. That file, called a “changeLog” contains set of directives for database changes, and can also contain the information needed to roll back a change.

xmlns="http://www.liquibase.org/xml/ns/dbchangelog/1.3" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog/1.3 http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-1.3.xsd">

The Liquibase library contains code to process this file and apply it. Each changeSet is applied as a transaction. The ids must be sequential and are commonly epoch timestamps.

The operations supported include:  update – performs a forward migration  rollback – reverses a migration  diff – compare a schema in the database with the results of the changeLog.  generateChangeLog – generate the change log to create the current database schema  dbDoc – using change information stored in the change logs and an existing database, LiquiBase can generate database change documentation  SQL output – all update and rollback commands have a “sql output” mode which do not execute anything against the database but instead save the resulting SQL to standard out or a specified file.

These operations provide a comprehensive set of database schema update capabilities. The operations can be carried out at the command line, or as part of an Ant or Maven script (there is also a Grails plugin).

Installing Liquibase Extract liquibase.zip. In it you will find a liquibase-VERSION.jar

This jar is all that is needed to run Liquibase from the command line, Ant, Maven, Spring, or a servlet container. You can run the command line version of the migrator with the same jar file by simply running java -jar liquibase- VERSION.jar.

Tutorial Step 1: Create a Changelog File:

The database changelog file is where all database changes are listed. It is XML based, so start with an empty XML file:

Step 2: Add a ChangeSet

Each change set is uniquely identified by an “id” attribute and an “author” attribute. These two tags, along with the name and package of the changelog file uniquely identify the change. If only an “id” needed to be specified, it would be too easy to accidentally duplicate them, especially when dealing with multiple developers and code branches. Including an “author” attribute minimizes the chances of duplications.

Think of each change set as an atomic change that you want to apply to your database. It's usually best to include just one change in your change set, but more are allowed and can make sense if you are insert multiple rows that should be added as a single transaction. Liquibase will attempt to run each change set as a single transaction, but many databases will silently commit and resume transactions for certain commands (create table, drop table, etc.)

Step 3: Run the ChangeSet

There are many ways to execute your change log including via command line, Ant, Maven, Grails, and a servlet listener. liquibase --driver=com.mysql.jdbc.Driver \ --classpath=/path/to/classes \ --changeLogFile=com/example/db.changelog.xml \ --url="jdbc:mysql://localhost/example" \ --username=user \ --password=asdf \ migrate

Step 4: Check Your Database

You will see that your database now contains a table called “department”. Two other tables are created as well: “databasechangelog” and “databasechangeloglock”. The databasechangelog table contains a list of all the statements that have been run against the database. The databasechangeloglock table is used to make sure two machines don't attempt to modify the database at the same time.

Writing the ChangeLog

Primary directives Add table Alter table Etc.

Preconditions Preconditions can be attached to change logs or changesets to control the execution of an update based on the state of the database.

There are several reasons to use preconditions, including:  Document what assumptions the writers of the changelog had when creating it.  Enforce that those assumptions are not violated by users running the changelog  Perform data checks before performing an unrecoverable change such as drop_Table  Control what changesets are run and not run based on the state of the database

If desired, a precondition can be the only tag in a .

Using LiquiBase Command Line: LiquiBase can be run from the command line by running liquibase [options] [command] [command parameters] (optionally, replace the liquibase command with java -jar ). The command line migrator works well when you want to do migrations on demand, but don't have Ant or Maven available such as on servers. The command line migrator also gives you more control over the process than the Servlet Listener, Ant, or Maven do, allowing you to run maintenance commands like outputting SQL and listing/releasing database changelog locks.

Any values found after the command on the command line invocation will be considered a command parameter. The command line processor will validate whether the command line parameters are allowed for the current command. If the current command does not allow command line parameters or the parameter appears to be an incorrect format, then an error message of 'unexpected command parameter' will be logged and the execution will terminate.

The command line migrator also allows you to  perform rollback operations and generate rollback scripts  generate "diff"s  generate creation scripts from existing databases  generate database change documentation

Ant: LiquiBase can be controlled via ant Tasks. To use, simply add the liquibase.jar to your Ant lib directory or the classpath referenced by “classpathref”.

The following tasks are available in Ant  updateDatabase  rollbackDatabase  rollbackFutureDatabase  tagDatabase  generateChangeLog  diffDatabase  diffDatabaseToChangeLog  dbDoc  changeLogSync  dropAllDatabaseObjects

Maven: to be filled in

Usage Hints Don’t change changeSets that have already been applied. The state that is maintained in the DB is simply the id of the latest change, so Liquibase will not know that the change should be made.

Run Liquibase at the end of a deployment sequence, as part of your Ant or Maven script.

You need to write information into the changeLog for those changes which are other than an add or a forward change.

Grails Plug-In for Liquibase The standard Grails Database Migration Plugin (http://grails-plugins.github.com/grails-database- migration/docs/manual/index.html) is built off Liquibase and is the officially supported Liquibase/Grails integration. This became available in August 2011.

Notes on Current Version The first step is to install the plugin: grails install-plugin database-migration

Typical initial workflow Next you'll need to create an initial changelog. You can use Liquibase XML or the plugin's Groovy DSL for individual files. You can even mix and match; Groovy files can include other Groovy files and Liquibase XML files (but XML files can't include Groovy files).

Depending on the state of your database and code, you have two options; either create a changelog from the database or create it from your domain classes. The decision tends to be based on whether you prefer to design the database and adjust the domain classes to work with it, or to design your domain classes and use Hibernate to create the corresponding database structure.

To create a changelog from the database, use the dbm-generate-changelog script: grails dbm-generate-changelog changelog.groovy or grails dbm-generate-changelog changelog.xml depending on whether you prefer the Groovy DSL or XML. The filename is relative to the changelog base folder, which defaults to grails-app/migrations.

Since the database is already correct, run the dbm-changelog-sync script to record that the changes have already been applied: grails dbm-changelog-sync Running this script is primarily a no-op except that it records the execution(s) in the Liquibase DATABASECHANGELOG table.

To create a changelog from your domain classes, use the dbm-generate-gorm-changelog script: grails dbm-generate-gorm-changelog changelog.groovy or grails dbm-generate-gorm-changelog changelog.xml If you haven't created the database yet, run the dbm-update script to create the corresponding tables: grails dbm-update or the dbm-changelog-sync script if the database is already in sync with your code: grails dbm-changelog-sync

Source control Now you can commit the changelog and the corresponding application code to source control. Other developers can then update and syncronize their databases, and start doing migrations themselves.

Earlier Version (used in Vive) The prior effort was the Grails Liqubase plugin, located at http://grails.org/plugin/liquibase. The current version is 1.9.3.6. This is used in the Vive code (actually, 1.9.3.5 is used). That dates back to 2010 or earlier.

Open Questions/Issues How to handle multiple databases?