Session: F04
DB2 On Rails
Philip Nelson ScotDB Limited
Monday May 7th, 2007 : 16:20 - 17:20.
Platform: DB2 for Linux, UNIX and Windows
Presentation F04 : DB2 on Rails This presentation examines using DB2 with the Ruby on Rails web framework.
Presenter's Biography Philip Nelson has worked with DB2 for z/OS since 1989 and with DB2 on distributed platforms since OS/2 Extended Services acquired a Database Manager component.
On a daily basis he administers DB2 for z/OS, DB2 for AIX, DB2 for Solaris and IMS databases for a large financial organisation in the United Kingdom. He is also the Database Technology Architect within this company.
In addition he is a principal of ScotDB Limited, a consultancy company focused on the SMB market and specializing in DB2 on the Linux platform. Over a number of years he has been central to delivering successful projects for customers in a range of industries. He has also produced and delivered both in house and public training courses on DB2 and related technologies.
He has been involved with IDUG since 1999 and currently is deputy editor of IDUG Solutions Journal.
1 AGENDA
• Introduction to Ruby • Introduction to Ruby on Rails • Integrating Ruby on Rails with DB2 • Developing a Ruby on Rails Application • Using Ruby on Rails with Legacy Databases • Deploying Ruby on Rails Applications • Improving DB2 support for Ruby on Rails • Ruby on Rails resource directory
2
Agenda Firstly we will provide a brief introduction to the Ruby programming language. before looking at the Ruby on Rails web framework and why it is “special”.
Then we will start exploring how DB2 can be used with Ruby on Rails (RoR). We'll show how to enable RoR to use DB2 as its persistance (database backend) layer. We will concentrate on the ibm_db2 driver developed by IBM.
Then we will demonstrate how to develop a RoR application, firstly in a “green field” development and then using an existing “legacy” database. We will describe the process all the way from initial project setup to production deployment.
Along the way we will point out problems encountered and the most critical “areas for improvement” within the DB2 support.
We will finish up a list of resources for RoR developers.
2 What is Ruby ?
z A programming language
z Interpreted
z Object Oriented
z Developed in Japan in the mid 90s (first public release in 1995) by Yakihiro Matsumoto (“Matz”)
z Really became popular in the wider world with the introduction of Ruby on Rails
3
Ruby is an open source, interpreted, object-oriented programming language, developed by Yakihiro Matsumoto (nickname and online handle “Matz”) in the mid 1990s. The first public release was in 1995. However it was virtually unknown outside Japan until David Heinemeier Hansson developed the Ruby on Rails web framework using Ruby as the programming language.
3 What is Ruby on Rails ?
z Web development framework
z Developed by David Heinemeier Hansson
z Came out of his work on developing the Basecamp web-based product (http://www.basecamphq.com)
z Uses Ruby as the underlying language
z Based on the MVC paradigm
z Model : the information layer
z View : the display layer
z Controller : the business logic layer
4
Ruby on Rails is a web development framework based on the MVC (model, view, controller) paradigm. Its initial creator was David Heinemeir Hansson.
David was heavily involved in the creation of a web-based collaboration product called Basecamp (http://www.basecamphq.com). The underlying framework he developed while writing Basecamp formed the basis of Ruby on Rails.
There are many development tools, both for web-based and traditional client-server deployment, based on the MVC paradigm. The underlying idea is to separate display (View), business logic (Controller) and data access (Model) and thus make the application easier to maintain.
Unfortunately many of these tools rely on complex configuration files to tie the three parts together. The Java-based J2EE framework, in particular, become so huge in J2EE 1.4 that there was virtually a “revolt” among developers against the complexity : later releases has sought to simplify things, with such initiatives as allowing POJO (Plain Old Java Objects) to be used instead of requiring an application to always produce EJBs (Enterprise Java Beans). But still the complexity is greater than many people can cope with.
4 Why is Rails “special” ?
z Speed of development
z Built-in support for testing
z Database support : “migrations”
z “Convention over Configuration”
z Greatly reduces the overhead of creating software
z Most benefit if willing to conform to conventions
z Can be overridden if required
5
Ruby on Rails seeks to eliminate much of the configuration file headache by emphasizing “Convention over Configuration”. The idea is that as long as a developer is willing to use the default behaviours for connecting the different parts of an application together there should be little or no need to set up and maintain configuration files. The challenge is to choose sensible default behaviours which meet a wide range of requirements. Because the framework enforced these defaults unless they are overridden explicitly, RoR is sometimes described as “opinionated”. The greatest benefit is therefore obtained when all the conventions are followed, which often means in “green field” (totally new) developments. While it is possible to successfully use RoR for development within an existing “legacy” environment, the further existing data models vary from the Rails conventions the less benefit can be obtained. Other interesting concepts within the RoR framework are extensive support for automated testing and maintenance of database schemas within the application development framework.
5 Rails Components • Rails draws on a number of Ruby modules • Active Support : common libraries • Rake : Ruby “make” • Active Record : model (ORM) • Action Pack : two closely coupled modules • ActionController • ActionView • Action Mailer : email support • Action Web Service (will be a separate plugin in Rails 2.0, due to REST support added in 1.2) • Rails itself is underneath, orchestrating interactions
6
Rails draws on a number of Ruby Gems (modules) to provide essential services. At the core of these are Active Record, which provides the Object-Relational Mapping (ORM) support at the heart of the “model” element. The “controller” and “view” components are supplied by Action Pack, which actually contains two closely couple modules, ActionController and ActionView. There is also a library of commonly used libraries (Active Support), a module to automate tasks (Rake = Ruby Make), a module providing email services (Action Mailer) and a module provide XML-RPC and SOAP web service support (Action Web Service or AWS). The last of these will become an optional plugin to Rails 2.0, since it is considered that this support is not strictly required any more (Rails 1.2 added REST support and this is considered preferable in many circumstances). Most of our interactions with Rails happen at the level of these modules. Rails itself sits underneath, orchestrating everything.
6 Where does DB2 fit in ?
z DB2 “adapter” (driver) from IBM
z Called ibm_db2 (there already is a non-IBM db2 adapter)
z Download from Alphaworks
z http://www.alphaworks.ibm.com/tech/db2onrails
z Very active community around it
z There is also another driver, not developed by IBM
z http://raa.ruby-lang.org/project/ruby-db2
z We will concentrate on the IBM driver here
7
RoR has gained popularity very quickly, and has acquired a very dedicated following. Thankfully there are folks within IBM who “caught the wave” and have set out to try to make support for DB2 as good as possible. As we shall see there is still a lot of work to be done. Before IBM became involved, an open source project had done some work to provide DB2 support, within the Ruby DBI framework. This seems to have progressed at a fairly leisurely pace and the IBM-developed support appears to have overtaken this very quickly. So in this presentation we will concentrate on using the IBM-developed support (called ibm_db2) available for download from IBM Alphaworks.
7 Getting Rails Running (Summary)
z Install Ruby language support
z Install Gems support
z Ruby “Gems” are the names given to add-on packages for Ruby
z Package manager command line called “gem”
z Install Rails Gem
z Windows package includes Ruby, Gem and Rails
8
Before we can start trying to make DB2 work with RoR, we need to actually get RoR up and running. There are number of steps involved in this. First we have to ensure we have the Ruby language available. Then we install a facility for managing Ruby Gems. Gems are a collection of add-on packages for Ruby : in many ways similar to the Perl modules available through CPAN (Comprehensive Perl Archive Network). RoR is packaged as a Ruby Gem, as do all the prerequisites. There are packages available for a number of environments which package Ruby, Gem and Rails together for easy installation. This is particularly necessary in the Microsoft Windows environment, where not everyone has easy access to the C compiler which is required to build Ruby and some gems from source.
8 Installing Ruby Language Support
• Source code available from http://www.ruby- lang.org • Prebuilt packages available for - • Windows (installer from site above) • Linux (packages for most distributions) • All work for this presentation on OpenSuSE 10.2 • Packages ruby-1.8.5-18 and ruby-devel-1.8.5-18 • Macintosh OS-X (multiple options from site above)
9
The Ruby language is open source, and the source code can be downloaded from http://www.ruby-lang.org : build facilities are available for a wide range of environments. For those who want to get up and running quickly, prebuilt packages are available for a number of popular environments. For Windows and Macintosh OS-X there are links to these packages from the main download site above. For Linux, most distributions provide Ruby packages which can be installed using the standard package management tools. To prepare this presentation, OpenSuSE Linux 10.2 was used, and Ruby was installed from standard RPM packages (ruby-1.8.5-18 and ruby-devel-1.8.5.18).
9 Install Gems Support
• Source available from http://rubyforge.org/projects/rubygems/ Setup from source as follows - $ tar xzvf rubygems-0.9.0.tar.gz $ cd rubygems-0.9.0 $ su - # ruby setup.rb • Comes as part of the default Windows and Mac OS X binary install • Packages for some Linux distributions available • On OpenSuSE package rubygems-0.9.2-4.1 10
Again Gems support can be downloaded and built from source, once the Ruby environment has been installed. The default Windows and Macintosh OS-X binary installations already come with Gems support. Some Linux distributions also supply packages : on OpenSuSE 10.2 the current version (early March 2007) is rubygems-0.9.2-4.1. Once installed the Gems support can be updated using the “gem” command line interface to the latest version using the command “gem update –system”.
10 Install Rails Gem (1)
• Some Linux distributions have Rails packages • OpenSuSE has the following - rubygem-activesupport-1.3.1-27 rubygem-actionpack-1.12.5-16 rubygem-activerecord-1.14.4-16 rubygem-rails-1.1.6-17 rubygem-actionwebservice-1.1.6-17 rubygem-actionmailer-1.2.5-16 • Be wary of mixing packages with Gems installed via “gem” command line
11
Some Linux distributions provide packages for Rails and its prerequisites. For example, OpenSuSE provides a series of packages with names starting “rubygem” which provide many of the most popular gems, including Rails. Care should be taken if installing gems using the Linux package manager and then updating them using the “gem” command line, since automatic installation of updated packages could regress the level of support. At the time of writing, the RoR support provided within OpenSuSE is 1.1.6, while the latest stable version is 1.2.2 : 1.2.x has significant improvements over 1.1.x, and some 1.1.x features have been deprecated or have API changes.
11 Install Rails Gem (2)
• Prepackaged Ruby on Rails environments - • Windows : Instant Rails (http://instantrails.rubyforge.org) • Mac OS X : Locomotive (http://locomotive.sourceforge.net) • To install using the “gem” command line - su - gem install rails –include-dependencies • To upgrade to latest version using “gem” su - gem update rails --include-dependencies
12
There are prepackaged RoR environments (including Ruby, Gems and Rails) for Windows (Instant Rails) and Mac OS X (Locomotive). On any environment which already has Ruby and Gems support installed, RoR can be installed from the command line by typing - gem install rails –include-dependencies If Rails has been installed by some other means it can be upgraded to the latest stable release by typing - gem update rails –include-dependencies
12 Adding DB2 Support (1)
z Get package from Alphaworks
z Source is DB2onRails_V2.1_Source.tar.gz (500k)
z Or, for Windows only, download complete package with DB2 Express-C for Windows (344 Mb)
z Here we concentrate on building from source (Linux)
z Unpack to temporary directory and move into it
z mkdir db2rails;cd db2rails
z tar -zxvf DB2onRails_V2.1_Source.tar.gz
z cd Source 13
DB2 support for Rails needs to be added manually (it isn't currently included in the standard Rails components). The IBM-developed DB2 components are currently available from IBM Alphaworks (see “Online Resources” at the end of this presentation for web address). Two downloads are offered : the source code for the DB2 adapter, and a complete environment for Windows, including DB2 Express-C (a download of nearly 350 megabytes). Earlier releases provided a Windows binary as part of the source install, but this option has been removed. Here we will concentrate on building the package for Linux. Create a directory to hold the source code and expanding the tarball into it. Change into the “Source” subdirectory.
13 Adding DB2 Support (2)
• Edit file config.yml with connection information source: test_dir: tests connection: database: DBAIR001 user: db2inst1 password: mypass hostname: localhost port: 50001 • Database must be UTF-8 and territory US or tests fail 14
A database is required to run the test suite for the DB2 driver. Unfortunately, not enough consideration has been given to the worldwide audience for this code, or for the fact that both Ruby and RoR started life outside North America. Unless the database has been created as a UNICODE (UTF-8) database with “TERRITORY US” the test suite fails in a number of places. Most of the errors seem to relate to strings representing dates being in American format. Once an appropriate database has been created, details of this database must be supplied in file config.yml before starting the build process. The database name, plus location (hostname / port) and authentication (user / password) details, must be supplied in this file.
14 Adding DB2 Support (3)
z Run “rake lib” to build library
z Running “rake” alone can have problems with non-US databases, due to tests using US date formats
z Copy ibm_db2.so to /usr/lib/ruby/1.8/i586-linux
z Copy ibm_db2_adapter.rb to /usr/lib/ruby/gems/1.8/gems/activerecord- 1.14.4/lib/active_record/connection_adapters
z (adapt versions to suit those you have installed)
15
Once the config.yml file has been edited appropriately, simply run “rake” to build the DB2 support and run the test suite against the specified database. If you simply want to build without running the test suite then run “rake lib”. Unfortunately the “Rakefile” doesn't have “rake install” option to install the DB2 support into the correct places, so this has to be done manually. There are two files which must be copied - Firstly ibm_db2.so, a shared library built from C code, must be copied into the subdirectory of the Ruby install set aside for operating system specific code. Then a code file written in Ruby must be moved into the directory containing database specific connection adaptor code within the Active Record Gem structure. Because DB2 support is not shipped as part of the default Rails install, any time Rails is upgraded the whole DB2 install process must be repeated.
15 Adding DB2 Support (4)
z Add ibm_db2 to supported adapters
z Edit /usr/lib/ruby/gems/1.8/gems/activerecord- 1.14.4/lib/activerecord.rb
z Find RAILS_CONNECTION_ADAPTERS
z Add ibm_db2 to the end of the list
16
The next thing we have to do is make Active Record aware of the new connection adapter which has been added. We must edit the main Active Record script (activerecord.rb). Within this script there is a variable RAILS_CONNECTION_ADAPTER which contains a list of all the available drivers, and ibm_db2 should be added to this list. Note that when Active Record is upgraded this file will be replaced and so the first hint that there is a problem is when Rails starts and and reports that ibm_db2 is not among its list of supported connection adapters.
16 Adding DB2 Generator Support
z Add DB2 to list of databases supported
z As root, go to rails install directory
z cd /usr/lib/ruby/gems/1.8/gems/rails-x.x.x
z Edit application generator code file -
z cd lib/rails_generator/generators/applications/app/
z edit app_generator.rb
z Find DATABASES and add ibm_db2 to the list
z Then add configuration template for DB2 -
z cd configs/databases
z edit ibm_db2.yml
z Contents of file on next page ... 17
Next we will set up Rails so that it can generate configuration files specific to DB2. By default Rails assumes that we will be connecting to a MySQL database. Within the Rails Gem are there is a file called app_generator.rb : this contains a variable called DATABASES which lists the databases for which we can generate Rails projects. Add ibm_db2 to the list. Then we must set up a DB2-specific connection template. The templates are found in the configs/databases subdirectory of the Rails Gem. We must add a file called ibm_db2.yml to those already there. These “yml” files are written in a simple macro language called YAML (Yet Another Macro Language), which is used extensively throughout the Rails environment : we will come across more of these when we are overriding default Rails behaviour. My ibm_db2.yml file contents are found on the next slide.
17 DB2 Config Template File (1)
# DB2 (default setup). Versions 9.1 is recommended (V8.2 will work). # Remember to Install the DB2 driver (ibm_db2) from Alphaworks defaults: &defaults adapter: ibm_db2 schema: DB<%= app_name %> host: localhost port: 50000 development: database: DD<%= app_name %> username: db2inst1 password: password <<: *defaults
18
This is the first of two slides which shows the complete contents of my ibm_db2.yml file. The first thing we need to notice is the “<%= app_name %>” definition. This is a variable which will be fed in during the creation of a Rails project. Here we have not done very much with the variable, and so it would be possible to feed in application names which would result in invalid DB2 schema and database names (basically if the application was longer than 6 characters this would be the case). Some of the more advanced features of YAML could be use to prevent this happening. The first section of the config file is a set default values. Note that indentations are very important in YAML : anything starting in position is the start of a new section (so the second section is the line containing “development:”). This first area is a set of definitions used multiple times within the file, and brought in using the line “ <<: *defaults”. Rails then defines by default three databases,which correspond to different parts of the software development life cycle. The first one, shown here, is called “development” and is where all the application development work is done.
18 DB2 Config Template File (2)
# Warning: The database defined as 'test' will be erased and # re-generated from your development database when you run 'rake'. # Do not set this db to the same as development or production. test: database: DT<%= app_name %> username: db2inst1 password: password <<: *defaults production: database: DP<%= app_name %> username: db2inst1 password: password
<<: *defaults 19
Here we see the other two default Rails database definitions. The “test” database is used when we run automatic tests. You should never put anything into this that you do not wish to lose, since when the test suite is run the contents of this database (data and structures) will be cleared down and refreshed. The structure is copied from the “development” database. The data loaded is defined within the test suite (see later). The last definition is for the “production” database. Here we have simply set this up like the other sections. In the real world we would not want to put production login credentials into a publicly accessible file, so only the basic details are normally supplied here and a Capistrano rule (see later) is often used to merge these in at deployment time. Capistrano is a tool, written in Ruby, which is designed to managed automated deployment of applications (it can be used for any kind of application, not just RoR applications).
19 Starting a Project (1)
z Type “rails appname --database=ibm_db2” (e.g. rails air001 --database=ibm_db2)
z cd appname (e.g. cd air001)
z Check config/database.yml and edit if needed
z Create database for at least “development” CREATE DATABASE DDAIR001 AUTOMATIC STORAGE YES ON /db2data/db2db/ddair001 DBPATH ON /db2data/db2db/ddair001 USING CODESET UTF-8 TERRITORY GB CATALOG TABLESPACE MANAGED BY AUTOMATIC STORAGE USER TABLESPACE MANAGED BY AUTOMATIC STORAGE TEMPORARY TABLESPACE MANAGED BY AUTOMATIC STORAGE
20
We now are in a position to start developing our first Ruby on Rails application. To generate the basic framework of a Rails application we issue the command - rails appname –database=ibm_db2 Using the template we have previously defined for the database config, appname should be six characters or less. The way my template is set up conforms to a set of naming standards I have used for a long time. This one command generates a directory called appname, and a series of subdirectories and files which together form the basic framework of a Rails web application. Among the files created is config/database.yml, which is produced using the template file we set up earlier. If we had not specified “--database=ibm_db2” the configuration file produced would have been for MySQL. Before we can go any further we must create at least the database specified in the “development” section of this configuration file. I've given the DDL I used to do this. We could have added this DDL as the first step of the first database “migration” (see later) we produced : however in this case we will just create it manually, as we are going to see what has been produced by default by this one command and we need this database to start the web application server.
20 Starting a Project (2)
• Initial development uses db under “development” area of config/database.yml • So “test” and “production” not needed right away • “test” db will be recreated from “development” (so don't store anything you want to keep in it) • During development we use a bundled web server • Called “WEBrick” • Written in Ruby • We will not use it for production (see later)
21
Rails comes with a bundled web server called WEBrick, written in Ruby. This is not robust, or fast, enough for production use but is excellent for development purposes.
21 Testing Generated Framework (2)
z Start the WEBrick web server -
scotdb@mahler:~/rails/air001>script/server => Booting WEBrick... => Rails application started on http://0.0.0.0:3000 => Ctrl-C to shutdown server; call with --help for options [2006-11-17 14:07:41] INFO WEBrick 1.3.1 [2006-11-17 14:07:41] INFO ruby 1.8.4 (2005-12-24) [i586-linux] [2006-11-17 14:07:41] INFO WEBrick::HTTPServer#start: pid=5888 port=3000 127.0.0.1 - - [17/Nov/2006:14:07:52 GMT] "GET / HTTP/1.1" 200 7552 22
Having checked that the generated config/database.yml is correct, and having created our “development” database we now want to start our WEBrick server to see what we have produced. When we are running any commands within a Rails environment we should always be in the application's “base” directory. Confirming we are in this directory we simply issue the command “script/server” and the WEBrick server starts up. By default it listens on port 3000, so we must make this port available if we are running any firewall or security software. To close down the server when we are finished we simply hit Ctrl-C. The script/ subdirectory of a Rails application contains a number of useful scripts, including an interactive command processor (script/console) and facilities for adding and removing various components of an application (script/generate and script/destroy).
22 Testing Generated Framework(2)
23
Now when we point a web browser at port 3000 (the WEBrick server) we can see that quite a lot has been produced. We have a very simple web application, with instructions on how to start building our own content. Let's move on to do just that ....
23 Building Rails Applications
• First we show building a “greenfield” application • We will make use of all RoR “convention over configuration” support to make things easier • Then we will look at developing against a legacy DB • We will demonstrate how to override conventions to make things fit
24
We will start our exploration of developing Rails applications by building a “green field” application : one where we are starting from scratch, and therefore can follow the Rails default conventions all the way. We will see that we can therefore very quickly produce useable applications. We will then look at developing a similar application against a “legacy” database which does not follow Rails conventions. We will show some of the ways we can override these conventions to make our existing database and our new Rails application fit together.
24 Application Description
• Aircraft Enthusiast's System • Records history of aircraft • Records when aircraft have been seen • Will allow the following actions - • Add / update / delete airport visit • Add / update / delete aircraft sightings on visits • Add / update / delete basic aircraft details • Add / update / delete aircraft identity history
25
Here we describe the application we are going to develop. I've rewritten this application over the years in many programming languages, including dBASE, Perl, Java and now Ruby. It allows me to try out new technologies within a known set of requirements. The application we are going to develop is a simple maintenance application, typical of many which are found in most businesses everywhere.
25 Application Data Model
26
This is the underlying data model for the application, with table names from its current “production” implementation (using DB2 with Perl DBD::DB2). The most common types of database relationships are represented : parent/child and a many-to-many relationship with a resolver table. In this definition we have a number of things which do not follow Rails conventions - •Table names •Primary Keys not called simply “ID” •Composite Primary Keys When we develop our “green field” application, we will modify this model to be totally “Rails compliant”. Then we will come back and attempt to write an application which talks to the original “non-compliant” application.
26 A Word About “Conventions”
• The following are Rails conventions • Each table will have a surrogate key called “ID” • Models are singular : associated tables are plural • e.g. model “product” expects a table “products” • Rails copes with many common “strange plurals” (but not all, as we shall see) • We will modify the data model to follow these
27
Here are some of the Rails conventions we are going to follow - •Every table will have a surrogate primary key called simply “ID”. •This ID column will have new values generated automatically •Names of models are singular, their associated table names are plural The last of these conventions is interesting. The reason for this is so that discussion about the application should follow normal linguistic conventions : for example, we would talk about inserting a Product (instance of class defining a model) into products (table). Rails supports many of the strange plurals we have in the English language (but not all of them). It only seems to provide support for English plurals, not for any other language.
27 Generating a Model
• Let's start by creating the AIRCRAFT model ~/rails/air001> script/generate model aircraft exists app/models/ exists test/unit/ exists test/fixtures/ create app/models/aircraft.rb create test/unit/aircraft_test.rb create test/fixtures/aircrafts.yml create db/migrate create db/migrate/001_create_aircrafts.rb 28
As we mentioned previous there is a script to allow us to generate application components. We are going to start by generating a model for the AIRCRAFT entity within our data model. From the application “base directory” we run - script/generate model aircraft This produces a number of files, which we will describe in more detail here. Firstly under the app/models/ subdirectory it creates a file aircraft.rb, which contains the definition of the Aircraft class which implements the model. We then see two files providing the basis for automated tests of the model. Under test/unit/ the file aircraft_test.rb is where we code tests for the Aircraft model. And under test/fixtures there is a file aircrafts.yml (a YAML macro file) where we define test data to go with these tests. Lastly under db/migrate there is a file 001_create_aircrafts.rb : this is a database “migration” file, where we define the table backing the model. We should also have spotted our first problem. Rails has come across a non- standard plural it does not know about ...
28 Our First Problem : “AIRCRAFTS” • The plural of AIRCRAFT is AIRCRAFT !!! • Undo the generation (do first so everything goes) - script/destroy model aircraft • Edit config/environment.rb to add this exception - Inflector.inflections do |inflect| inflect.irregular 'aircraft', 'aircraft' end • Then regenerate again - script/generate model aircraft • Everything is now generated “correctly” !!!
29
... of course the plural of Aircraft is Aircraft (not Aircrafts). Thankfully we can add to Rails knowledge of strange plurals. But first we want to undo the damage we have already done : we should do this before fixing the problem otherwise it will unpick everything apart from the bad plurals. Run - script/destroy model aircraft and all the files for the model we have just created will be removed again. Now we add some code to config/environment.rb to “inflect” the strange plural. There are a number of options available, including using regular expressions, but here we have simply defined the two way translation between singular and plural. Having done that we repeat the “script/generate” and this time we have correctly named files throughout.
29 Generate the Other Models
• Create the models for the other tables • Ensure you do this in an RI-friendly order script/generate model aircraft_history script/generate model airport_visit script/generate model aircraft_sighting script/generate model unknown_aircraft_sighting
30
Now we have successfully added one model, we generate the models for the other four models in our simple application. If we intend to use the “migrate” (database maintenance) facilities of Rails we need to ensure that we create the models in an “RI-friendly” order. While not strictly necessary, it does allow us to define all the foreign key creation in the migration scripts associated with the appropriate tables, rather than bunching them all together at the end. Therefore we should really have a good idea of the entities making up our data model before we start. It is possible to wind back and roll forward model and database table creations, using “script/destroy” (as previously seen) and some facilities within the migration support, but we risk losing customisations we have made in the process.
30 Creating Tables Using “Migrations”
• Directory db/migrate contains “migration” scripts • Each contains “create” and “drop” code • They will be used in numerical order 001_create_aircraft.rb 002_create_aircraft_histories.rb 003_create_airport_visits.rb 004_create_aircraft_sightings.rb 005_create_unknown_aircraft_sightings.rb • Let's examine one of these ....
31
Now to start looking at database “migrations”. If we look in the db/migrate subdirectory we will see one file for each of models we have defined. We will notice that the file names all start with a three digit number : this is significant, since it is the order which the Rails migration process will process them. We can add our own files directly into this directory (for example, to add some static data to some tables or make some table alterations) but for the automated processes to work they should always be in ascending order without any gaps in the sequence. Now we are going to look inside one of these files to see what Rails has done ...
31 Generated Migration Template
class CreateAircraft < ActiveRecord::Migration def self.up create_table :aircraft do |t| end end
def self.down drop_table :aircraft end end
32
This is the default migration template for creating the table to back the Aircraft model. It is a class, named by concatenating the literal “Create” and the model class name (“Aircraft”) and having as its parent class ActiveRecord::Migration. There are two methods defined : “up” and “down”. The “up” method is a serious of steps to make to add database objects and the “down” method is a series of steps to unpick (rollback) the “up” steps. Since this migration class was generated when we produced a model, the default “up” action is to call “create_table” method of ActiveRecord::Migration. As of yet there are no column definitions present. The default “down” action is to call the “drop_table” method of ActiveRecord::Migration. Note that create_table and drop_table are methods (function calls). Ruby does not require brackets round the parameters, and this “feature” is being leveraged to make the function call read almost like DDL. Now let us move on to add some addition information ...
32 Updated Migration For Table
class CreateAircraft < ActiveRecord::Migration def self.up create_table :aircraft, :options => ' IN SAIR001 ' do |t| t.column :aircraft_type , :string , :null => false, :limit => 100 t.column :constructors_number, :string , :null => false, :limit => 50 t.column :line_number , :integer t.column :aircraft_status , :boolean Adds table end end def self.down drop_table :aircraft Removes table end end
• Here we call methods from ActiveRecord::Migration • Note addition of tablespace details in “:options” hash • SQL statements can be included via “execute” method ... 33
We have added a number of things to the generated template. Firstly we have added an “:options” hash to the create_table call. This supplies database specific DDL to be added at the end of the generated “CREATE TABLE” statement. Then we have added definitions of all the columns in the table, including their data types and other parameters. Note that the data types involved are Rails- defined, not database-specific : and in some cases we are defining data types not even supported by DB2 (e.g. boolean). The definition of the table is very readable. You can go a long way in developing RoR applications without understanding the complex Ruby facilities used to underpin this readability : most non-Rubyists would not even realise that from “create_table” down to after the first “end” is a huge method call. And that's one of the beauties of RoR : it uses the power of Ruby to hide complexity from the vast majority of users. But that power is all there for folks who want to add functionality (as in the composite_primary_keys support module we will look at later).
33 Another Example (With DDL)
class CreateAircraftHistories < ActiveRecord::Migration def self.up execute “create tablespace sair002 managed by automatic storage” create_table :aircraft_histories, :options => ' IN SAIR002 ' do |t| t.column :aircraft_id , :integer, :null => false t.column :history_sequence, :integer, :null => false t.column :registration , :string , :null => false, :length => 10 t.column :aircraft_details, :string , :null => false, :length => 250 end execute "alter table dbair001.aircraft_histories add constraint r002001a foreign key (aircraft_id) references dbair001.aircraft (id) <== Foreign key DDL on delete restrict" end def self.down drop_table :aircraft_histories end end 34
Here's another example of a migration class definition.
The big thing to notice here is that we have added a couple of “execute” method calls. This is a “pass through” method, which allows us to execute any DDL directly on the database : in this case we are running “CREATE TABLESPACE”, which is extremely DB2-specific. We are also defining a foreign key using the “ALTER TABLE” statement. We notice in this DDL that the tables have a schema of DBAIR001, and we have to specify this in the raw DDL. However the create_table and drop_table methods do not have this specified : it picks this up from the config/database.yml file we generated right at the start of the project. And the observant among you will have noticed that we really should have an 'execute “DROP TABLESPACE SAIR002”' at the end of the “down” method, so that we totally unpick all we have down in the “up” method (and make the process repeatable).
34 Running a Migration
scotdb@mahler:~/rails/air001> rake db:migrate VERSION=1 (in /home/scotdb/rails/air001) == CreateAircraft: migrating ======-- execute("create tablespace sair001 managed by automatic storage") -> 2.1619s -- create_table(:aircraft, {:options=>" IN SAIR001 "}) -> 0.2193s == CreateAircraft: migrated (2.3815s) ======
• VERSION parameter (optional) is used to go to a specific point in migration tree (based on numeric prefix of script) • To go back specify a version earlier than the current one • Current position held in table
Now let us see how we execute these migrations. There is a rake task defined for this. To execute this task we run - rake db:migrate This default command will process all the migrations in the db/migrate directory in numeric order. In this case we actually specified an extra parameter to stop at a particular point (VERSION=1), so we only run the first script. Rails, under the covers, has created a table called SCHEMA_INFO to record the current position of the migration process. It is possible to go back to a previous position as well as move forward, assuming the “up” and “down” methods in all the migrations are all mirrors of each other. And you can see we have added the tablespace creation code into this first migration script too.
35 Generating a Basic Controller scotdb@mahler:~/rails/air001> script/generate controller aircraft exists app/controllers/ exists app/helpers/ create app/views/aircraft exists test/functional/ create app/controllers/aircraft_controller.rb create test/functional/aircraft_controller_test.rb create app/helpers/aircraft_helper.rb
• Generates three files - • aircraft_controller.rb : the controller itself • aircraft_controller_test.rb : functional test script • aircraft_helper.rb : helper methods (which assist views) • Note no view files have been produced
36
Now that we have the model / table defined we move on to generating the controller. The controller is where the business logic of the application will be implemented. Again we use the “script/generate” command.
Three files are created - app/controllers/aircraft_controller.rb : the controller code itself test/functional/aircraft_controller_test.rb : tests of controller functionality app/helpers/aircraft_helper.rb : helper methods (assisting views)
Note that no view (display logic) files have been produced.
36 A Basic Table Maintenance App • Edit aircraft_controller.rb as highlighted -
class AircraftController < ApplicationController scaffold :aircraft end
• The one line table maintenance application • A “scaffold” is a basic application dynamically generated at runtime • Now start WEBrick (“script/server”) and point web browser at http://localhost:3000/aircraft/
..... (see over)
37
Opening up the aircraft_controller.rb file, we see that it defines a class called AircraftController, which is derived from ApplicationController.
To very quickly produce a maintenance application, we can use “dynamic scaffolding”. By simply adding the line “scaffold :aircraft” to the generated code we produce something workable (see next page)
37 The Application Works ...
Note hiperlinks for “New Aircraft” “Show” “Edit” “Destroy”
It isn't fancy but it works (I've added a record)
38
Here is the result of that one line edit.
It has the ability to view, edit, delete and create aircraft records.
The only problem is that there isn't any means of controlling the views presented.
38 Changing The Display (View)
• The “dynamic scaffold” generates views at runtime • Great for prototyping • Will always display the latest table structure • Output is very untidy (and gets worse as number of columns increases) • To be able to change the way the output looks we must generate static view code • Will be identical to dynamically generated ones • But can then be edited
39
The “dynamic scaffolding” command generates the views at runtime, based on the current table definitions. This is great for prototyping. However there is no way of controlling the format of the display : and for more than a few short columns in a table the view becomes extremely ugly.
We need to generate the view code into static views, which we can then edit as required.
39 Generating The Scaffold View
scotdb@mahler:~/rails/air001> script/generate scaffold aircraft ... dependency model ... create app/models/aircraft.rb create test/unit/aircraft_test.rb create test/fixtures/aircraft.yml create app/views/aircraft/_form.rhtml create app/views/aircraft/list.rhtml create app/views/aircraft/show.rhtml <== View Templates create app/views/aircraft/new.rhtml create app/views/aircraft/edit.rhtml create app/controllers/aircraft_controller.rb create test/functional/aircraft_controller_test.rb create app/helpers/aircraft_helper.rb create app/views/layouts/aircraft.rhtml create public/stylesheets/scaffold.css
40
Here we repeat the “script/generate” process. This time however, instead of using the keyword “controller” we use “scaffold”. This time we see that as well as the controller code we say earlier we also see a number of files being produced under app/views. There is an “.rthml” type file for each of the actions (methods) within the controller : these are HTML files with embedded Ruby code,allowing us to dynamically produce content at runtime.
40 Generated List View Code (1)
Listing aircraft
HTML with embedded Ruby| <%= column.human_name %> | via array @aircraft <% end %> (instance variable)|||
|---|---|---|---|
| <%=h aircraft.send(column.name) %> | to methods on Aircraft <% end %><%= link_to 'Show', :action => 'show', :id => aircraft %> | <%= link_to 'Edit', :action => 'edit', :id => aircraft %> | <%= link_to 'Destroy', { :action => 'destroy', :id => aircraft }, :confirm => 'Are you sure?', :method => :post %> |
Here is a sample of one of the .rhtml files (list.rhtml).
Where we use Ruby code to carry out dynamic content generation we see the code between <%= and %> tags.
Data is fed into this view definition from the model via an instance variable array @aircraft. Information about the table comes via calls to methods on the Aircraft class.
41 Generated List View Code (2)
<%= link_to 'Previous page', { :page => @aircraft_pages.current.previous } if @aircraft_pages.current.previous %> <%= link_to 'Next page', { :page => @aircraft_pages.current.next } if @aircraft_pages.current.next %>
<%= link_to 'New aircraft', :action => 'new' %>
• The various “link_to” lines are calls to a Rails method (despite not having brackets around parameters)
42
This is the rest of the list.rhtml file.
Here we see some method calls (again without brackets) being used to produce links.
42 Adding Validation
• We want to add the following validation - • AIRCRAFT_TYPE, CONSTRUCTORS_NUMBER and AIRCRAFT_STATUS must be “not null” • LINE_NUMBER must be numeric • AIRCRAFT_STATUS must be one of - • C = current • N = non-current • U = unknown • Validation is done at the MODEL level • Note that the table definition isn't enough 43
To add validation to the application we need to make edits to the model code.
Note that having constraints on a table does not produce constraint checking within the model automatically (although there is work being done on a module called “Dr Nic's Magic Models” which does seem to do this).
43 Model With Validation
class Aircraft < ActiveRecord::Base validates_presence_of :aircraft_type, :constructors_number, :aircraft_status validates_numericality_of :line_number, :allow_nil => :true
protected def validate errors.add(:aircraft_status, "must be C N or U") if aircraft_status !~ /[cCnNuU]/ end end 44
Here is the Aircraft model with added validation.
There are various built-in validation functions.
And when a function does not currently exist, we can add custom code within a method called “validate” to perform additional checks.
44 Defining Foreign Key Relationships
• Foreign keys definitions on the tables are not enough • They must be defined in the model • And there should be on both “parent” and “child” • Use functions has_many and belongs_to • By default all ID columns are not displayed in views • Must code in display / selection of these • Definition of links as above allows nested access to parent and child tables in code
45
Again when it comes to the Rails application understanding relationships between tables, it is not enough to simply define the foreign keys on the tables. We must explicitly define the relationships within the models. And we must specify the appropriate relationship in the models at both ends of the relationship.
For standard parent / child relationships, two methods are available : has_many is used at the parent, and belongs_to is used at the child.
When it comes to displaying data involved in relationships, we have to take additional steps. By default all columns ending in “ID” are not displayed in views. So we must edit views if we want information to appear (whether it is the ID value itself or the extended value from a related lookup table).
45 RI Related Tables : Models
class Aircraft < ActiveRecord::Base has_many :aircraft_histories end
class AircraftHistory < ActiveRecord::Base belongs_to :aircraft end
• Convention dictates the default column names - aircraft_history.aircraft_id ==> aircraft.id 46
So here are the two models Aircraft and AircraftHistory with their relationships defined.
We don't need to specify the columns involved in the relationship if we have named them according to contention. For example the statement “belongs_to :aircraft” assumes that column AIRCRAFT_ID on the child table points to column ID on the AIRCRAFT table.
46 Accessing Related Table Data
• Now we have defined the RI we can obtain data from related tables aircraft_history.aircraft.aircraft_type • We also have access to related table class level functions - collection_select 'aircraft_history', 'aircraft_id', Aircraft.find_all, "id", "displayname"
47
Once the relationships are defined we can “drill down” through the relationships to get to full details in the related tables. For example, to get AIRCRAFT_TYPE from table AIRCRAFT when we have have an AIRCRAFT_HISTORY record (containing an AIRCRAFT_ID) we would simply code - aircraft_history.aircraft.aircraft_type
47 Many-to-Many Relationships (1)
• Pure resolvers (no additional attributes) • Use has_and_belongs_to_many (often seen as “habtm”) on models for two related tables • Must create a resolver table using the names of the two related tables combined (no model) • Example : related tables cars and drivers • Create resolver table (no model) cars_drivers • cars model has_and_belongs_to_many :drivers • drivers model has_and_belongs_to_many :cars
48
When we have a many-to-many relationship which is a pure resolver (i.e. the resolving table has no attributes in its own right) we use a method on the linked models called has_and_belongs_to_many (often referred to as HABTM).
We must create the resolver table using the names of the two models involved. We do not create a model for the resolver table itself, since it has no meaning apart from the two models.
48 Many-to-Many Relationships (2)
• For resolvers with additional attributes • Create a model for the resolver table and define has_many and belongs_to links • Additional has_many with :through parameters • Example : we record the dates and destinations when cars were driven by particular drivers • Resolver could maybe now be known as Trip • Example models on next page ...
49
Where the resolution of a many-to-many relationship identified an entity with attributes in its own right, we should create a model to support this. We would then put appropriate has_many and belongs_to definitions in place on the (now three) models. We can define :through parameters on the has_many method calls to define how to get between the two tables in the many-to-many situation via the model for the resolver tables.
An example is shown on the next slide...
49 Many-to-Many Relationships (3)
class Trip < ActiveRecord::Base belongs_to :car belongs_to :driver end class Car < ActiveRecord::Base has_many :trips has_many :drivers, :through => :trips end class Driver < ActiveRecord::Base has_many :trips has_many :cars, :through => :trips end
50
Here the resolver between the Car and Driver models is the Trip model. We see on the two outer models that we define their relationship “:through => :trips”.
50 Supporting Legacy Databases
• Now let us assume we have a legacy database • Table names do not follow Rails conventions • Surrogate primary keys are not called “ID” • Some primary keys are composites • Some primary keys are natural keys • How do we develop a Rails application against this ? • We have to override the default conventions
51
Now we are going to look at writing a Rails application which uses an existing database structure which does not conform to the Rails conventions.
Examples of the conventions we may wish to override include -
•Table names which aren't the same as the model names or are not pluralized •Tables which do not use surrogate keys (ID columns) •Tables which have surrogate keys but the columns are not called ID •Tables which have composite primary keys
51 Supporting Different Table Names • Example difference from our “greenfield” example - AIRCRAFT_SIGHTINGS ==> T2010SEENLIST • Generate a conforming model name script/generate model aircraft_sighting • We will not use migrations to create these tables • Also turn off auto table pluralisation • Edit config/environment.rb and add line to bottom ActiveRecord::Base.pluralize_table_names = false • Then override the table name in the model class AircraftSighting < ActiveRecord::Base def self.table_name() "t2010seenlist" end end 52
First we need to access a table which does not conform to the naming standards within RoR. We want a model called AIRCRAFT_SIGHTINGS but the table we want to access is called T2010SEENLIST. First generate a conforming model name. We will not be using the migration facilities for creating tables : the tables will normally exist anyway. However this is likely to have a knock-on effect on testing and deployment support. We can turn off automatic pluralisation for the whole application. This is done by adding a line to config/environment.rb ActiveRecord::Base.pluralize_table_names = false Within the model we can the specify the table name we want to use. Here is the new model code - class AircraftSighting < ActiveRecord::Base def self.table_name() “t2010seenlist” end end
52 Supporting PKs not Called ID
• Key of T1000AIRCRAFT is AIRCRAFT_ID • Edit model to include line set_primary_key “aircraft_id” • Note that code in controllers still refers to “:id” • e.g. from app/controllers/aircraft_controller.rb def show @aircraft = Aircraft.find(params[:id]) end
53
To use a primary key column which is not named “ID”, we add a set_primary_key method call to the model - set_primary_key “aircraft_id” Since the model is basically translating references to ID to AIRCRAFT_ID, when we get to the controller we still refer to the key as “:id” - @aircraft = Aircraft.find(params[:id]) We sometimes can run into problems when we try to use non-numeric columns as primary keys because the default database expected by RoR is MySQL, which does not adhere as strictly to the typing of literal strings representing SQL data column values than most other databases, Things get really interesting when we are trying to support composite primary keys (see later).
53 Supporting Composite PKs
• Rails by default does not support composite keys • Must install extension gem composite_primary_keys mahler:~ # gem install composite_primary_keys Bulk updating Gem source index for: http://gems.rubyforge.org Successfully installed composite_primary_keys-0.7.5 Installing ri documentation for composite_primary_keys- 0.7.5... Installing RDoc documentation for composite_primary_keys-0.7.5... • One of a number of “gems” by Dr Nic Williams (http://www.drnicwilliams.com)
54
By default, RoR does not support the use of composite keys. Thankfully there is an extension gem called composite_primary_keys which does add this functionality. This, and a number of other useful gems, can be accessed from the website of Dr Nic Williams - http://www.drnicwilliams.com This can be installed using the standard gem installation process.
54 Defining “Routes” • “Routes” are URLs specific to records • Rails has a default convention for these • Defined in config/routes.rb map.connect ':controller/:action/:id' (e.g. aircraft_history/show/123) • Must override for composite keys - map.connect 'aircraft_history/show/:id', :controller => 'aircraft_history', :action => 'show', :id => /\d+,\d+/ (where key is two integer values) • For legacy systems recommended to ALWAYS define routes. 55
One of things we have to do when we are using composite keys is explicitly specify the URL format for each request : in RoR terms these URL formats are known as routes. Like most other things Rails has a default convention for routes, which is used unless it is overridden. This format, defined in config/routes.rb is - :controller/:action/:id (e.g. aircraft/show/123) When we are using composite keys we need to perform transforms on the route to make it useable. For example, if we have a key made up of two integer values we might have a route of the format - aircraft_history/show/123,100 To get the two part key mapped across to an :id which is an array of these values we would use a regular expression - :id => /\d+,\d+/ as part of the mapping of the route. For applications built on top of legacy data sources, it is recommended to always define routes. In fact, there is a school of thought which recommends defining all routes for all applications.
55 Issue with Multiple Integer Keys • “Default” database for Rails is MySQL • Allows “WHERE col = '1'” for integer columns !!! • This format used by composite_primary_keys to simplify coding • Does not work for most other databases • Method “show” in controller modified - def show # Default find code # @aircraft_history = AircraftHistory.find(params[:id]) # Replacement find code @aircraft_history = AircraftHistory.find(params[:id].split(',').map {|id| id.to_i}) end 56
We discovered that we were experiencing errors when using the default generated show and edit methods with multiple numeric keys. We tracked this down to a “feature” of the default RoR database support (with MySQL). MySQL allows SQL of the format “WHERE col = '1'” for numeric columns. Most other databases do not allow this syntax. But the composite_primary_keys developers have used this feature to reduce the complexity of the code (they just put quotes round columns of any data type). So basically this breaks the default find for virtually all databases apart from MySQL. The default methods therefore have to be modified to produce input parameters to the find method which do not have quotes. We should a modified “show” method here.
56 Testing Support
• Testing templates generated with components • See under test/ subdirectory fixtures (test data) functional (CONTROLLER : single actions) integration (CONTROLLER : flow through) mocks (stub out functionality : dev and test only) test_helper.rb unit (MODELS)
57
When we generate application components we also always generate appropriate testing templates. These can be found under the test/ subdirectory of a RoR application. The files are grouped into five distinct areas, which are put into a further level of subdirectories - •fixtures : test data definition •functional : for testing single methods within a controller •integration : for testing end-to-end flow through a controller •mocks : to allow us, in dev and test only, to stub out functionality for testing •unit : for testing models We will examine these in more detail in the following slides.
57 Unit Testing Models
• Default (from test/unit/aircraft_test.rb) require File.dirname(__FILE__) + '/../test_helper'
class AircraftTest < Test::Unit::TestCase fixtures :aircraft
# Replace this with your real tests. def test_truth assert true end end 58
Firstly we are going to look at is testing our models. Here we show the default generated “unit test” code for a model. For the Aircraft model it creates an AircraftTest class, which is a subclass of Test::Unit::TestCase. It calls the fixtures method to set up test data (see later). It also defines a very simple method called test_truth, which should always return “true” (or that is the theory ... see later). Note that all test methods should be prefixed test_*
58 Producing Test Database
• Running the default test fails miserably • Reason is that “test” level database not there • Should be able to rake db:test:prepare • Task not supported by 'ibm_db2' • Needs work on the DB2 driver • Could db2look be improved to allow this to work ? • Or maybe use ALTOBJ() ? • Or even use definitions in migrations ? • In the meantime used backup / restore to create DB
59
When we initially run the default test file (from the command line) it actually fails despite the fact the only test should always return “true”. The reason is that we have not yet set up the database defined in the test section of config/database.yml. Given an empty database, we should be able to run - rake db:test:prepare to set up all the objects in the database. Unfortunately the ibm_db2 adapter does not implement the methods required to support this rake task. There are a number of facilities that could be looked at to help implement this functionality : (improved) db2look, ALTOBJ() or using the migration definitions. In the meantime, we simply copied the development database to test using a backup and redirected restore.
59 Fixtures
• “Fixtures” are test data sets • Defined using YAML macro language • Can contain dynamic (generated) values good_aircraft: id: 100 aircraft_type: Boeing 737-7H4 constructors_number: 32517 line_number: 2133 aircraft_status: C
60
Fixtures are definitions of the test data required to carry out tests. Remember that we are using a database dedicated to testing, and that we said previously that you should never put anything into this database which you don't want to lose. When the fixtures method is called it will clear down the current table contents and will then load up the test data defined in the fixtures files. These are YAML macro files. The example given is defining a set of literal information for a test data record. It is also possible to generate dynamic values, either using the macro language facilities (perhaps to get today's date) or by calling methods within your application (e.g. to generate suitable password column values using a password generation method in the application).
60 Problems With Testing Support
• Support for testing in ibm_db2 is basically missing • Previously mentioned problem with test DB creation • Found functionality not implemented in driver so cannot simply “switch on” ibm_db2 (like migrate) • Other problems found include - • Refreshing test data from fixtures does not take account of foreign keys not being “CASCADE” • Non-standard table details not filtering through to testing support (this may be a Rails rather than DB2 driver problem)
61
It was discovered that most of the RoR testing functionality was virtually useless because of lack of implementation of required methods within the ibm_db2 adapter. This is different from migration not working initially, where we could simply tell RoR that ibm_db2 was now a valid option : in this case it needs the work done in the adapter to support the functionality. Even where the functionality exists there are problems. When refreshing test data from fixtures, it does not use the referential integrity definitions (either in the database or defined in the RoR models) to determine the order of having to clear down tables. So unless all foreign keys are defined as CASCADE this will fail. We also found that the definitions in the overall application environment and models which override conventions were not being handled correctly. For example when we overrode a table name the fixtures processing tried to delete rows from the conventional table name (based on the model name), even although we had defined a non-standard table name. This last problem may be a RoR bug rather than a problem with the DB2 adapter.
61 Production Deployment
• WEBrick web server not suitable for production • Front end should be Apache (or equivalent) • Requests forwarded to Rails services at back end • FastCGI used to be preferred method • Now HTTP proxying to Mongrel application servers is favoured • Deployment handled via Capistrano (Ruby-based utility for repeatable deployments) • Deploys via source code control system
62
We now come to look at the final phase of development : deploying the application into a production environment. The first thing to be aware of is that the embedded WEBrick web server is not considered suitable for production use : it is really designed for development use. It is expected that web requests will be received by a production quality web server, with Apache being expected (others will work but most of the documentation available is about setting up RoR with Apache). The web server will then forward requests to back-end Rails services. Until Rails 1.2 the recommended method of doing this from Apache was using the FastCGI module : however this was difficulty to get working reliably. With Rails 1.2 the preferred method is to use HTTP proxying to send requests through to a cluster of Mongrel application servers. Deployment is done via Capistrano, a utility written in Ruby for carrying repeatable deployments, with support for rollback to previous versions. It will always deploy from a source control system, with Subversion being the default option (most others, such as CVS, are also supported directly and it should be fairly easy to create configurations for any that are currently not supported).
62 Production Deployment Steps (1)
• Set up server • Install DB2, Apache, Ruby (with Gems support) • Install gems : rake, rails, termios, capistrano, mongrel, mongrel_cluster (plus dependencies) • Create database for production use • Edit production part of config/database.yml with connection details • A Capistrano hook can be used to keep password details out of config files on development boxes
63
The next few slides will go through the process of setting up a production environment and deploying a RoR application into it. The first thing to do is set up a server (or servers) with DB2, Apache and Ruby with Gems support. Then a number of gems have to be installed : you will need these on both your development environment and your production server, since most of the configuration work is done on the development side. The empty production database should then be created and the production section of config/database.yml needs to be set up. Of course, you don't really want to put your production login credentials into your projects in your Subversion repository for everyone to check out. Thankfully Capistrano has hooks which allow you to fill in details from secure areas at deployment time.
63 Production Deployment Steps (2)
• Configure mongrel (on dev project) scotdb@mahler> mongrel_rails cluster::configure \ > -e production -p 8000 -a 127.0.0.1 -N 2 \ > -c /usr/share/rails/air001/current Writing configuration file to config/mongrel_cluster.yml. • Will configure two instances of Mongrel listening on the localhost interface at ports starting with 8000 • The deployment directory (-c) always should end in /current
64
Mongrel is a fast HTTP library and server for Ruby that is intended for hosting Ruby web applications of any kind using plain HTTP rather than FastCGI or SCGI. It is framework agnostic, so is used by other web frameworks as well as Ruby on Rails. We have already installed Mongrel as two Gems (mongrel and mongrel_cluster) on both our development environment and our production server. Within our application we need to set up the execution environment definitions for Mongrel. We use the mongrel_rails command, with the cluster::configure option, to do this. The example on the slide shows us defining that cluster should run in production mode (-e production), starting up two instances (-N 2) of the application at the location defined by -c parameter and cause these to listen on the localhost address (-a 127.0.0.1) at ports starting from 8000 (-p 8000). The deployment directory should always end in /current : Capistrano will be able to then switch back and forward between different versions of the application by moving a logical link for /current around.
64 Production Deployment Steps (3)
• Add Capistrano files to project scotdb@mahler:~/rails/air001> cap –apply-to \ > /home/scotdb/rails/air001 air001 exists config create config/deploy.rb exists lib/tasks create lib/tasks/capistrano.rake • deploy.rb : recipes to deploy application • capistrano.rake : ability to call Capistrano tasks from rake (Ruby make)
65
Now that we have added Mongrel definitions to the application, we are now ready to start deploying it with Capistrano. Again we have installed Capistrano as a Gem on both our development and production environments. We use the “cap” line command to add the Capistrano files to our application. Two files are added in this process - config/deploy.rb : holds Capistrano “recipes” to deploy the application lib/tasks/capistrano.rake : allows Capistrano tasks to be run from Rake
65 Production Deployment Steps (4)
• Check Rails project into source control (in my case Subversion, but most are supported) - svnadmin create /usr/local/svnrepos svn import /home/scotdb/rails/air001 \ file:///usr/local/svnrepos/air001 \ -m “Initial import” • Edit config/deploy.rb • Add “require 'mongrel_cluster/recipes'” to top • Edit repository location details • Set deployment target details • ....
66
Now that we have finished the preparation, we should check our development into our source code control system. The example here shows us checking the application into Subversion. Remember that every time you make changes to files in your project you will need to check the project back into your source code control system (otherwise your changes won't be picked up). We then change config/deploy.rb to configure our Capistrano deployment rules. The first thing we do is add the recipes for setting up Mongrel clusters, so that we can use these during our deployment. We then define the source code repository and the target details for our deployment. An example file is shown on the next slide.
66 Production Deployment Steps (5) • Sample config/deploy.rb set :application, "air001" set :repository, "http://svn.scotdb.com/svn/#{application}/trunk" set :deploy_to, "/usr/local/rails/#{application}" set :mongrel_conf, "#{current_path}/config/mongrel_cluster.yml" #======# ROLES #======role :web, "www.scotdb.com" role :app, "www.scotdb.com" role :db, "www.scotdb.com"
67
This is the deployment definition. There are many options, but this is a simple example. We can see that variables defined earlier in the script are used further down the script, to save having to make changes in multiple places. In this example we define •the application name •the location of the source code repository : note we've added the /trunk subdirectory expected by convention in Subversion setups : we had to edit our generated RoR application manually to insert this extra level •where we will deploy to : note we haven't specified the /current location since Capistrano will create a version level directory and then point a link from /current to this •the location of the Mongrel configuration details •roles : there are three default roles we define : the web server, app server and database server
67 Production Deployment Steps (6)
• All commands run from local box (not on server) • Set up initial directory structure rake remote:setup • Deploy the application and start Mongrel servers rake remote:cold_deploy • Discovered Mongrel did not start cleanly because database tables had not been created • Again ibm_db2 is lacking support • Created manually (running rake db:migrate on server) and then Mongrel servers start cleanly 68
We are now ready to do our initial deployment. All commands are run from the development area, not on the production server. To set up the initial directory structure we run - rake remote:setup We then have a command to deploy the application and start up the Mongrels - rake remote:cold_deploy However we discovered that the Mongrels were not running. On digging into this we discovered that none of the tables had been created within the production environment. And on further investigation, we again found that the ibm_db2 adapter was again lacking the necessary support to allow this to work. We created the tables manually, by running rake db:migrate on the production server. We could add an extra Capistrano task to do this automatically. Then the Mongrel servers started cleanly. During our investigations we found that many other database adapters didn't provide this support either.
68 Production Deployment Steps (7) • Configure Apache to forward requests to Mongrels • Use Apache proxy module with load balancing • Repeated deployments can then be done • Check code into source control system • Run rake remote:deploy • Also possible to go back to a previous version • Run rake remote:rollback • Rails does not clean up session details • Schedule script to do this : example - find /tmp -name 'ruby_sess*' -ctime +12h -delete
69
We then configure our Apache (or equivalent) web servers to pass requests to the Mongrels. On Apache, we use the proxy module with load balancing definitions for each of the Mongrel servers. We are now able to repeat deployments by checking our modified source into our source code control system and running - rake remote:deploy We can also go back to the previous version using - rake remote:rollback That's our application working in production. We have to consider what we have to do to keep it running. One thing we discovered is that Ruby on Rails does not clean up session information automatically. So we had to create and schedule a script to do this, otherwise we will quickly run out of space in our temporary file area. A typical script is - find /tmp -name 'ruby_sess*' -ctime +12h -delete
69 Application Development IDEs
• A number of IDEs now support Rails development • RadRails : Eclipse Plugin (open source / free) • Komodo IDE from Activestate (commercial) • Windows only : RDE (commercial) • Various editors have Ruby plugins including JEdit, Emacs, Zeus • Personally have been using RadRails
70
Here we have listed some details of IDEs which support RoR.
70 RadRails Screenshot
71
Here's a screenshot of RadRails, the RoR IDE running on Eclipse. Notable within this IDE - •Stopping, starting and monitoring RoR servers within the IDE •Internal web browser to view the application without leaving the IDE •Ability to run Rake tasks •Ability to run generate and destroy commands •Rails console •Automatic generation of Rails application framework •Syntax highlighting for Ruby, rhtml and yml code
71 What Needs Done ? • Get ibm_db2 into standard Rails deployment • Otherwise every time Rails upgraded ibm_db2 needs to be reinstalled from scratch • Add support for testing • Test database generation • Fixtures • Add support for Capistrano deployment • Automatic database deployment • Produce Mac OS-X (client at least) for DB2 • Since this is very popular with Rails developers
72
This slide summarises what needs done to make DB2 a good choice for Ruby on Rails development. The first problem area is that the DB2 support has to be added manually after the standard Rails install, and has to be reinstalled manually after every update to the Rails environment. Support for ibm_db2 therefore needs added to the standard Rails deployment. However we would not want to break a Rails deployment for those who do not have DB2 available. Currently if any of the adapters listed in the RAILS_CONNECTION_ADAPTERS variable within ActiveRecord are actually not present then a failure occurs. This needs modified : perhaps to set this variable automatically via the list of adapters which are actually installed. The second big problem is that there is virtually no support for automated testing of applications. Integrated testing support is one of the key features of RoR and the fact that it is missing is a big issue. We also need to ensure that automated deployment with Capistrano works properly. Lastly, many Rails developers favour Mac OS-X. To encourage uptake of DB2 within the Rails community we really need to have support for DB2 on this platform. A client would be a good start, but to really allow things to take off a build of DB2 Express-C for Mac OS-X is really needed.
72 Useful Online Resources
• Web http://www.rubyonrails.org • Web http://rubyforge.org • Web http://www.alphaworks.ibm.com/tech/db2onrails • IRC channel #rubyonrails on irc.freenode.org • Mailing List [email protected] • Mailing List [email protected] • Mailing List [email protected]
73
A list of online resources (websites, IRC channels and mailing lists) related to Ruby on Rails.
73 Bibliography
• Agile Web Development with Rails (2nd Edition) (ISBN 0-9776166-3-0) • Rails Recipes (ISBN 0-9776166-0-6) • Rails Cookbook (ISBN 0-596-52731-4) • Ruby for Rails (ISBN 1-932394-69-9) • Programming Ruby (ISBN 0-9745140-5-5)
74
Books which are virtually required reading for Ruby on Rails development.
The first book on the list is virtually the “official guide”.
74 Any questions ?
75
75 Session: F04 DB2 on Rails
Philip Nelson ScotDB Limited [email protected]
Thank you very much for listening
76
76