Automating Drupal Development: Make!Les, Features and Beyond

Automating Drupal Development: Make!les, Features and Beyond

Antonio De Marco Andrea Pescetti

http://nuvole.org @nuvoleweb Nuvole: Our Team

),3.0<4 0;(3@ )Y\ZZLSZ 7HYTH Clients in Europe and USA Working with Drupal Distributions Serving International Organizations Serving International Organizations

Trainings on Code Driven Development Automating Drupal Development

1. Automating code retrieval 2. Automating installation 3. Automating site conﬁguration 4. Automating tests Automating1 code retrieval Core

Modules Contributed, Custom, Patched

Themes

External Libraries

Installation Pro!le

Drupal site building blocks drupal.org github.com

example.com The best way to download code Introducing Drush Make

Drush Make

Drush make is a Drush command that can create a ready-to-use Drupal site, pulling sources from various locations. In practical terms, this means that it is possible to distribute a complicated Drupal distribution as a single text ﬁle. Drush Make

‣ A single .info ﬁle to describe modules, dependencies and patches

‣ A one-line command to download contributed and custom code: libraries, modules, themes, etc... Drush Make can download code Minimal make!le: core only

; distro.make ; Usage: ; $ drush make distro.make [directory] ; api = 2 core = 7.x projects[drupal][type] = core projects[drupal][version] = "7.7" Minimal make!le: core only

$ drush make distro.make myproject drupal-7.7 downloaded.

$ ls -al myproject

-rw-r--r-- 1 ademarco staff 174 May 16 20:04 .gitignore drwxr-xr-x 49 ademarco staff 1666 May 16 20:04 includes/ -rw-r--r-- 1 ademarco staff 529 May 16 20:04 index.php -rw-r--r-- 1 ademarco staff 688 May 16 20:04 install.php drwxr-xr-x 70 ademarco staff 2380 May 16 20:04 misc/ drwxr-xr-x 43 ademarco staff 1462 May 16 20:04 modules/ drwxr-xr-x 6 ademarco staff 204 May 28 13:28 profiles/ -rw-r--r-- 1 ademarco staff 1561 May 16 20:04 robots.txt drwxr-xr-x 13 ademarco staff 442 May 16 20:04 scripts/ drwxr-xr-x 5 ademarco staff 170 May 16 20:04 sites/ drwxr-xr-x 8 ademarco staff 272 May 16 20:04 themes/ -rw-r--r-- 1 ademarco staff 19338 May 16 20:04 update.php -rw-r--r-- 1 ademarco staff 2051 May 16 20:04 web.config -rw-r--r-- 1 ademarco staff 417 May 16 20:04 xmlrpc.php Downloading a module

; views.make ; Usage: ; $ drush make views.make --no-core . ; api = 2 core = 7.x projects[views][subdir] = contrib projects[views][version] = 3.1 Downloading a module

$ drush make views.make --no-core . views-7.x-3.1 downloaded.

$ ls -al sites/all/modules/contrib/views/ total 64 ... -rw-r--r-- 1 ademarco staff 16067 May 28 13:28 views.info -rw-r--r-- 1 ademarco staff 20358 May 28 13:28 views.install -rw-r--r-- 1 ademarco staff 78204 May 28 13:28 views.module ... Drush Make can apply patches Applying patches

; distro.make ; Usage: ; $ drush make distro.make [directory] ; api = 2 core = 7.x projects[drupal][type] = core projects[drupal][version] = "7.7"

; Make system directories configurable to allow tests in profiles/[name]/modules to be run. ; http://drupal.org/node/911354 projects[drupal][patch][911354] = http://drupal.org/files/issues/911354.43.patch

; Missing drupal_alter() for text formats and filters ; http://drupal.org/node/903730 projects[drupal][patch][903730] = http://drupal.org/files/issues/drupal.filter-alter.82.patch

... Applying patches

$ drush make distro.make myproject drupal-7.7 downloaded. drupal patched with 911354.43.patch. drupal patched with drupal.filter-alter.82.patch. drupal patched with 995156-5_portable_taxonomy_permissions.patch. Generated PATCHES.txt file for drupal Drush Make supports recursion

"exslider / "exslider.make

; Flex Slider api = 2 core = 7.x libraries[flexslider][download][type] = "get" libraries[flexslider][download][url] = "https://github.com/.../zipball/master" libraries[flexslider][directory_name] = "flexslider" libraries[flexslider][type] = "library" Downloading "exslider

; flexslider-module.make ; Usage: ; $ drush make flexslider-module.make --no-core . ; api = 2 core = 7.x projects[flexslider][subdir] = contrib Recursive make!le parsing

$ drush make flexslider-module.make --no-core .

Project flexslider contains 4 modules: flexslider_views_slideshow, flexslider_views, flexslider_fields, flexslider. flexslider-7.x-1.0-rc3 downloaded. Found makefile: flexslider.make flexslider downloaded from https://github.com/.../zipball/master.

$ tree -l sites/all/ sites/all/ libraries flexslider ... modules contrib flexslider ... Drush Make supports inclusion Including an external make!le

; distro.make ; ; $ drush make buildkit.make [directory] ; api = 2 core = 7.x

; Include Build Kit distro makefile via URL includes[] = http://drupalcode.org/project/buildkit.git/../7.x-2.x:/distro.make Build Kit Extendable distribution, reusable .make ﬁle

Including Build Kit module set

; ; myproject.make ; api = 2 core = 7.x

; Include Build Kit install profile makefile via URL includes[] = http://drupalcode.org/project/buildkit.git/../drupal-org.make

; Modules ======projects[views_bulk_operations][subdir] = contrib projects[coffee][subdir] = contrib Build Kit: A closer look Demonstrating makeﬁle inclusion and recursion. Build Kit provides 2 make !les

‣ distro.make: Drupal core with possible core patches and a link to download drupal-org.make

‣ drupal-org.make: a selection of common modules from drupal.org Build Kit: full content

$ git clone --branch 7.x-2.x http://git.drupal.org/project/buildkit.git

...

$ ls -al buildkit/ total 48 drwxr-xr-x 9 ademarco staff 306 Jun 14 15:45 . drwxrwxr-x@ 20 ademarco staff 680 Jun 14 15:45 .. drwxr-xr-x 13 ademarco staff 442 Jun 14 15:45 .git -rw-r--r-- 1 ademarco staff 3868 Jun 14 15:45 README.txt -rw-r--r-- 1 ademarco staff 583 Jun 14 15:45 buildkit.info -rw-r--r-- 1 ademarco staff 151 Jun 14 15:45 buildkit.install -rw-r--r-- 1 ademarco staff 6 Jun 14 15:45 buildkit.profile -rw-r--r-- 1 ademarco staff 849 Jun 14 15:45 distro.make -rw-r--r-- 1 ademarco staff 902 Jun 14 15:45 drupal-org.make buildkit / distro.make

api = 2 core = 7.x projects[drupal][type] = core projects[drupal][version] = "7.14"

; Use vocabulary machine name for permissions ; http://drupal.org/node/995156 projects[drupal][patch][995156] = http://drupal.org/files/issues/995156-5_portable_taxonomy_permissions.patch projects[buildkit][type] = profile projects[buildkit][download][type] = git projects[buildkit][download][url] = http://git.drupal.org/project/buildkit.git projects[buildkit][download][branch] = 7.x-2.x Your project: 2 make !les

‣ distro.make: Drupal core with possible core patches and a link to download myproject.make

‣ myproject.make: includes BuildKit’s drupal-org.make plus project-speciﬁc modules and themes drush make distro.make Found myproject.make

drush make distro.make Run myproject.make

Found myproject.make

drush make distro.make Automating2 installation Installation Pro!le

Installation pro!le components

$ tree myproject-profile myproject-profile README.txt distro.make drushrc.php myproject.info myproject.install myproject.make myproject.profile

Pro!les: just like modules

‣ An .info ﬁle to specify installation dependencies

‣ An .install ﬁle to perform installation tasks and upgrades

‣ Fully customizable via .proﬁle ﬁles

‣ Can include makeﬁles and other stuff myproject.info

name = Myproject core = 7.x description = Myproject installation profile.

; Core dependencies[] = book dependencies[] = field_ui dependencies[] = file ...

; Contrib dependencies[] = admin dependencies[] = colorbox dependencies[] = ds ...

; Features dependencies[] = myproject_core dependencies[] = myproject_blog myproject.pro!le

/** * Implements hook_install() */ function myproject_install() { // Enable custom theme theme_enable(array('custom_theme')); variable_set('theme_default', 'custom_theme'); } myproject.pro!le

/** * Implements hook_form_FORM_ID_alter(). */ function myproject_form_install_configure_form_alter(&$form, $form_state) {

$form['site_information']['site_name'] ['#default_value'] = 'Drupalissimo'; $form['site_information'] ['site_mail']['#default_value'] = '[email protected]';

$form['admin_account']['account'] ['name']['#default_value'] = 'admin'; $form['admin_account']['account'] ['mail']['#default_value'] = '[email protected]';

$form['update_notifications'] ['update_status_module']['#default_value'] = array(1 => FALSE, 2 => FALSE); }

myproject.install

/** * Implements hook_install_tasks() */ function myproject_install_tasks() { return array( 'myproject_create_terms' => array( 'display_name' => st('Create taxonomy terms'), ), ... ); }

myproject.install

/** * Implements hook_install_tasks() callback */ function myproject_create_terms() { $terms = array(); $vocabulary = taxonomy_vocabulary_machine_name_load('category');

$terms[] = 'Solution'; $terms[] = 'Client'; $terms[] = 'Use case';

foreach ($terms as $name) { $term = new stdClass(); $term->vid = $vocabulary->vid; $term->name = $name; taxonomy_term_save($term); } }

Introducing Drush Bake A Drush command by Nuvole to create installation proﬁles based on templates Installation pro!le template

$ git clone git.nuvole.org:/var/git/starter-profile.git

...

$ tree starter-profile starter-profile/ README.txt distro.make drushrc.php starter.info starter.install starter.make starter.profile

Template distro.make

$ cat starter-profile/distro.make api = 2 core = 7.x

; Include Build Kit distro makefile via URL includes[] = http://drupalcode.org/project/buildkit.git/blob_plain/refs/heads/7.x-2.x:/distro.make

projects[starter][type] = profile projects[starter][download][type] = git projects[starter][download][url] = git.nuvole.org:/var/git/starter-profile.git Template starter.make (aka myproject.make)

$ cat starter-profile/starter.make api = 2 core = 7.x

; Build Kit ======includes[] = http://drupalcode.org/project/buildkit.git/.../drupal-org.make

; Modules ======projects[coffee][subdir] = contrib ... projects[twist][type] = theme projects[twist][download][type] = git projects[twist][download][url] = git.nuvole.org:/var/git/twist.git Template starter.install (aka myproject.install)

$ cat starter-profile/starter.install

/** * Implements hook_install(). */ function starter_install() { theme_enable(array('twist')); variable_set('theme_default', 'twist'); } Generating a new installation pro!le

$ drush bake starter-profile/distro.make myproject Generating a new installation pro!le

$ # Create new installation profile starting from starter-profile template $ drush bake starter-profile/distro.make myproject

...

$ tree myproject-profile myproject-profile README.txt distro.make drushrc.php myproject.info myproject.install myproject.make myproject.profile

Generated distro.make

$ cat myproject-profile/distro.make api = 2 core = 7.x projects[drupal][type] = core projects[drupal][version] = "7.12"

... projects[myproject][type] = profile projects[myproject][download][type] = git projects[myproject][download][url] = git.nuvole.org:/var/git/myproject-profile.git Full bootstrap

$ # Push profile folder to Nuvole server following internal conventions $ git nuvole myproject-profile

$ # Build the project $ drush make myproject-profile/distro.make myproject drupal-7.12 downloaded. [ok] drupal patched with 911354.43.patch. [ok] drupal patched with drupal.filter-alter.82.patch. [ok] drupal patched with 995156-5_portable_taxonomy_permissions.patch. [ok] Generated PATCHES.txt file for drupal [ok] myproject cloned from git.nuvole.org:/var/git/myproject-profile.git.[ok] Found makefile: myproject.make [ok] libraries-7.x-1.0 downloaded. [ok] rubik-7.x-4.0-beta7 downloaded. [ok] twist cloned from git.nuvole.org:/var/git/twist.git. [ok] feature_core cloned from git.nuvole.org:/var/git/feature_core.git. [ok] Found makefile: feature_core.make [ok] ... Bootstrap your project in 3 steps

$ drush bake starter-profile/distro.make myproject

$ git nuvole myproject-profile

$ drush make myproject-profile/distro.make myproject Automating3 site con!guration Features The best way to package conﬁguration

What is a feature?

‣ A collection of Drupal elements which taken together satisfy a certain use-case.

‣ A modular piece of functionality for a Drupal site.

‣ A way to export conﬁguration into PHP code, in the form of a module.

‣ http://drupal.org/project/features Con!guration in Database Packaged as Features Creating a Feature

It’s all in the feature’s .info !le Features are Modules core = "7.x" description = "Core feature, stuff we need all the time." dependencies[] = "colorbox" dependencies[] = "ds" dependencies[] = "features" dependencies[] = "insert" dependencies[] = "markdown" dependencies[] = "menu" dependencies[] = "pathauto" dependencies[] = "strongarm" dependencies[] = "token" features[ctools][] = "ds:ds:1" features[ctools][] = "strongarm:strongarm:1" features[ds_view_modes][] = "core_small_teaser" features[filter][] = "core_rich_text" features[menu_custom][] = "main-menu" features[menu_links][] = "main-menu:" features[user_permission][] = "access content" features[user_permission][] = "use text format core_rich_text" features[variable][] = "admin_toolbar" features[variable][] = "date_format_long" ... A feature can have a .make !le too Drush Make operates recursively api = 2 core = 7.x

; Modules ======projects[colorbox][subdir] = contrib projects[colorbox][version] = 1.0-beta4 projects[insert][subdir] = contrib projects[insert][version] = 1.1

; Libraries ======libraries[colorbox_library][download][type] = "get" libraries[colorbox_library][download][url] = "http://colorpowered.com/colorbox/colorbox.zip" libraries[colorbox_library][directory_name] = "colorbox" libraries[colorbox_library][destination] = "libraries"

feature_core.make A feature can specify where to !nd its own dependencies How to download your projects from any custom repository $ cat starter-profile/starter.make api = 2 core = 7.x

; Build Kit ======includes[] = http://drupalcode.org/project/buildkit.git/blob_plain/refs/heads/7.x-2.x:/drupal-org.make

; Modules ======projects[libraries][subdir] = contrib projects[libraries][version] = 1.0

; Features ======projects[feature_core][type] = module projects[feature_core][subdir] = features projects[feature_core][download][type] = "git" projects[feature_core][download][url] = git.nuvole.org:/var/git/feature_core.git

; Themes ======projects[twist][type] = theme projects[twist][download][type] = git projects[twist][download][url] = git.nuvole.org:/var/git/twist.git Introducing Feature Servers Keep features, themes, makeﬁles, etc... organized.

$ cat starter-profile/starter.make api = 2 core = 7.x

; Build Kit ======includes[] = http://drupalcode.org/project/buildkit.git/blob_plain/refs/heads/7.x-2.x:/drupal-org.make

; Modules ======projects[libraries][subdir] = contrib projects[libraries][version] = 1.0

; Features ======projects[feature_core][subdir] = features projects[feature_core][location] = http://fserver.nuvole.org/fserver

; Themes ======projects[twist][type] = theme projects[twist][location] = http://fserver.nuvole.org/fserver Feature Server and Drush

$ # Drush goodies $ drush | grep fserver

fserver-distro Update packaging for a distribution. fserver-package Update packaging for a project. fserver-status Determine the packaging status of a project.

$ # Build releases for feature_news based on GIT tags and branches $ drush fserver-package feature_news

$ # Check releases status $ drush fserver-status

Method Project New tags git Feature News 7.x-1.0 git Feature Pages 7.x-1.0 ...

Automating4 tests Don't depend on trust Automatically test every component Meet Continuous Integration (CI)

Use Hudson/Jenkins to automatically test: 1. Makeﬁle 2. Installation 3. Conﬁguration

Building a CI Job

‣ Create a job for testing your site

‣ Triggered:

‣ Manually

‣ Scheduled

‣ By events (git push or other jobs)

‣ A job can consist of ant scripts or simple shell commands (including drush)

Test #1: Make!le

‣ Clone your code from git

‣ Run drush make

‣ Test that the proﬁle is downloaded

‣ Test that modules are placed in the expected folders Test #1: Shell commands

‣ drush -y --pipe make distro.make

‣ test -d proﬁles/myproject

‣ test -d proﬁles/myproject/modules/contrib

‣ test -d proﬁles/myproject/modules/custom

‣ test -d proﬁles/myproject/modules/features

Test #2: Installation

‣ Triggered by successful completion of Test #1

‣ Run drush site-install:

drush -y site-install ... myproject

‣ Expect successful completion

Test #3: Con!guration

‣ Triggered by successful completion of Test #2

‣ Relies on simpletest

‣ Run drush test-run

‣ Expect successful completion

Thank You.

More on Code-Driven Development http://nuvole.org/blog http://nuvole.org/trainings