Drupal 7 Upgrades with Scripts and Drush

AutomationA Drupal upgrade between major versions rarely means doing the upgrade procedure once. A new configuration, changing content, or needing to test different module versions typically means redoing the upgrade from scratch a number of times, a tedious, error-prone and time consuming process. Fortunately there's a solution: automation!

Scripting was always possible, but Drush has made it so much simpler. What follows is a review of the Drush commands I used via script to upgrade my Group 42 website to Drupal 7. The hand full of shell commands are basic and covered many other places on the net. The full upgrade scripts are at the end of the post.

If you're familiar with Drush I recommend jumping directly to the scripts.

Starting Notes

-y / --yes option: This option skips the yes/no confirmation question required by some Drush commands. Since the examples in this post come from scripts this option is shown in the examples.

Drush site aliases: I almost always use a Drush site alias (the "@site-name) parameter, with a command. Some of the commands outlined in this post will only work if you have a site alias. Among other things, it means you can run the command from anywhere.

Drush site-upgrade command: One command you won't find in this post is the Drush site upgrade command. After 20 minutes of trying to use it I gave up. As nearly as I can tell my site wasn't configured the way the command needed it configured. Your results could be different so you'll want to check it out.

Watchdog Log Commands

At various points in the upgrade it's a good idea to check the watchdog log:

drush wd-show

Module Commands

The term module is commonly used to refer to an entire Drupal project even though a typical Drupal project contains more than one module. This can be confusing if the project also has a module with the same name, such as Views. Drush downloads and uninstalls projects, but lists, enables, and disables modules. For example, to remove the Views project in Drupal 6 you disable the Views, Views UI, and Views Export modules, then uninstall the Views project.

Multiple modules can be acted on at once. Examples are shown for both a single and multiple modules.

Listing modules

Drush is an easy way to list installed modules. This is useful for module research lists and creating a list of modules to cut and paste into your script.

For a full summary:

drush pm-list

For a lists all of enabled contributed modules displayed one module per line:

drush pm-list --no-core --type=module --status=enabled –pipe

Downloading Projects

Download the current recommended version of the project:

drush @g42dev dl -y backup_migrate
drush @g42dev dl -y codefilter google_analytics mollom

Download a specific version of the project (required for development versions):

drush @g42dev dl -y cck-7.x-2.x-dev globalredirect-7.x-1.x-dev

Enabling Modules

drush @g42dev pm-enable -y toolbar
drush @g42dev pm-enable -y toolbar overlay shortcut

update.php (Database Updates)

Running update.php using Drush means you don't need to log on as user 1 or use $update_free_access if you forget. Although Drush will tell you if an update error occurred, I like displaying the watch dog log as well.

drush @g42dev updatedb --yes
drush @g42dev wd-show

Disabling Modules

drush @g42dev -y pm-disable backup_migrate
drush @g42dev -y pm-disable views views_export views_ui

Uninstalling Projects

drush @g42dev -y pm-uninstall views 
drush @g42dev -y pm-uninstall admin_menu tagadelic advanced_help

Theme Commands

Themes are enabled and disabled in the same way modules are. You can also set the current theme, administration theme, and theme settings from the Drush command line.

Enable a theme

drush @g42dev -y pm-enable garland

Disable a theme

drush @g42dev -y pm-disable barron

Set the default theme

drush @g42dev -y vset theme_default garland

Set the default administration theme

drush @g42dev vset -y admin_theme seven

Set the theme settings

Theme settings are a bit challenging. In Drupal 7 they're stored as an array. As of writing the current version of Drush could not set arrays with the vset command. However, using the php-eval command it's possible to use the Drupal variable_set() function to set the theme variables.

The following example has been broken into multiple lines for display purposes, but is one, long wrapped line when executed:

drush @g42dev php-eval "variable_set('theme_barron7_settings', array('toggle_logo' => 0,
'toggle_name' => 1, 'toggle_slogan' => 1, 'toggle_node_user_picture' => 0,
'toggle_comment_user_picture' => 0, 'toggle_favicon' => 1, 'toggle_main_menu' => 1, 
'toggle_secondary_menu' => 0, 'default_logo' => 1, 'logo_path' => '',
'logo_upload' => '', 'default_favicon' => 0,
'favicon_path' => 'sites/all/themes/barron7/favicon.ico',
'favicon_upload' => '', 'favicon_mimetype' => 'image/vnd.microsoft.icon'))"

File Copy/Synchronization Commands

If you have your site aliases configured with server and directory information you can save yourself looking up directory paths and rsync paramters by using the built-in Drush rsync:

drush rsync @g42local @g42dev -y

Database Backup/Restore Commands

The Backup and Migrate module also has Drush support, but during an upgrade the module isn't installed.

Database Backup

drush @g42local sql-dump --result-file=/tmp/group42db_d7.sql

Database Restore

It's always best to drop all the database tables before doing a restore, especially during major version upgrades.

drush @g42dev sql-drop --yes
`drush @g42dev sql-connect` < /tmp/group42db.sql

July 13, 2011: Per the comment from dalin, the following syntax is probably better than using the BASH back-tick syntax. If I had known about it I would have used it instead.

drush @g42dev sql-cli < /tmp/group42db.sql

Changing Settings

Many Drupal system and contributed module settings are easily changed using the variable set command. A number of other settings require a direct database update. Since you can easily damage your database doing direct updates I wouldn't recommend using this option unless you know what you're doing.

Site Online / Offline

Since I run my scripts on a development server with no users I don't usually bother with offlining my site. I've included it here because it's easy and something others may want to do in their upgrades.

Whether from Drush or the web administration interface, I've found I need to clear the page cache if it's enabled.

Site Offline

drush vset site_offline 1 --yes
drush cc all

Site Online

drush vset site_offline 0 --yes
drush cc all

Site Information

In the early stages of upgrade testing I'll change my site name and slogan to make my test site easy to tell apart from my production site.

Site Name

drush @g42dev vset --yes site_name "G42 Clone"

Site Slogan

drush @g42dev vset --yes site_slogan "Drupal 7 Upgrade Testing"

Block Position

Since you can corrupt your database with a bad SQL command I wouldn't recommend doing this command unless you're comfortable that you know what you're doing.

drush @g42dev sql-query 'UPDATE {block} SET region = "sidebar_first", status = 1, 
weight = -10 WHERE module = "views" AND delta = "recent_comments-block"'

Path Auto Settings

The tokens used by Path Auto have changed and require updating. Fortunately, they're stored as variables so it's easy to update them by script so you only have to do it once.

drush @g42dev -y vset pathauto_node_story_pattern "[current-page:title]"
drush @g42dev -y vset pathauto_user_pattern "user/[user:name]"
drush @g42dev -y vset pathauto_taxonomy_term_pattern "[term:vocabulary]/[term:name]"
drush @g42dev -y vset pathauto_node_note_pattern "note/[current-page:title]"

I noticed path aliases didn't start working until I saved a node. Perhaps a byproduct of the scripting, or perhaps something else. In any event, this is also easy to do in a script:

drush @g42dev php-eval 'node_save(node_load(6))'

July 13, 2011: The following comment is worth considering if you find yourself in this same situation: http://www.group42.ca/comment/4722#comment-4722.

Path Alias

The default feed URLs for taxonomy terms have changed from /taxonomy/term/n/0/feed to /taxonomy/term/n/feed. There was only one taxonomy term feed I was worried about (Drupal Planet!), so I created an alias for it:

drush @g42dev php-eval 'path_save($a = array("source"=>"taxonomy/term/2/feed", 
"alias"=>"taxonomy/term/2/0/feed"))'

Performance Settings

The performance settings are cache, block_cache, cache_lifetime, page_cache_maximum_age, page_compression, preprocess_css, and preprocess_js. To turn these off for testing:

drush @g42dev vset --yes cache 0
drush @g42dev vset --yes block_cache 0
drush @g42dev vset --yes page_compression 0
drush @g42dev vset --yes preprocess_css 0
drush @g42dev vset --yes preprocess_js 0

The Attached Scripts

The two attached scripts are the actual scripts I used for the Drupal 7 upgrade of this website. They are intended as real-world examples and NOT meant to be prescriptive. Everyone's workflow tends to be different, and these scripts are in service of my workflow. I recommend adapting them to yours. Also, I only loosely followed the upgrade sequence outlined in UPDATE.txt. My upgrade sequence may not work for your site. Take what works for you and leave the rest behind!

In case it's useful, my upgrade workflow was:

  1. Clone the production site as required, either to reset for an upgrade run or to refresh content
  2. Run the upgrade script and test as required
  3. Merge the upgraded file base into the main trunk
  4. Upgrade production site by switching to the new file base and copying the database from the upgrade test site to production site

clonetodev.sh script

This script copies the source website to the upgrade testing site. It's used to reset the upgrade site for repeated testing.

upgrade_d7.sh script

The replacement of Drupal core files is done with a script I describe in this post: Drupal File Upgrade Bash Script.

As I mentioned, this script does not follow the same sequence outlined in UPGRADE.txt. If you use this script as a starting point it's your call whether you maintain the sequence, change it to follow UPGRADE.txt, or adopt a sequence of your own. The nice thing about scripts is it's easy to try different scenarios.

drupalupgrade script

This script is called from upgrade_d7.sh. A full explanation of the drupalupgrade script is in the post: Drupal File Upgrade Bash Script .

Conclusion

I hope the examples prove useful. If you have alternative methods or your own examples I encourage you to share them in your own blog post, the post comments, or in the Drupal.org documentation.

July 21, 2011: Added reference to drupalupgrade script thanks to feedback.

AttachmentSize
Plain text icon clonetodev.sh.txt1.15 KB
Plain text icon upgrade_d7.sh.txt3.16 KB

Comments

Yeah I'm busy doing this exact thing now, from Drupal 5 to 7. Drush is fantastic, but man the process is very error-prone, especially CCK from 6 to 7!

This site has one node-type using CCK and it isn't used anymore, so I don't have a lot of experience yet with CCK updates. I did run the CCK field upgrade just to see what would happen and it worked OK. It's not a complicated content-type, though.

For a complicated build or taking fields from Drupal 5 to 7 it might almost be better to build the Drupal 7 site fresh and import the date from the old site. There's a growing set of tools and scripting techniques that make this straightforward.

Good luck!

While your database restore command will work, this might be easier to grok:

drush @g42dev sql-cli < /tmp/group42db.sql


drush @g42dev php-eval 'node_save(node_load(6))'

It's very rare to find a site that this works without breaking something and you don't find it until three months later (This should get fixed in D8 with the Configuration Management core initiative).

The better way to get all pathauto rules working for existing nodes is to go into the UI and run the batch tool.

Thanks for the sql-cli heads-up, dalin. Using sql-connect came from the Drush documentation page: http://drush.ws/#sql-connect. I like the sql-cli syntax better.

Regarding path auto, I discovered the load/save by accident when I edited and saved a single node while troubleshooting and saw all of the path aliases suddenly start working. The node_load/node_save is a code version of the manual process. Point taken on the batch tool.

Hi, thanks for the useful post.

Maybe I missed something, but I think you need to add a link to the following as
an attachment.

http://www.group42.ca/files/drupalupgrade.txt

Your 'upgrade_d7.sh.txt' makes a call to 'drupalupgrade 7.4', but there's no
other reference to 'drupalupgrade'. I found the link above thru Google.

David

Thanks for pointing that out, David.

The first draft of this post explained where that script came from. Unfortunately I edited it out! (One of the dangers of being your own editor)

As you've discovered, that script is explained in the post: Drupal File Upgrade Bash Script. Ironically, I wrote that post first just so I could reference it in this post.

I'll update the post to include the information.

Hi Dale,

You missed a very useful drush command:

drush site-install standard --db-url=mysql://username:pass@localhost/databasename

Do note it sets your user 1 username/password to admin/admin.

Sometimes it's just easier to build D7 from scratch and then re-apply the content...

Best,
Sam

At the time that this post was written, the site-upgrade command was of limited use for anything but the simplest sites; the new version makes the upgrade process significantly easier, though. The beta release is out today; for more information, see the project page at http://drupal.org/project/drush_sup