A Rake Task to Grab Data From Your Production Site

Posted by Craig Ambrose on April 17, 2007 at 10:17 PM

When you’re fixing bugs that occur in a production rails application, the first step is often to reproduce them locally in development mode. Every now and then, I find that I want to make my local development site run with the same data as my live production site, so here’s a little rake task to make it easier.


desc "Make local develoment site look like the live site"
task :sync do
host = 'craigambrose.com'
path = '/var/www/apps/craigambrose/current' db_config = YAML.load_file('config/database.yml') system "ssh #{host} \"mysqldump -u #{db_config['production']["username"]} -p#{db_config['production']["password"] } -Q —add-drop-table -O add-locks=FALSE -O lock-tables=FALSE —ignore-table=#{db_config['production']["database"]}.sessions #{db_config['production']["database"]} > ~/dump.sql\""
system "rsync -az —progress #{host}:~/dump.sql ./db/production_data.sql"
system "mysql -u #{db_config['development']["username"]} -p#{db_config['development']["password"]} #{db_config['development']["database"]} < ./db/production_data.sql"
system "rsync -az —progress #{host}:#{path}/public/system/ ./public/system"
end

Installation

Rake tasks can go in any file ending in .rake inside your lib/tasks directory (within a rails application). You might put this in a file specific to your application, like craigambrose.rake. You can also namespace rake tasks by putting them inside a blog like:


namespace :craigambrose do
# your tasks go here
end

With my task, you’ll need to set the value of two local variables, host and path. Or, perhaps you can think of a way to read those from your deploy.rb?

Usage

If you’ve namespaced it like I have, then it’ll be something like:


rake craigambrose:sync

Note that this uses a lot of shell commands, and basically assumes that you are developing on a unix box of some sort. You can delete the rsync for /public/system if you don’t store any files there.

Tags: (none)
Hierarchy: previous, next

Comments

There are 6 comments on this post. Post yours →

Koz

One other thing to consider is to make sure you overwrite any important stuff. In particular, sending out emails from staging can be kinda embarrassing, so I usually have a second step which updates the email addresses, blanks out any potentially sensitive data etc.

www.ratepoint.com

What is the last rsync for??

What do you store in your ./public/system folder that you need to rsync back and forth for a DB backup??

Fantastic! I just had to add one more parameter to the dump command:

-h #{db_config‘production’}

Thanks!

Very useful! We have this exact problem with Creataplace

@ratepoint.com – The reason for the ./public/system folder is because there are also production-specific files which need to be synced. Images or documents a user might have uploaded etc.

David is quite right there about the ./public/system folder.

Also, if you have any ERB tags in your database.yml file, the above wont work. Jack Danger sent me a fix:


Ahoy Craig!
I’ve been using your db:sync rake script and I was having trouble with it not processing the ERB in my database.yml file. I made the following change to it so it now processes database.yml the same was as Rails does:

old:
db_config = YAML.load_file(‘config/database.yml’)
new:
db_config = YAML::load(ERB.new(IO.read(‘config/database.yml’)).result)

Thanks for the code!

::Jack Danger
http://6brand.com

Hi
A slightly different approach that copies data using models:
http://justbarebones.blogspot.com/2007/10/copy-model-data-between-databases.html

- Chetan

Post a comment

Required fields in bold.