About a year ago, file_column was one of the most popular plugin for storing files, particularly images, in rails applications. These days, the most popular plugin is Rick Olsen’s attachment_fu.

The main advantage of attachment_fu is it’s ability to store the files either on the file system, in binary fields in the database, or on amazon s3. The pluggable nature of the code also makes it fairly easy to support some other storage service.

I’ve avoided moving over because file_column actually provides more comprehensive image manipulation features, but there comes a time in the life-cycle of most applications where file system storage just doesn’t cut it in a multi-server environment.

There are already tutorials on using attachment_fu, I’m presuming that you already know how to use it. I’m just going to help you make the switch. Lets start with some code, here’s my migration for moving across the data:

class CreateProfilePhotosFromFileColumn < ActiveRecord::Migration
  def self.up
    for profile in Profile.find(:all)
      image_filename = select_value "SELECT image FROM profiles WHERE id = #{profile.id}" 
      unless image_filename.blank?
        image_path = RAILS_ROOT + "/public/system/profile/image/#{profile.id}/#{image_filename}" 
        image_file = File.open(image_path, 'r')        photo = ProfilePhoto.new(:profile_id => profile.id)
        photo.set_from_file(image_file)
        photo.save!
      end
    end
  end  def self.down
    execute "DELETE FROM profile_photos" 
  endend

In this example, I previously used file column in the field called “image” of my Profile model. Now, I have a new model called ProfilePhoto, which belongs_to Profile.

Although I’m happy to loop through all profiles using regular active record finders, note that I didn’t use the image method of profile. I know that after this works, I’m about to remove everything to do with file column, and so to play it safe, I use assert_select to fetch the file column image name directly from the database. This is ugly, but good policy in general for producing migrations that keep working after the code changes.

The other trick here is the call to “set_from_file”. This method doesn’t exist in attachment_fu, and was the first (of several) glaring omissions that I noticed in this plugin. To make this migration work, you’ll need to make a few changes to attachment_fu.rb yourself.

The following goes inside the InstanceMethods module:

def set_from_file(source_file)
  source_file_extension = File.extname(source_file.path).reverse.chomp('.').reverse
  source_file_name = File.basename(source_file.path)
  self.content_type = self.class.mime_type_from_extension(source_file_extension)
  self.filename = source_file_name
  self.temp_data = source_file.read
end

The following goes inside the ClassMethods module:

def mime_type_from_extension(extension)
  MIME::Types.type_for(extension).first.simplified
end

And the following goes at the top of the file:

require 'mime/types'

You’ll also need to “gem install mime-types”, although you will already have this if you installed the amazon s3 library.

This code is not hugely error tolerant. It presumes that all your records with file columns contain valid image files that are going to be accepted by your attachment_fu model. It also assumes that you set up attachment_fu correctly of course.

If it works, I would then add further migrations to remove the old file_column field from the Profile model, and to remove the file_column files themselves from the hard disk.

You’ll probably find the set_from_file method to be a valuable addition to attachment_fu for other purposes too. Our applications often receive their data in ways other than just HTTP post, and being able to save a file object seems like a pretty obvious addition.

I’ve moved this blog from Typo, to Simplelog, both rails based blogging systems. Normally I don’t do this sort of meta-blogging, but I though I’d share a few tips about the migration, and some comments on Simplelog.

Installation

Like the other rails blogs, simplelog has some installation instructions for how to get the simplelog files on your sever, and like all rails developers, I promptly ignored them, and setup a copy of simplelog in a subversion repository of my own and deployed to my server using capistrano. This makes deployment and upgrading much easier, as I can test new simplelog versions on my local machine first, and then roll out updates using cap deploy as normal.

Migrating your Data

There don’t seem to be any nice scripts for migrating data from Typo to Simplelog. There are some scripts for exporting typo in data into Moveable Type or word press format, and you could probably hook together two or three different scripts to eventually get from Typo to Simplelog, but I couldn’t figure out a simple way to do that without also installing a PHP app, and that just seemed way too complex, so I thought I’d roll my own.

The bad news I’m afraid, is that I haven’t bothered to sufficiently generalise it to make it work for everyone, but if you want to do this too, then you can use my code as a starting point and refine it for your needs.

When you want to script some little task in rails, the tool for the job is of course rake. I started working on a little rake task inside my typo installation, to export the data in a format I could use for simplelog. I started by comparing both database schemas, and seeing how I could generate SQL to use with simplelog.

It turns out that both these databases are non-trivial. They use single-table inheritance, and the also both have cache columns full of keywords and search text and so forth. The active record classes in both apps have the code to do this, so the best way to convert your data is to leverage that code. You could write a ruby app that loads both model layers and connects to both database, but that didn’t seem too trivial, so what I ended up doing was creating a typo rake export task that generated a simplelog rake import task.

The Code

Here’s the code, it only copies the data I was interested in (articles and comments), and it does so with a few specific values (such as a fixed author id). It also, on large databases, will hit some id conflicts as it does so. To fix this, run it multiple times until the auto-numbered ids become bigger than the ids in your old database (or solve that problem properly). Also, it’s butt ugly, remember, I just wanted to do this in an hour and never see it again.

Before running this script, make sure that you have setup your simplelog database (with the migrations), and created a user (my had id 2, which is hardcoded in the script). Also, set the default markup format in simplelog to be whatever you used in typo (I was using textile).

def q(value)
  value = '' if value.nil?
  if value.is_a? String
    escaped = value.gsub(/\\/, '\\\\').gsub(/\r\n/, "\n").gsub(/'/, "\\\\'").gsub(/\n/, '\' + "\\\\n" + \'')
    return "'#{escaped}'" 
  end
  value
end

task :typo_export => [:environment] do
  file = File.new("/path_to_craigs_simplelog/lib/tasks/import_from_typo.rake", 'w')  file << "task :import_from_typo => [:environment] do \n" 
  file << "  require 'find'\n" 
  file << "  require 'post'\n" 
  file << "  require 'preference'\n" 
  file << "  require 'application'\n"   file << "  Post.delete_all\n" 
  file << "  update_posts = \"\"\n" 
  for article in Article.find(:all)
    file << "  new_record = Post.create(:author_id => 2, :created_at => '#{article.created_at.to_s(:db)}'.to_time, :modified_at => '#{article.updated_at.to_s(:db)}'.to_time, :permalink => #{q article.permalink}, :title => #{q article.title}, :body_raw => #{q article.body}, :comment_status => 1, :text_filter => #{q 'textile'})\n" 
    file << "  update_posts += \"UPDATE posts SET id = #{article.id} WHERE id = \#{new_record.id}; \"\n\n" 
  end
  file << "  Post.connection.execute update_posts\n\n"   file "\n\n" 
  file << "  Comment.delete_all\n" 
  file << "  update_comments = \"\"\n"   for comment in Comment.find(:all)
    file << "  rew_record = Comment.create(:post_id => #{comment.article.id}, :name => #{q comment.author}, :email => #{q (comment.email.blank? ? 'anon@noemail.com' : comment.email)}, :subject => '', :body_raw => #{q comment.body}, :ip => #{q comment.ip}, :is_approved => 1, :url => #{q comment.url})\n" 
    file << "  update_comments += \"UPDATE comments SET id = #{comment.id}, is_approved = 1 WHERE id = \#{new_record.id}\"\n\n" 
  end
  file << "  Post.connection.execute update_comments\n\n"   file << "end\n" 
end

Once this is run, you can go to the simplelog installation and run rake import_from_typo. If it doesn’t work, figure out what went wrong, modify the script, and repeat. Please notice that this is a destructive import (note the calls to “delete_all”).

This is a long post already, so I’ll post comments on simplelog next time.