I’m using the file_column plugin a lot. It lets you treat a file as an attribute on one of your models, and all you need to do to store it is provide a string column in your database table.

A lot of people have expressed concern that file_column doesn’t seem to have been worked on by it’s author at all this year. This maybe so, but don’t let that deter you from the fact that it is still currently the simplest and best way to handle images in rails applications.

One of the first snags that I’ve hit with file_column so far took me a few hours to nut out today, so I’d like to share it with you, gentle reader, in the hope that it might save you the same effort. I had a model object, which contained a file_column (and thus, potentially, a file), and I wanted to clone it. As a result of this, I wanted the new object to have it’s own separate copy of the file.

If you just naively copy the file_column (in my case, called “image”, on a model called “gift”), then file_column warns you that you’re just giving it a string, and thus you probably forgot a multipart declaration on your form. As you can see, file_column is pretty focussed on uploading files, not copying them between models.

Enough waffle, let’s just straight to my final clone method:

    def clone
      result = Gift.new(:name => name, :price => price, :description => description)
      unless image.nil?
        result.image = File.open(image)
        FileUtils.cp(image, result.image)
      end
      result
    end

The main bit of magic here is that I’m passing file_column (on the new object), a file object, rather than just a string. The file_column plugin has some basic support for this, but it’s pretty basic indeed. It seems that it can manage to copy the file out of the temp location into the final destination when I actually save the new model (not done inside clone), but it doesn’t copy it into the temp dir correctly (that normally happens on update. So, I put the copy in myself, and it all works.

You may not want to duplication your images on clone, but obviously this same technique is useful for any copying of file_column files between models, of the same or different model class.

When I get some “free” time (heheh), I’d like to do a bit of work on file column and fix this so that the hack isn’t needed, and perhaps look at some other design improvements. I’ve been through just about every line of it’s code this year on various projects, and while overall I’m greatly impressed, I think there’s more work that could be done to improve it.

Post continues, click to read more...

Welcome back. This series is going through the process of setting up a scalable production VPS for a ruby on rails app, from a non-sysadmin perspective. The first post is here.

Since my last post, I’ve been doing a number of different things with my server in an attempt to find the best configuration and share it with you. Before I move forward with the specifics of that, I wanted to take a brief moment to share some of the things I’ve learnt while experimenting with this.

Load Balancing is Fun

I haven’t talked about it much yet, but you probably already know that for serious rails deployment, we’re talking about multiple instances of the Mongrel web-server, and some sort of software load balancer to pass the HTTP requests through to them.

I’ve tried out two different load balancers on my ‘experimentation server’. The first was Pound, which was nice because it’s configuration file really is just so damn simple (mainly because it does nothing but load balance). On the downside, the Ubuntu packages for it were too old to actually work nicelly, so I had to compile it. Then, I had to ensure that it restarted on reboot (I wrote my own init.d script). This isn’t a bad solution, but Pound doesn’t serve static content, and Mongrel isn’t well suited to it either, so it typically requires the installation of another web server as well, such as Apache or Lighttpd.

I also tried out using Apache2, with mod_load_balancer. At this point, I have a confession to make:

I was scared of apache configuration files.

There, I’ve said it. I’ve used apache for years, for static sites as well as PHP, and Rails, and it’s always worked fine. However, I always felt that I needed to stay ‘on the beaten path’ of the config that my linux distro provided for me, and look closely at other people’s recipes before considering anything but the simples change to my apache configuration.

Now you may well think that I’m a bit silly, but I’m well pleased to now be free of this notion. Apache configuration is very straightforward. I think it’s made a bit complicated by distros including config files fill of default setting which I don’t really need. Standard configuration of apache is simple, virtual hosts are simple, and mod load balancer is also straight forward to configure and seems to be rock solid to use.

So, I’m going to stick with the apache option, and in my next post on this subject, I’ll be talking about that setup in more detail.

RAM is Your Biggest Cost for Small Sites

VPS prices are heavily dependent on how much RAM you get. This is because servers can only have so much ram, and so the amount of available RAM tends to determine how many VPSs can fit on a single physical server.

I’m told, by people in the know, that this year we’re starting to see physical servers with a lot more available RAM than previously (ie; 32 gig instead of 4 gig). I’ll be interested to see if this means that we’ll see a lot more cheap VPSs which have insufficient CPU power. For now, however, I generally find that although I can max out the CPU on a small VPS by writting crappy code (for experimental purposes only, I assure you), it is RAM which is the most limiting resource.

One of the cheapes VPS options that I know of, is that US$20 plan from slicehost.com. That gives you 256 MB of RAM, which I’m quickly realising is not enough for a rails app with a bit of load. That’s right, 256 MB is not enough.

Why is this so? Well, Mongrel seems to be the main culprit. Mongrel keeps the whole rails stack in RAM, as it should to perform fast enough, which means that it ends up taking a pretty constant slice of the RAM. Unlike running a PHP site with Apache, RAM doesn’t really go down so much during light load, unless you dynamically reduce the number of Mongrels that you are running. So, this means that you get predictability, but it also means that you’re running pretty close to a permanent worse case scenario. Once you understand this, it’s a pretty good solution, proving you plan for it.

Zed Shaw has written some great stuff on how to find out how many mongrels you should run for your given situation. There is obviously a limit, because eventually you run out of CPU cycles so adding more mongrels doesn’t help. However, my rough guess at the moment is that initially you need a few, and with a 256 MB VPS, I only feel comfortable running TWO mongrel processes.

According to my quick and dirty figures, a mongrel process with a decent sized rails app requires about 60 MB. MySQL can gobble up RAM under load, growing from maybe 22 MB when it’s idle, to several times that when processing your nasty un-optimised active record queries. I’m a bit unsure about how much RAM apache needs under real load, but I’m currently estimating that at around 50 MB.

Problems with running out of RAM are problems of the nastiest sort, so you want to leave yourself a buffer. If you try and sneak a third Mongrel process into a 256 MB VPS, then you are running too close to the wire, and a single large database query might bring your site grinding to a halt. So don’t do it. Leave a good chunk free, and stick with two, or move to a 512 MB VPS. For a proper web application, even one just starting out, I’m currently recommending the latter.

Coming up Next

While it’s kind of fun investigating this stuff, my mate Mike Bailey (who knows far more about this than I do), has just gone and automated a lot of it with a new gem containing capistrano tasks. He’ll be announcing that in the next couple of days, and I’m testing it for him over the weekend. So, if things go well, I’m hoping that my next post includes some words about installing using that process. Anything that we do repeatedly, and particularly anything that can go wrong, should be automated.

Post continues, click to read more...

Welcome back. This series is going through the process of setting up a scalable production VPS for a ruby on rails app, from a non-sysadmin perspective. The first post is here.

Setting up Sudo

So last time we had a root login, and we’d created a non-root user for day to day use. Obviously we’re going to have a lot of tasks that require root access, we want this user to be able to execute commands as root, providing they enter their password, which is what the sudo command does.

On our cut down Ubuntu installation, sudo is not installed by default. Logged in as root, perform the following.

  apt-get update
  apt-get dist-upgrade
  apt-get install sudo
  visudo

The first line above updates our local directory of available software, followed by an upgrade of any packages that have changed since our version of Ubuntu. Then “apt-get install” installs the new software package sudo. Apt-get is the debian (and thus Ubuntu) package management system. It grabs software, and resolves dependencies, and makes the world a bright shiny place. We’ll be using it a lot today.

Once sudo is installed, the visudo command brings up a file which you can edit, allowing you to add your new user to the list of people who can use sudo. Add the following line to the file (replacing YOUR_USER with your non-root username):

  YOUR_USER ALL=(ALL) ALL

After that, test it out. Log in as YOUR_USER, and try prefixing a command with sudo, such as “sudo ps”. It should ask for YOUR_USER’s password the first time you do this for each session.

Using SSH keys

For increased security, it’s really recommended that you use ssh keys, rather than just passwords. If your local development machine is a *nix box of some sort (including a mac), then this is really straightforward. It’s basically a case of creating a private and public key pair locally, and then putting the public key on the server. This is described well elsewhere, so I suggest that you follow the instructions here:

http://www.ece.uci.edu/~chou/ssh-key.html

And then come back. You probably have SSH version 2. Unlike the article above, however, I strongly suggest setting a passphrase for your key (the article linked above just presses return, you aren’t going to be that silly). You should also ensure that the authorized_keys file has the minimum necessary permissions. Logged in as YOUR_USER, perform:

  chmod 0700 ~/.ssh
  chmod 0600 ~/.ssh/authorized_keys

When this is done, make sure that you can login as YOUR_USER, and you should only be asked for your ssh key passphrase locally, not your password. If everything is working fine, you can turn off passwords completely, meaning that you need your ssh key to login (so make a backup of it). You can also turn off external root logins (since we can use sudo).

In the file, /etc/ssh/sshd_config, set the following values:

  PasswordAuthentication no
  UsePAM no
  UseDNS no
  PermitRootLogin no

Installing Ruby and Rails

Now, this is an area that is very well documented. The best source of information, full of nice wordy explanations (much like this article), is on the rails wiki here:

http://wiki.rubyonrails.org/rails/pages/RailsOnUbuntu

I encourage you to go and read it, because I’m not going to repeat it all here. You can skip the SSL support and ImageMagick sections, unless you’re are using those libraries. Where the article has two alternatives, being “the proper way”, and “the recommended way”, I also recommend that you use “the recommended way”, which is to install rails using gems rather than debian packages.

As a quick reference, below is a summary of the steps to perform, taken directly from that guide.

Uncomment the following two lines from /etc/apt/sources.list (requires sudo).

  # deb http://archive.ubuntu.com/ubuntu/ dapper universe
  # deb-src http://archive.ubuntu.com/ubuntu/ dapper universe  # deb http://security.ubuntu.com/ubuntu dapper-security universe
  # deb-src http://security.ubuntu.com/ubuntu dapper-security universe

Execute the following commands.

  sudo apt-get update
  sudo apt-get install ruby irb ri rdoc ruby1.8-dev build-essential
  wget http://rubyforge.org/frs/download.php/11289/rubygems-0.9.0.tgz
  tar xzf rubygems-0.9.0.tgz
  cd rubygems-0.9.0
  sudo ruby setup.rb
  sudo gem update
  sudo gem install rails —include-dependencies

There you are, you should have rails working and installed. If you hit any problems, have a look at the actual wiki article, as I left out all the discussion in my quick cut and paste.

Installing MySQL

I’m no expert on the pros and cons of various databases, and so mysql is still my database of choice, mainly due to its ubiquitousness rather than any more well informed reason.

To install:

  sudo apt-get install libmysql-ruby mysql-server

Coming up Next

We’ll get our rails application running on a high port with a single mongrel process, to test that everything works. Also, we’ll talk a bit about good places to install such things, and we’ll make sure that we can deploy and upgrade our application with Capistrano.

Credits

I’m not actually responsible for most of the recipes above. Technical proof-reading has been supplied by my colleague Mike Bailey. Articles referenced include Mike’s currently unreleased rails setup recipes, and the following great wiki pages.

http://wiki.rubyonrails.org/rails/pages/RailsOnUbuntu
http://wiki.slicehost.com/doku.php?id=slice_setup_from_onrails.org

Post continues, click to read more...

This is the first part of a step-by-step case study as I walk through the technical issues surrounding setting up a production server for a rails app, and deploying it in a manageable way. I’m a programmer, not a system administrator, but the rapidly changing requirements for best-practices rails deployment seem to have created a need for specialist knowledge that neither sysadmins, or programmers, always have.

So, this is a guide written for programmers, with only a very basic knowledge of how to get around on a linux box. I’m going to talk about the reasons for my decisions, as well as just what shell commands I executed, and I’m going to dedicate each installment of this series of posts to one solid and fully working step on the path to having your dream app out there in the wild and running smoothly.

Arranging Hosting

Life Used to be Simple (with Shared Hosting)

If you’ve worked on PHP or Perl web sites, then like me you’ve probably used a lot of shared hosting. Shared hosting is cheap, because the hosting company doesn’t really have to tell you how many sites are on the same machine, providing that many of them don’t take up many resources. With static HTML, or even PHP and Perl, shared hosting actually works pretty well. You can’t guarantee uptime, or do anything really wacky, but it’s a great way to get your code out there without needing to know anything about administering a server.

Unfortunately, this approach doesn’t get you quite as far with rails. The problem is, most shared hosting is apache based, and rails just doesn’t run too well with Apache and fastcgi. Having said that, that’s the exact method used to host this blog, so clearly it’s not to be ruled out entirely.

All the Cool Kids Have Mongrels

At some point, however, you want a bit more speed and reliability. For me, it was because I wanted to release a new web application, and although I didn’t expect massive interest, I didn’t want to deal with scaling issues in the first few months.

Mongrel looks like the go. It’s the current darling of the rails crowd. It’s designed specifically to run rails, and “they” say that it does so very fast for quite a few major production applications. Also, it’s the same server that we generally use in development, and that helps me feel confident that I know how it’s going to perform.

Obviously I’ll be going into mongrel in more detail later, but the point to consider now is that suddenly we talking about running a fairly non-standard and long running process. We’re entering the area of managing our own server here, and that means no more shared hosting. Our application isn’t the next flickr just yet, so we can’t afford a dedicated machine, so the solution is a VPS.

VPS Hosting Options

A Virtual Private Server feels like using a whole machine, but obviously it’s only allocated a finite slice of the hardware resources. There are heaps of providers offering VPS hosting out there. There are a very select few, who actually know how to do the rails deployment thing, and will manage it for you (such as RailsMachine, and RailsPlayground). This is a pretty good option for many, although it will cost you about four or five times the price of doing it yourself. Still your time is worth money, so it might be worth leaving to the experts.

But, we haven’t done that have we. Otherwise I wouldn’t have written this guide, and you wouldn’t be reading it. So lets presume that you’re going to go ahead and buy a cheap bare bones VPS hosting package, and do the rest yourself. I went with slicehost.com. They’re dirt cheap, at US$20 per month for a VPS with 256 Mb of RAM. The also provide really simple tools for upgrading your account, re-creating your VPS, and picking your OS.

Of their choice of operating systems, I’ve picked Ubuntu Linux, Dapper (the latest major release). I don’t care for any arguments about which linux distros are the best, but my only linux knowledge comes from using it on the desktop, so I’m moderately familiar with Ubuntu. Also, it’s common enough that there is ubuntu specific information on the net about ruby on rails.

Our Shiny New VPS

We now have our very own operating system running in relative isolation on a machine humming away in a data-center somewhere. In the case of slice host, that machine is in Texas somewhere (which is the opposite side of the planet from me).

We’re going to approach our setup tasks in programmer oriented fashion. Slowly, carefully, and with an eye to how to automate things (because we’re essentially lazy people, so automation is our friend). With slice host, the first thing that you get is the root password, and they only display it once for you, so you need to write it down quickly, so that you can log in, and change it.

Although these setup instructions wont be specific to Slice Host, some of them will be specific to Ubuntu Dapper. In particular, Ubuntu on slicehost is initially setup with very little software installed. This is a common approach, and a good one, so hopefully we’ll only install things that we understand well enough to administer properly.

Getting Access

Changing the Root Password

At the moment, we have a root username and password, and it’s also a password that’s been transmitted across the web, so we really should change it. We also know that we want to do most of our work on the site as a non-root user. We’ll try and follow the philosophy of allowing the least access to the system needed, both for security reasons, and also to help prevent our own silly mistakes.

We were given a static IP address for our VPS. I’m going to use the string IP_ADDRESS in this text, and you can replace that with your servers IP address in any commands that I specify. We don’t have a domain name to use yet, so we’ll be using the IP by itself for a little while.

So, let’s log in:

    ssh root@IP_ADDRESS

First, we’re going to change that root password. Type the command “passwd”. It will ask for for a new password (twice), and then it will be set. You don’t need to use this password often, so don’t pick something simple that you use everyone else. Use a good solid password.

Creating a Non-Root User

Ok, we need a user that isn’t root. This is you, on the machine. You will want to use a username that makes sense to you, it also helps if it’s the same username as your local machine (if you use a *nix machine), since that saves you having to type it. I’ll refer to your username as YOUR_USER.

    adduser YOUR_USER
    su YOUR_USER
    passwd
    exit

What we did there was create the new user, then change to the new user using the “su” command, and change our password. Typing “exit” leaves us as root again.

Coming Up Next

We’ll go over using ssh keys to protect your login, and then installing ruby and rails and other required software on the new server.

Post continues, click to read more...