Jake@Nitobi

We’ve been looking at ways to get one of our Rails apps, RobotReplay, a web ‘cinelytics’ package to scale out as we add users. EC2 is an interesting possibility. There are a few issues with deploying that we need to deal with. I’ll detail them in this post and update it as we start making headway.

What’s my IP?

The central problem for deploying RobotReplay as a multi-server application on EC2 was the number of locations we need the other servers’ IP addresses. On an EC2 instance you can get your global IP by calling an EC2 web service:

curl http://169.254.169.254/latest/meta-data/public-ipv4

You can get the local IP address for inter-EC2 instance communication by calling another service:

curl http://169.254.169.254/latest/meta-data/local-ipv4

(You can also use ifconfig if that’s better for you).

Server Replication

Each new DB node needs to insert itself in a Multi-Master ring of MySQL databases. What that means is that we need to keep track of the IPs in the ring, and modify both the new instance, and it’s two new neighbors. We use memcached to cache in-progress recording sessions, so we also need to modify every server’s environment.rb’s to add the new memcached server. Another choice is to ignore that multi-server aspect of memcached and make sure that a session is always routed to the same appserver by our load balancer. Using HAProxy, the load balancer needs to be updated with information about the new server. That’s why we’d love to move to Swiftiply, a proxy that lets backend processes announce their availability.

Persistence

There’s nothing up there that’s too daunting, really nothing out of the ordinary for any multi-server app. But on top we add the impermanence of EC2 instances. If one goes down our data won’t suffer because we’re replicating it, but the Master-Master replication loop is broken. The hassle is that bringing up another instance won’t bring it up with the same old IP, so all your settings need to be updated: for memcached, for your load balancer, and for your database ring. Don’t forget actually deploying changes to your app. Capistrano is great, awesome, wicked, but needing to rewrite your deploy.rb as you add and remove servers on EC2 is teh sux.

Spoiled?

I’m spoiled by getting to work in Ruby all day - doing the heavy lifting of actually RUNNING a web app is considerably less fun. The tools for Rails and EC2 are happening, but it seems like that’s just the tip of the iceberg.

This entry was posted on Thursday, June 7th, 2007 at 3:04 pm and is filed under Development, RobotReplay. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.