Friday, April 2, 2010

EC2 Marks the Spot

I've been using Amazon EC2 at work and for personal projects for about three years now. It's got to the stage where I can't remember how we used to manage without being able to spin up a test or development machine on demand.

One downside of EC2 compared to other hosting options is the price. The cheapest on-demand Linux instance costs 8.5 cents per hour, which works out to just over $60 a month - a bit expensive if you just want to run something like a web server and don't need full sytem access. However, Amazon just came up with a way to cut this cost significantly by introducing Spot Instances.

You can read all the details by following the link, but the basic idea is that you bid on spare EC2 capacity by specifying the maximum price per hour that you're prepared to pay. If the current spot price (which varies continually based on supply and demand) is less than your bid, Amazon will start up your instance and keep it running until the spot price becomes higher than your bid. At that point your instance will be terminated.

The surprising thing I found after monitoring the small Linux instance spot price for a few days is that it's a LOT less than the on-demand price - it's stayed between 2.9c and 3.1c per hour. That means that if you bid, say, 5 cents per hour for your spot instance, you'll be pretty certain of getting a cheap, long-running instance unless there is a sudden spike in demand and the spot price goes over 5c / hour.

The only down side I've found so far, apart from the necessity to be prepared for the instance shutting down without warning (which you ought to do for all EC2 instances anyway) is that you seem to have to wait a while longer for your instance to be started up; the ones I tried took one or two hours, rather than the usual 15 minutes or so.

As a way to warn me if the EC2 spot price is getting close to my bid price, I wrote a Nagios plugin, check_ec2_spot_price, that will send me a warning if the spot price goes above a specified value. You can download it from the Nagios Exchange.

I'm also working on a Munin plugin that will graph the EC2 spot price over time; I'll post that here too when it's ready.