Showing posts from May, 2011

Setting up RAID 0 across ephemeral drives on EC2 instances (and surviving reboots!)

I've been experimenting with setting up RAID 0 across ephemeral drives on EC2 instances. The initial setup, be it with mdadm and lvm, or directly with lvm, is not that hard -- what has proven challenging is surviving reboots. Unless you perform certain tricks, your EC2 instance will be blissfully unaware of its new setup after a reboot. What's more, if you try to mount the new striped volume at boot time by adding it to /etc/fstab, chances are you won't even be able to ssh into the instance anymore. It happened to me many times while experimenting, hence this blog post.

Update: I realize I didn't go into details about the use case of this type of setup. This is useful if you don't want to incur EBS performance and reliability penalties, and yet you have a data set that is larger than the 400 GB offered by an individual ephemeral drive. Of course, if your instance dies, so do the ephemeral drives (after all they are named like this for a reason...) -- so make sure yo…

Managing infrastructures in the cloud, with lessons learned the hard way

Here is a collection of blog posts I wrote over the last 3 years or so. Some of them are practical step-by-step tutorials on using various tools for managing cloud instances, while others talk about lessons learned the hard way, by deploying large-scale infrastructures in the cloud. I am aggregating them here for ease of future reference:

Lessons learned
Experiences deploying a large-scale infrastructure in Amazon EC2 (April 2009)Lessons learned from deploying a production database in EC2 (April 2011)Dark launching and other lessons from Facebook on massive deployments (July 2009)You're not a cloud provider if you don't provide an API (February 2009)Working with EC2-specific tools Experiences with Amazon EC2 and EBS (September 2008)Update on EC2 and EBS (October 2008)Deploying EC2 instances from the command line (December 2008)Working with Amazon EC2 regions (December 2008)Load balancing (ELB and HAProxy) Using AWS Elastic Load Balancing with a password-protected site (January 201…

Upgrading the GD library in Ubuntu

We needed to use ImageFlow for some internal testing of image manipulations (esp. reflections). With a stock php5/libgd2 install in Ubuntu 10.04, some calls to the ImageFlow library would fail with:

"GD library is too old. Version 2.0.1 or later is required, and 2.0.28 is strongly recommended."
The libraries installed by Ubuntu were:
$ dpkg -l | grep libgd2 rc libgd2-noxpm 2.0.36~rc1~dfsg-3ubuntu1.9.04.1 GD Graphics Library version 2 (without XPM s ii libgd2-xpm 2.0.36~rc1~dfsg-3ubuntu1.9.04.1 GD Graphics Library version 2 $ dpkg -l | grep php5-gd ii php5-gd 5.2.6.dfsg.1-3ubuntu4.6 GD module for php5 The issue here is that Ubuntu does not use the version of GD which is bundled with PHP. See this discussion for more details.

So...some googling around later, I stumbled on this great howtoforge post by patusovniak on "Recompiling PHP5 with bund…