Amazon EC2 Metadata

Question

We're in the process of migrating from a datacenter to Amazon. We're a small company and rather than upgrade our hardware, we've found it to be enticing to move to "the cloud." We've put together custom AMI's from scratch and are currently in process of deciding on how to configure the environments.

I've been using Amazon's management console and Elasticfox to manage the resources but I've run into an issue of longer term management. When we have multiple servers running, it's difficult to tell which are which. The same issue exists with EBS resources and their snapshots. There doesn't seem to be any way through the AMI tools to add metadata to the resources to differentiate them with nice alias. I read a response to another question where security groups were used to "name" the AMI instances, but that still leaves me with EBS/snapshot management.

I know there are services out there like RightScale and Scalr that I believe add these features, but I'm wondering how others are handling this on their own?

Answer 1

The easiest way to name your servers and EBS volumes / snapshots is to use RightScale's free service to manage your instances; this lets you create an alias for each running EC2 instance, EBS volume or snapshot.

If you don't want to use RightScale, you could pass in user data to your instance at startup time and use this to set the host name or some other value that would let you uniquely identify the instance. I'm not sure what you would do for EBS volumes, though. Take a look here for more details (the AWS doc seems to use URL masking, so go to Using Amazon EC2 / Launching and Using Instances / Instance Metadata)

Answer 2

You may have already found a solution but I thought I'd answer just in case.

I started an open source project a few months ago to help organize EC2 infrastructures. It stores metadata in SimpleDB.

You create your machine configuration in a Ruby DSL where everything is organized into environments and roles. Here's a typical configuration:

env :stage do
  size 'm1.small'                # Default EC2 machine type for the 'stage'

  role :app do
    positions 1                  # Only 1 machine
    addresses '11.22.33.44'      # Define an elastic IP

    disks do                     # Define EBS volumes
      path "/rudy/disk1" do      # The path to mount
        size 100                 # The size in GB
        device "/dev/sdr"        # The unique disk device
      end
    end
  end
  role :db do
    size 'm1.large'              # Use more powerful machine for db
    ami 'ami-dc1038a8'           # A 64-bit debian
  end
end

You can then launch the environment from the command-line:

$ rudy startup
The following machines were started:
m-us-east-1b-stage-app-01  ec2-11-22-33-44.us-east-1.compute.amazonaws.com

$ rudy -r db startup
The following machines were started:
m-us-east-1b-stage-db-01  ec2-79-125-50-26.us-east-1.compute.amazonaws.com

$ rudy machines
m-us-east-1b-stage-app-01  ec2-11-22-33-44.us-east-1.compute.amazonaws.com
m-us-east-1b-stage-db-01  ec2-79-125-50-26.us-east-1.compute.amazonaws.com

$ rudy disks 
disk-us-east-1b-stage-app-01-rudy-disk1  vol-eee10486;  100GB; /dev/sdr; mounted

You can login with:

$ rudy -u root ssh

The project is called Rudy. Here are a couple links for more info and feel free to contact me directly if you have any questions:

• Getting Started
• GitHub Project