ChisBlog

ATLRUG Chef Presentation

Last week I gave a presentation to the Atlanta Ruby User’s Group discussing Maxmedia’s use of Opscode Chef (and the Opscode Platform) for managing our Rackspace Cloud infrastructure. The presentation was titled “Scaling People: The 5 Ws of Chef @ MaxMedia” and talks about how Chef not only helps scale one’s infrastructure but can also help scale a small team of people:

Lucky for all of you my ugly mug never appears on camera but…you still have to listen to my voice! If you live in Atlanta and are even remotely interested in Ruby I highly recommend making the monthly ATLRUG meetings….if for nothing else come for the free pizza!

Nodes 2 Hosts

As MaxMedia’s virtualized infrastructure grows, I’ve found myself tiring of looking up IP addresses every time I need to SSH into a box. Deciding it was time to work smarter, I threw together a quick ruby script that will generate a local hosts file that maps a nodes ip address to it’s fully qualified domain name (fqdn). This task was a piece of cake with Knife (Chef’s command-line utility), the Chef server search index, and a little ruby-foo:

Just paste the output into your Mac’s /etc/hosts file and your ready to roll.

Cooking with Opscode Chef

My company, MaxMedia, has been in the process of moving our traditional co-located hosting to a virtualized cloud-based platform (we chose the Rackspace Cloud). It’s been great way for a small interactive agency to save some money and reinvest it in neat toys like the 16 TB ZFS JBOD we built for our motion team! The tool we have chosen to manage all of this is virtualization is Opscode Chef.

Chef is a state-based, configuration management tool written entirely in Ruby. It is scalable and completely open-source. The 37 Signal’s team describes it as “Rails for sysadmins”! With Chef you don’t write scripts to configure your infrastructure you write code that programs it. Chef builds on the following core concepts:

• Nodes - these are the actual servers you want to configure. Chef performs all work down on the nodes…this makes scaling Chef to large multi-thousand node deployments easy. A node has a list of recipes and a bag of attributes (both discussed below).
• Recipes - represent a declarative description of how you want part of a node to be configured. They are written in an easy to understand Ruby DSL and allow you to drop into regular Ruby anytime you want. These recipes describe a series of resources that should be in a particular state - for example, packages that should be installed, services that should be running, or files that should be written.
• Cookbooks - Recipes (along with templates and attributes) are grouped together into Cookbooks. They should be as generic as possible and work across many different nodes and platforms (ie Ubuntu and CentOS). A cookbook is also the Chef unit of distribution, shared among the community, that live in a central Git repository.
• Attributes - used to differentiate the same cookbook on two different nodes. The attributes in a cookbook represent a set of sane default values for “tunable” settings which are then overridden by a node or role. Attributes also persist between Chef runs.
• Roles - roles represent a higher level of functionality…something like an application server or a database server. A role may contain multiple cookbooks and be applied across multiple nodes.

Being a developer turned infrastructure architect, it’s been nice to write familiar Ruby code that describes every aspect of our infrastructure. This Chef code also acts as documentation for that infrastructure, which can be rebuilt at any point in time. This has opened up some amazing possibilities: think about redeploying data centers or changing cloud providers in minutes…not weeks.

In the spirit of the growing Chef community I’ve also written a few Chef cookbooks which are free for everyone to use:

• MongoDB - installs and configures MongoDb on a node.
• Scout Agent - installs the Scout Agent gem and schedules it to run via Cron.

Fork them and add support for RHEL/CentOS (we’re an Ubuntu shop!) or some other platform!

Change the Time Zone in the Cloud

We recently ran into an issue with one of our virtual servers running up in the Rackspace Cloud . Some legacy application code required the time zone on our server to be set to EST but we could not get the UTC offset to stick. We tried all the usual suspects on our Ubuntu 8.04 LTS instance:

• changed /etc/default/rcS
• ran /usr/bin/tzselect

….nothing seemed to worked. My sysadmin and I were at a loss, but we finally stumbled across a site with some good advice about setting a time zone in the cloud:

As a virtual server hosted using Xen virtualization technology, such as those offered by RackSpace Cloud Server offer, system time will get reset by Xen to match the physical machine’s clock whenever the virtual server is restarted.

This time-reset behavior can be modified if you can change the Xen configuration. However as a user of the cloud computing service, you most likely do not have such access.

You can however do the following to set your system to the time you want. Changing the time zone of your system will set your system time to the correct time of that time zone, given that your virtualization provider gives your server a correct UTC time.

Here’s how we finally made the offset stick on our configuration:

…bonus points…no reboot required