ChisBlog

Nodes 2 Hosts

As MaxMedia’s virtualized infrastructure grows, I’ve found myself tiring of looking up IP addresses every time I need to SSH into a box. Deciding it was time to work smarter, I threw together a quick ruby script that will generate a local hosts file that maps a nodes ip address to it’s fully qualified domain name (fqdn). This task was a piece of cake with Knife (Chef’s command-line utility), the Chef server search index, and a little ruby-foo:

Just paste the output into your Mac’s /etc/hosts file and your ready to roll.

Cooking with Opscode Chef

My company, MaxMedia, has been in the process of moving our traditional co-located hosting to a virtualized cloud-based platform (we chose the Rackspace Cloud). It’s been great way for a small interactive agency to save some money and reinvest it in neat toys like the 16 TB ZFS JBOD we built for our motion team! The tool we have chosen to manage all of this is virtualization is Opscode Chef.

Chef is a state-based, configuration management tool written entirely in Ruby. It is scalable and completely open-source. The 37 Signal’s team describes it as “Rails for sysadmins”! With Chef you don’t write scripts to configure your infrastructure you write code that programs it. Chef builds on the following core concepts:

• Nodes - these are the actual servers you want to configure. Chef performs all work down on the nodes…this makes scaling Chef to large multi-thousand node deployments easy. A node has a list of recipes and a bag of attributes (both discussed below).
• Recipes - represent a declarative description of how you want part of a node to be configured. They are written in an easy to understand Ruby DSL and allow you to drop into regular Ruby anytime you want. These recipes describe a series of resources that should be in a particular state - for example, packages that should be installed, services that should be running, or files that should be written.
• Cookbooks - Recipes (along with templates and attributes) are grouped together into Cookbooks. They should be as generic as possible and work across many different nodes and platforms (ie Ubuntu and CentOS). A cookbook is also the Chef unit of distribution, shared among the community, that live in a central Git repository.
• Attributes - used to differentiate the same cookbook on two different nodes. The attributes in a cookbook represent a set of sane default values for “tunable” settings which are then overridden by a node or role. Attributes also persist between Chef runs.
• Roles - roles represent a higher level of functionality…something like an application server or a database server. A role may contain multiple cookbooks and be applied across multiple nodes.

Being a developer turned infrastructure architect, it’s been nice to write familiar Ruby code that describes every aspect of our infrastructure. This Chef code also acts as documentation for that infrastructure, which can be rebuilt at any point in time. This has opened up some amazing possibilities: think about redeploying data centers or changing cloud providers in minutes…not weeks.

In the spirit of the growing Chef community I’ve also written a few Chef cookbooks which are free for everyone to use:

• MongoDB - installs and configures MongoDb on a node.
• Scout Agent - installs the Scout Agent gem and schedules it to run via Cron.

Fork them and add support for RHEL/CentOS (we’re an Ubuntu shop!) or some other platform!

Change the Time Zone in the Cloud

We recently ran into an issue with one of our virtual servers running up in the Rackspace Cloud . Some legacy application code required the time zone on our server to be set to EST but we could not get the UTC offset to stick. We tried all the usual suspects on our Ubuntu 8.04 LTS instance:

• changed /etc/default/rcS
• ran /usr/bin/tzselect

….nothing seemed to worked. My sysadmin and I were at a loss, but we finally stumbled across a site with some good advice about setting a time zone in the cloud:

As a virtual server hosted using Xen virtualization technology, such as those offered by RackSpace Cloud Server offer, system time will get reset by Xen to match the physical machine’s clock whenever the virtual server is restarted.

This time-reset behavior can be modified if you can change the Xen configuration. However as a user of the cloud computing service, you most likely do not have such access.

You can however do the following to set your system to the time you want. Changing the time zone of your system will set your system time to the correct time of that time zone, given that your virtualization provider gives your server a correct UTC time.

Here’s how we finally made the offset stick on our configuration:

…bonus points…no reboot required

What the heck is NoSQL?? => my takeaways from NoSQL East 2009

This past week I was lucky enough to attend NoSQL East 2009 which was not only the east coast’s first major NoSQL conference but one of the first conferences for the movement as a whole.  Thanks for everyone that put NoSQL East together, in particular Brad Anderson and Chris Williams.  There are a lot of in-depth conference reports out there, written by people way smarter than me, so I’ve decided to do a high level overview of what the heck “NoSQL” even means.  Much of the credit/inspiration for this post goes to Emil Eifrem (creator of Neo4j) who gave a great talk about Neo4j and the NoSQL movement in general.

Most people assume NoSQL literally means “no more sql” or “replace all relational database with some shiny new non-relational toy”.  But the reality is the movement is less of an coup and more of a new-world colonization.  “Not Only SQL” (@emileeifrem), and “NoSQL is not !SQL” (@rklophaus) were two of my favorite movement clarifications batted around at the conference.  The basic gist here is a NoSQL approach/architecture should compliment, not replace, a traditional relational database architecture.

Emile Eifrem broke the NoSQL movement into the following emerging categories:

• Key/Value Stores
  • inspiration => based on the theories laid forth in Amazon’s Dynamo paper
  • data model => global collection of key/value pairs
  • projects => Riak, Dynomite, Voldemort, Tokyo Cabinet, Redis
• BigTable Clones
  • inspiration => based on Google’s BigTable whitepaper
  • data model => big table column families (think huge spreadsheets)
  • projects => HBase, Hypertable, Cassandra
• Document Databases
  • inspiration => chiefly inspired by Lotus Notes (and it’s kick ass replication engine)
  • data model => a collection of key/value collections
  • projects => CouchDB, MongoDB
• Graph Databases
  • inspiration => based on the theories from Euler’s Graph Theory
  • data model => nodes, relationships and key/values on both
  • projects => Neo4j

I’m sure I left out someone’s favorite NoSQL project…mostly because there are so damn many of them out there!  The running joke throughout the whole conference was 2009’s trend of rolling your own NoSQL product was like the last few years trend of rolling your own web framework.

The final thought I want to leave with everyone is this: a lot of the NoSQL data stores not only scale way UP…but many can also scale way DOWN.  Neo4j is a small 500k jar which runs mostly in memory…one of CouchDB’s use cases is running on mobile devices (bonus peer-to-peer replication).  To me this is f-ing awesome…the same software leveraged in large-scale internet companies like Digg, Twitter and Facebook has a home in a small interactive agency like MaxMedia.