Welcome to EMC Consulting Blogs

Before I go on I apologise up front but I just can’t resist the cooking theme

The purpose of this series of blog posts is to explore using Chef to configure instances in the cloud. My particular cloud for this just happens to be AWS.

In this post I’ll be describing how to actually use Chef to spin up & configure an ec2 instance. I am making the assumption you are familiar with the contents of the previous two posts in this series. Even if you don’t have any experience with  AWS I hope it will still be easy enough to follow.

In my last post I described the configuration of a windows workstation to prepare for this but I hit a problem which meant I have had to resort to using a Linux instance running on AWS. I have a bug logged with OpsCode. For those of you who are keen to make use of your windows machines as your user environment I will post an update when the bug is resolved.

I am starting off with spinning up a Linux instance as it will be easier to grasp the concepts and what you need to do to use Chef effectively. My last post in this series will deal with using chef with a windows target.

The first thing I needed to do was walk through what I wanted to achieve , in what order .

I wanted to  spin up an ec2 instance and install tomcat so Chef will need to:

·         Fire up an ec2 instance

·         Bootstrap it so it can run a chef-client

·         Install java  if required ( Note as I am using an aws linux instance java is preinstalled)

·         Install tomcat

Yes I know this can all be easily done using cloudinit and passing user data to your instance but I feel it’s pretty self evident how Chef or a similar tool would be a more appropriate enterprise solution with a large target ec2 estate with various roles .  The simplicity of the example is to illustrate the steps that need to be gone through.

The really cool thing with Chef is that you can rerun cookbooks against a node and it will not do anything it has already done i.e it will not change the end result on the target node as defined by the recipes being run against it. So you will always get the same outcome no matter what state the node and actions will not be taken if already done (and conversely run if detected it has not been run).  When reading about Chef you will see this described as being idempotent (There I’ve saved you looking it up).

At this stage I have a chef ready user environment, an OpsCode organisation set up and now I want to start by spinning up an ec2 instance.

I will just be talking about the key chef configuration steps for the rest of this post but if you are following along then use the table at the end of this post that details all he steps I followed. I will not be going into any depth re the ec2 specifics as that would make this post far too long.

The first thing I needed to do was create a cookbook for installing tomcat.  The Chef community (admittedly in the opensource world) though is full of off the shelf recipes that you can use to get started with.

I did a quick search from the OpsCode cookbooks section of the web console to see if there were any tomcat cookbooks 

I could have downloaded it from the console but I feel it was neater to use knife .

I also need the cookbook java  and apt cookbooks .

Before proceeding it would probably be a good idea to take time out and read the Opscode  Chef Recipe wiki   which has a nice clear explanation on cookbook name spaces. Also remind yourself of the components that make up a cookbook  it’s worth noting that recipes manage resources and those resources will be executed in the order they occur.

As my target platform will be ‘amazon’ I decided to amend the default.rb to reflect that platform.  The Amazon AWS linux ami is based on centos so it was a minor change to add the platform.  I  did not need to do this modification as the centos bootstrap will work fine with no changes  but I felt it was a worthwhile exercise as it shows the sorts of steps you would need to go through typically.

I then downloaded the Chef repository so that I could obtain a template bootstrap file. The purpose of the bootstrap file is to install chef on the target node and set the node up as a chef client

At this stage I have a cookbook with a recipe ready to upload and run and I have a bootstrap file.  Nearly ready to try it all out but before I stitch it all together I need to introduce you to the concept of roles.

I then uploaded my modified tomcat cookbook to my OpsCode chef instance and created a role that would basically run the tomcat recipe in this cookbook.

I then ran the following command to instantiate my ec2 instance:

knife ec2 server create "role[tomcat]"  --region eu-west-1  -Z eu-west-1a  -i  ami-75d4e101 -f t1.micro -G GraceHome -I ~/.ssh/GM-EC2.pem -S GM-EC2 --ssh-user ec2-user -d amazon-gems

Note: see table at end for explanation of the ec2 specific flags

On running the command a long series of actions occur which you can review via a log file

When I saw  the following I then verified everything was  completed as expected:

The new node appears in my node list:

I can now consistently reproduce as many tomcat nodes as I want. What I’m not too sure of is how I can tie this in with AWS  auto scaling  but figure that’s something to look into later.

The table below assumes that your user environment workstation is also set up to use the ec2 api tools

Table 1: Configuration steps for starting an  AWS Amazon  Linux ec2 instance and installing tomcat