AWS CloudFormation is a service which provides users with a way to describe the AWS resources they want created, in a declarative and re-usable fashion, through the use of simple JSON or YAML formatted text files. It supports a wide variety of AWS services, includes the ability to pass in user supplied paramaters, and some functions you are able to use in the JSON files which can make your life simpler. AWS is home to everything from a few of our clients who are only running a few servers, to Netflix who runs thousands of servers delivering content to millions of customers, as well as everything in between.
AWS CloudFormation allows you to model your entire system as a “stack” in a JSON file. A “stack” is every AWS component necessary for your system to operate and it is bundled as a single atomic unit. This could be 15 EC2 servers, 3 ELBs, 10 IAM users, 6 S3 buckets, and any number of other components that are all configured to work together. This creates a change of perspective when thinking about your AWS resources. You no longer think of each component by itself and how to configure them to work together, but as a single holistic stack.
If you ever want to change any sub-component, you don’t change it directly; You modify the definition of the stack as a whole and then redeploy. Using this JSON definition file we can spin up any number of AWS components as a single unit. If there are any errors in the creation of the stack, AWS automatically rolls back every component. Think of it as a database transaction. If we create 10 EC2 Servers, 2 Elastic Load Balancers, and 15 Route53 DNS records successfully then the stack creation fails on creating an S3 bucket because you typed the name of the bucket incorrectly, AWS removes all of those 10 EC2 Servers, 2 Elastic Load Balancers, and 15 Route53 DNS records and notifies you of the failure. CloudFormation actions are treated as all-or-nothing transactions.
Now imagine that you no longer need the stack and you want to remove it. Before CloudFormation you would have to go to each service and delete all of the components by hand. This may be easy if you are running a small infrastructure, but imagine if you had hundreds or thousands of AWS components. It could take ages! With CloudFormation you can delete a stack and it removes all components as a single atomic unit.
Here is a snippet of a sample CloudFormation Template. I will dig deeper with each of these sections.
CloudFormation consists of six sections:
The AWSTemplateFormatVersion section identifies the capabilities of the template. The latest template format version is 2010-09-09 and is currently the only valid value. This section is optional.
The description allows you to provide arbitrary comments about your template. It is a JSON literal string that is between 0 and 1024 bytes in length. You cannot set the description via a parameter or function result.
The optional Parameters section enables you to pass values into your template at stack creation time. Using parameters you can replace hard-coded values with variables set at runtime. Parameters allow you to make your stack definitions more reusable.
Hard-coded one-scenario-use templates like this:
Become more modular and reusable like this:
Now the ‘InstanceType’ of the ‘EC2Node’ is set via a variable that AWS prompts you for at runtime.
Conditions are defined in the top-level ‘Conditions’ section of the CloudFormation template using intrinsic functions that evaluate to a boolean condition at runtime. You can then use those boolean values to control resource creation. It sounds a little complicated but let’s look at an example.
Consider an example where you have a staging and production stack. In the production stack you want to create an instance for Kibana log monitoring; You’ve decided in the staging stack you don’t really need that extra node.
Using a combination of ‘Parameters’ and ‘Conditions’ we can accomplish this without having to have two different stack definition files.
In this example AWS will ask you at runtime what you want to set the ‘EnvType’ parameter to. If you set it to ‘production’, the ‘CreateProdResources’ condition will evaluate to ‘true’ and when AWS evaluates the ‘KibanaNode’ resource definition, it will recognize the ‘true’ value of the ‘CreateProdResources’ condition and create the ‘KibanaNode’. Inversely, if you had set ‘EnvType’ to ‘staging’, causing the ‘CreateProdResources’ condition to evaluate to ‘false’, AWS would skip creating the ‘KibanaNode’ resource.
‘Mappings’ is an optional section where you can build a key value store to help simplify your stack definition file. A common usage is mapping AMI IDs to regions. In this example you need to pull the AMI available in the region you are currently working in.
After you have setup this mapping, you can access it in the resources section of the template.
In this example, the ‘Fn::FindInMap’ function will reference the AWS region you are currently working in via the globally available variable ‘AWS::Region’ and map that region to the correct AMI that you have available for that region.
As the previous sections were optional, the resources section is the required segment of the CloudFormation stack definition file where you declare all of the AWS resources you want as part of your stack.
In this example, we will setup a ‘t2.micro’ EC2 instance, an S3 Bucket, and an RDS Database:
You can setup any number of AWS resources in the ‘Resources’ section. Before getting started on your own resource definitions, you should read through the Resource Type documentation on Amazon’s official CloudFormation documentation. Another great resource is just digging around github for real world examples of CloudFormation being used in the wild.
The Outputs section enables you to return one or more values to the user in response to the AWS CloudFormation ‘describe-stacks’ command. To declare the Outputs section, the double-quoted key name Outputs is followed by a single colon. All outputs declared in the Outputs section are contained within a single set of braces, and are delimited by a comma.
You can use this section to get quick access to facts about your stack you may find useful in the future. Instead of going to the RDS section of AWS and looking up the node that is associated with the stack you are looking for so that you can find the public DNS name, you can just set that fact up as a fact in the outputs section like below:
Outputs are evaluated as the very last step after a successful stack build. Using the example above, ‘RDSDatabaseEndpointDetail’ will show up in the CloudFormation panel for quick reference.
AWS CloudFormation is an incredibly useful tool for building your AWS application stacks in a scaleable and manageable way. I’ve only touched the surface of introducing you to CloudFormation and as always encourage you to read through Amazon’s official docs.