Create both Development and Production-Ready AWS EKS Clusters using AWS CDK

As a Xerris Solutions Architect, I sometimes get customers asking about how to maintain a Kurbernetes cluster in AWS the easiest way possible. Kubernetes is becoming the de-facto standard for running container workloads and provides many benefits over traditional virtual machine based architectures. It enables you to scale your compute resources seamlessly while providing fast development and deployment cycles and fast rollbacks. The cost of this previously has been the high levels of administration that come with maintaining your own cluster but AWS has a managed service that aim’s just to ease this problem.

AWS EKS AND CDK

AWS EKS is a fully managed Kubernetes service that frees you from having to deal with the day-to-day cluster maintenance and instead let’s you focus on the applications running on your cluster. It also is deeply integrated into the AWS ecosystem with services such as Amazon CloudWatch, Auto Scaling Groups, AWS Identity and Access Management (IAM), and Amazon Virtual Private Cloud (VPC). In addition to this, AWS will automatically apply the latest security patches to your cluster so you can know that any known vulnerabilities are taken care of.

When it comes to deploying your cluster you have a couple options as well including Cloudformation and Terraform but the tool that I am going to use here is called the AWS Cloud Development Kit (CDK). It allows you to describe your infrastructure using existing programming languages like C# or Python.

In this post, we will set up both a cost effective development cluster and a highly-available production cluster from scratch using CDK and C#.

Prerequisites

Before we start we need to get our CLI setup. Install the AWS CLI on your machine with a user who has been given AdministratorAccess. This is due to the extensive access that is needed for the CDK. In a production environment this should be limited to only the needed permissions for creating the infrastructure. Now we need to install the CDK which can be done through npm:

npm install -g aws-cdk

Now let’s create a CDK project:

cdk init app --language dotnet

You can open up your project in Visual Studio and install the CDK NuGet Package and we can get started. It’s going to be a lot of code but at the end it should all come together into a neat and maintainable way to manage your infrastructure.

VPC Setup

First let’s start by setting up the VPC that our cluster will reside in. We are setting it up with large subnets for future growth in our cluster as well as creating them over 4 AZ’s for maximum availability.

ECR Setup

We will need a place to store our container images so we will create an ECR repository for each deployment environment we plan to run in.

EKS Setup

Now we have to actually create our cluster. This involves setting up an administrator IAM role to access the cluster as well as outputs that allow us to quickly extract these values and login after the stack has finished creating.

Node Group Setup

Here we are setting up an Abstract class that both our development and production node groups can inherit from. In here we also describe the basic autoscaling policy that both will use.

Development Node Group Setup

Spot instances are a great way to save money on workloads that can be terminated without notice. While a production environment might not fit here, a development workload is a perfect situation to leverage spot instances to save cost. Here we are calculating our spot price with the list price and discount then launching our cluster with md5.large nodes in an ASG.

Production Node Group Setup

For our production node group we want to maximize availability and reliability. We do this by creating our node groups across all AZ’s in our VPC . We also define our autoscaler manifest which will be created after cluster creation.

Putting it all Together

Here we put all the pieces together into two stacks. A development stack with the spot instance node group as well as a production stack with the high availability node group.

Deploying our Infrastructure

To deploy the development environment all you need to do it run:

cdk bootstrap
cdk deploy dev-demo

Or the production environment:

cdk bootstrap
cdk deploy prod-demo

Then once everything is deployed it will output what you need to configure your kubectl to connect to your cluster. The format will be similar to:

aws eks update-kubeconfig --name demo--role-arn arn:aws:iam::123456789:role/dev-demo-cluster-cluster-administrator --region us-west-2

Then to start the autoscaler:

kubectl apply -f autoscaler.yaml

Deleting our Infrastructure

Once you are done you can delete everything by going:

cdk delete dev-demo
cdk delete prod-demo

Conclusions

Now that we have our cluster up and running we can now look at tools to deploy our images like Flux or monitoring tools like Prometheus. This is out of scope for this post but the sky is the limit with Kubernetes and it’s extensibility leads to lots of great workflows. Thank you for reading and you can find the full CDK code here. If you want to find out more about scaling up your infrastructure using Kubernetes feel free to get in touch with us at Xerris and we can help you craft innovative cloud focused solutions for your business.