18
Provisioning an RDS Database with CloudFormation
In part 1, we automated the provisioning of your Amazon EC2 instance using AWS CloudFormation. When you built the EC2 instance manually in the past, you were seeing inconsistencies between environments, had to manually test your infrastructure setup, manually deploy your team's web app, which happens all to infrequently. All of this has been error prone and time consuming. Bugs introduced into production code are more difficult and expensive to fix and your customers were ultimately the ones who suffered.
In addition to the EC2 instance, you also need a Postgresql database. In the past, it was running on the same instance as your webapp and sometimes web app traffic impacted the database and vice versa. In this post, part 2, you'll add an Amazon RDS Postgresql database to the CloudFormation template you built in part 1 so that both the EC2 instance and the database can be provisioned together as a set of resources. And since RDS is a managed service, you won’t have to do operating system patches or database patches — it’s all managed for you. And eventually, you want to introduce a continuous integration/continuous deployment (CI/CD) pipeline to automate the build, test, and deploy phases of your release process. Both part 1 and part 2 set you up to do that.
As a reminder, here's what we covered and where we're going:
- automate the provisioning of your Amazon EC2 instance using AWS CloudFormation (part 1),
- add an Amazon RDS Postgresql database to your stack with CloudFormation (this post, part 2), and
- create an AWS CodePipeline with CloudFormation (part 3).
To work through the examples in this post, you’ll need:
- an AWS account (you can create your account here if you don’t already have one),
- the AWS CLI installed (you can find instructions for installing the AWS CLI here), and
- a key-pair to use for SSH (you can create a key-pair following these instructions).
Unfamiliar with CloudFormation or feeling a little rusty? Check out part 1 or my Intro to CloudFormation post before getting started.
First we'll add the RDS database and then we'll add a security group to allow inbound traffic on port 5432. At the end of this post, you’ll delete the stack you’ve created and any snapshots so that you don’t incur any charges and then you can (quickly) customize and recreate the stack in the future.
Just want the code? Grab it here.
Let’s get started!
First, we'll add an RDS database resource with the type AWS::RDS::DBInstance to the CloudFormation template. We set the Engine to the database engine we want to use, in this case postgres.
# 05_rds.yaml
AWSTemplateFormatVersion: 2010-09-09
Description: Part 2 - Add a database with CloudFormation
Parameters:
AvailabilityZone:
Type: AWS::EC2::AvailabilityZone::Name
EnvironmentType:
Description: 'Specify the Environment type of the stack.'
Type: String
Default: dev
AllowedValues:
- dev
- test
- prod
AmiID:
Type: AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>
Description: 'The ID of the AMI.'
Default: /aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2
KeyPairName:
Type: String
Description: The name of an existing Amazon EC2 key pair in this region to use to SSH into the Amazon EC2 instances.
DBInstanceIdentifier:
Type: String
Default: 'webapp-db'
DBUsername:
NoEcho: 'true'
Description: Username for Postgresql database access
Type: String
MinLength: '1'
MaxLength: '16'
AllowedPattern: '[a-zA-Z][a-zA-Z0-9]*'
ConstraintDescription: Must begin with a letter and contain only alphanumeric characters.
Default: 'postgres'
DBPassword:
NoEcho: 'true'
Description: Password Postgresql database access
Type: String
MinLength: '8'
MaxLength: '41'
AllowedPattern: '[a-zA-Z0-9]*'
ConstraintDescription: Must contain only alphanumeric characters.
Mappings:
EnvironmentToInstanceType:
dev:
InstanceType: t2.nano
test:
InstanceType: t2.micro
prod:
InstanceType: t2.small
Resources:
WebAppInstance:
Type: AWS::EC2::Instance
Properties:
AvailabilityZone: !Ref AvailabilityZone
ImageId: !Ref AmiID
InstanceType: !FindInMap [EnvironmentToInstanceType, !Ref EnvironmentType, InstanceType]
KeyName: !Ref KeyPairName
SecurityGroupIds:
- !Ref WebAppSecurityGroup
WebAppSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: !Join [ '-', [ webapp-security-group, !Ref EnvironmentType ] ]
GroupDescription: 'Allow HTTP/HTTPS and SSH inbound and outbound traffic'
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 80
ToPort: 80
CidrIp: 0.0.0.0/0
- IpProtocol: tcp
FromPort: 443
ToPort: 443
CidrIp: 0.0.0.0/0
- IpProtocol: tcp
FromPort: 22
ToPort: 22
CidrIp: 0.0.0.0/0
WebAppEIP:
Type: AWS::EC2::EIP
Properties:
Domain: vpc
InstanceId: !Ref WebAppInstance
Tags:
- Key: Name
Value: !Join [ '-', [ webapp-eip, !Ref EnvironmentType ] ]
WebAppDatabase:
Type: AWS::RDS::DBInstance
Properties:
DBInstanceIdentifier: !Ref DBInstanceIdentifier
AllocatedStorage: '5'
DBInstanceClass: db.t2.small
Engine: postgres
MasterUsername: !Ref DBUsername
MasterUserPassword: !Ref DBPassword
Tags:
- Key: Name
Value: !Join [ '-', [ webapp-rds, !Ref EnvironmentType ] ]
DeletionPolicy: Snapshot
UpdateReplacePolicy: Snapshot
Outputs:
WebsiteURL:
Value: !Sub http://${WebAppEIP}
Description: WebApp URL
WebServerPublicDNS:
Description: 'Public DNS of EC2 instance'
Value: !GetAtt WebAppInstance.PublicDnsName
WebAppDatabaseEndpoint:
Description: 'Connection endpoint for the database'
Value: !GetAtt WebAppDatabase.Endpoint.AddressIn the template above, we've also added new parameters to customize the name of the database, the username, and password, as well as two new outputs to make our work easier: WebServerPublicDNS and WebAppDatabasePublicDNS. We'll use both of these outputs in step 2.
If you're creating your stack from scratch, you can run the create-stack command:
$ aws cloudformation create-stack --stack-name rds-example --template-body file://05_rds.yaml \
--parameters ParameterKey=AvailabilityZone,ParameterValue=us-east-1a \
ParameterKey=EnvironmentType,ParameterValue=dev \
ParameterKey=KeyPairName,ParameterValue=jenna \
ParameterKey=DBPassword,ParameterValue=Abcd1234Or, if you're updating the stack you created in part 1, you can use the update-stack command instead.
Notice that we're only specifying the new DBPassword parameter and relying on the default values for DBInstanceIdentifier and DBUsername that are specified in the template. If you'd like to customize these values, you can add them to the command in the same ParameterKey,ParameterValue format shown above.
You may have also noticed the DeletionPolicy and UpdateReplacePolicy properties set to Snapshot. If and when you ever need to re-provision the database, you'll want to make sure you don't lose your precious data. Some template changes will require the resource to be recreated (as opposed to updated). When this happens, you'll want to be in control of how that happens and what happens to your data. These two properties give you that control. In the case of Snapshot, CloudFormation will create a snapshot of the database when the stack is updated or deleted. You can read more about update and delete behaviors of stack resources here.
After creating or updating the stack, you'll now have your EC2 instance (and supporting resources from part 1) and an RDS database. Right now, no one can access that database instance from the outside world, so next we'll enable inbound traffic to the Postgresql port.
To allow inbound traffic on port 5432 so that our EC2 instance can talk to the RDS database, we'll add a security group with type AWS::EC2::SecurityGroup.
# 06_rds.yaml
AWSTemplateFormatVersion: 2010-09-09
Description: Part 2 - Add a database with CloudFormation
Parameters:
AvailabilityZone:
Type: AWS::EC2::AvailabilityZone::Name
EnvironmentType:
Description: 'Specify the Environment type of the stack.'
Type: String
Default: dev
AllowedValues:
- dev
- test
- prod
AmiID:
Type: AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>
Description: 'The ID of the AMI.'
Default: /aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2
KeyPairName:
Type: String
Description: The name of an existing Amazon EC2 key pair in this region to use to SSH into the Amazon EC2 instances.
DBInstanceIdentifier:
Type: String
Default: 'webapp-db'
DBUsername:
NoEcho: 'true'
Description: Username for Postgresql database access
Type: String
MinLength: '1'
MaxLength: '16'
AllowedPattern: '[a-zA-Z][a-zA-Z0-9]*'
ConstraintDescription: Must begin with a letter and contain only alphanumeric characters.
Default: 'postgres'
DBPassword:
NoEcho: 'true'
Description: Password Postgresql database access
Type: String
MinLength: '8'
MaxLength: '41'
AllowedPattern: '[a-zA-Z0-9]*'
ConstraintDescription: Must contain only alphanumeric characters.
Mappings:
EnvironmentToInstanceType:
dev:
InstanceType: t2.nano
test:
InstanceType: t2.micro
prod:
InstanceType: t2.small
Resources:
WebAppInstance:
Type: AWS::EC2::Instance
Properties:
AvailabilityZone: !Ref AvailabilityZone
ImageId: !Ref AmiID
InstanceType: !FindInMap [EnvironmentToInstanceType, !Ref EnvironmentType, InstanceType]
KeyName: !Ref KeyPairName
SecurityGroupIds:
- !Ref WebAppSecurityGroup
WebAppSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: !Join [ '-', [ webapp-security-group, !Ref EnvironmentType ] ]
GroupDescription: 'Allow HTTP/HTTPS and SSH inbound and outbound traffic'
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 80
ToPort: 80
CidrIp: 0.0.0.0/0
- IpProtocol: tcp
FromPort: 443
ToPort: 443
CidrIp: 0.0.0.0/0
- IpProtocol: tcp
FromPort: 22
ToPort: 22
CidrIp: 0.0.0.0/0
WebAppEIP:
Type: AWS::EC2::EIP
Properties:
Domain: vpc
InstanceId: !Ref WebAppInstance
Tags:
- Key: Name
Value: !Join [ '-', [ webapp-eip, !Ref EnvironmentType ] ]
WebAppDatabase:
Type: AWS::RDS::DBInstance
Properties:
DBInstanceIdentifier: !Ref DBInstanceIdentifier
VPCSecurityGroups:
- !GetAtt DBEC2SecurityGroup.GroupId
AllocatedStorage: '5'
DBInstanceClass: db.t2.small
Engine: postgres
MasterUsername: !Ref DBUsername
MasterUserPassword: !Ref DBPassword
Tags:
- Key: Name
Value: !Join [ '-', [ webapp-rds, !Ref EnvironmentType ] ]
DeletionPolicy: Snapshot
UpdateReplacePolicy: Snapshot
DBEC2SecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: !Join [ '-', [ webapp-db-security-group, !Ref EnvironmentType ] ]
GroupDescription: Allow postgres inbound traffic
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 5432
ToPort: 5432
SourceSecurityGroupName:
Ref: WebAppSecurityGroup
Tags:
- Key: Name
Value: !Join [ '-', [ webapp-db-security-group, !Ref EnvironmentType ] ]
Outputs:
WebsiteURL:
Value: !Sub http://${WebAppEIP}
Description: WebApp URL
WebServerPublicDNS:
Description: 'Public DNS of EC2 instance'
Value: !GetAtt WebAppInstance.PublicDnsName
WebAppDatabaseEndpoint:
Description: 'Connection endpoint for the database'
Value: !GetAtt WebAppDatabase.Endpoint.AddressIn the template above, we've added the database security group and added it to the VPCSecurityGroups property. We've also set SourceSecurityGroupName to the WebAppSecurityGroup we created earlier, to restrict the inbound traffic to that coming from the WebAppSecurityGroup, or our EC2 instance.
To update your stack, you'll use the update-stack command with the same parameters and values as before:
$ aws cloudformation update-stack --stack-name rds-example --template-body file://06_rds.yaml \
--parameters ParameterKey=AvailabilityZone,ParameterValue=us-east-1a \
ParameterKey=EnvironmentType,ParameterValue=dev \
ParameterKey=KeyPairName,ParameterValue=jenna \
ParameterKey=DBPassword,ParameterValue=Abcd1234Now, you should have your EC2 instance (and supporting resources from part 1), an RDS database, and another security group exposing the database port. But how do we know the EC2 instance can talk to the database?
To test your security group to make sure your EC2 instance can talk to the RDS database you provisioned in the last step, you can SSH into the instance and use the psql client to connect to the database.
If you don't remember how to SSH into the instance, you can read more about that in part 1 or grab the WebServerPublicDNS from the Outputs. Using the command below, replace the WebServerPublicDNS and YOUR_KEY_PAIR_NAME parts and SSH into the intstance:
$ ssh -i "YOUR_KEY_PAIR_NAME.pem" ec2-user@WebServerPublicDNSOnce you're in the EC2 instance, you'll need to enable the postgresql library from the Amazon Linux Extras repository and then install it. We could do this all in our CloudFormation template, but this is only for demonstration purposes, so we'll do it manually this one time. You can run these commands to do that:
sudo amazon-linux-extras list # Find the postgresql library and version to enable in the next command
sudo amazon-linux-extras enable postgresql13 -y
sudo yum clean metadata
sudo yum install postgresqlNow that psql is available, you can test the connection to the database like this, replacing WebAppDatabaseEndpoint with the corresponding value in the Outputs:
$ psql -h WebAppDatabaseEndpoint -U postgres -d postgres
Password for user postgres:
psql (13.2, server 12.5)
SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, bits: 256, compression: off)
Type "help" for help.
postgres=>Congrats! You've successfully allowed your EC2 instance to talk to your RDS database.
If the connection isn't successful, check the CloudFormation console to make sure the RDS database and security group resources were created successfully. Additionally, you can check to make sure the database has the security group attached and the inbound rule opens up port 5432 (the default postgres port).
Don’t forget to delete your stack so you don’t accrue charges. You can do that with the delete-stack command:
$ aws cloudformation delete-stack --stack-name rds-exampleIf you left the DeletionPolicy and UpdateReplacePolicy properties set to snapshot and you no longer need those snapshots, then you can also delete those snapshots using the AWS Console so you don't accrue charges for those either.
Navigate to the RDS Management Console. From there, go to the Snapshots menu option. Select the snapshots created from your stack (hint: they will have a snapshot name that starts with your stack name) and select Delete snapshot from the Actions menu.
In this post, we updated the CloudFormation template from part 1 to provision an RDS database and enabled inbound traffic for the database. Now that both your EC2 instance and RDS database (and supporting resources) are all managed with code, you can setup and teardown the stack of resources together. This sets you up for part 3, where we'll create an AWS CodePipeline with CloudFormation (part 3) so we can build, test, and deploy our web app to the through each environment to production.
You can grab the final CloudFormation template we created here.
18