AWS Secrets Manager, how to work with it, part 1

What is AWS Secrets Manager? AWS describes it very clearly on webpage: https://aws.amazon.com/secrets-manager/

AWS Secrets Manager helps you protect secrets needed to access your applications, services, and IT resources.

And basically, this is it. The biggest advantage of Secrets Manager is the possibility to rotate the keys in a very convenient way. But this will be the second part of this tutorial. In this part I am going to show you how to create and use keys in simple Lambda function.

Create Secrets

But before I create a Secret, some terms must be explained.

What is a Secret? It is a “storage” for credentials (and other data). This means that one Secret is not equal to one value, since a Secret can contain multiple data, like:

{ 
  "myUser": "Username",
  "myPassword": "secretpassword",
  "myConnectionstring": "mydatabase"  
}

Ok, so as this is clear, let’s check if there are any Secrets already created

$ aws secretsmanager list-secrets
{
  "SecretList": []
}

I received an empty list, which means I do not have any Secrets yet.

Before I create a Secret, I prepare a file with data for my Secret. Of course, I can run a command and add all values manually, but I prefer to use a file (AWS also recommends this way). Especially, if I have in mind CI/CD process. Of course, the file cannot be just like in the example, it must be properly stored and encrypted. But this is only an exercise.

So this is how I create my file (using bash):

$ touch mysecrets1.json
$ cat > mysecrets1.json << EOF
> {
> "firstName": "Pawel",
> "lastName": "Piwosz",
> "age": 45
> }
> EOF

Now I am ready to create my first Secret using this file

$ aws secretsmanager create-secret --name mySecret --description "My first Secret" --secret-string file://mysecrets1.json
{
  "ARN": "arn:aws:secretsmanager:eu-west-1:01234567890:secret:mySecret-sfer",
  "Name": "mySecret",
  "VersionId": "12345678–1abc-1234–1234–01234567890"

Let’s check if the Secret has actually been created

$ aws secretsmanager list-secrets
{
  "SecretList": [
    {
      "ARN": "arn:aws:secretsmanager:eu-west-1:01234567890:secret:mySecret-sfer",
      "Name": "mySecret",
      "Description": "My first Secret",
      "LastChangedDate": 1596409523.734,
      "SecretVersionsToStages": {
        "12345678–1abc-1234–1234–01234567890": [
          "AWSCURRENT"
        ]
      }
    }
  ]
}

Yes, the Secret is there. What is inside?

aws secretsmanager describe-secret --secret-id mySecret

This will return the same information about Secret. To see what is inside and what values I set there, I need to run

aws secretsmanager get-secret-value --secret-id mySecret

This command will return all values. There is an additional parameter to return the specific version, but I am not using it here.

At this point I have a Secret created. It is time to retrieve the data from it using a Lambda function.

Cool feature

A nice thing about Secrets Manager is that if I open the GUI and then navigate to my Secret, I can find there snippets prepared in many languages. I do not need to do any coding, just use them. So it is very easy to have basic understanding how to use Secrets in the code.

Create Lambda Function

How to demonstrate Secrets Manager in real life? Let’s suppose I have an application (for example serverless application) and I want to connect my Lambda functions to a specific resource. Database, Cache, some external API, you name it. I can hardcode the data. This will be the worst possible way of doing it (but still, not that uncommon, unfortunately).

I can use environment variables. Great! That may work. But first of all, everyone who has access to a Lambda console will see my variables. I don’t want to expose such data to all developers (for example). And another flaw of this is very simple. Imagine, I have 100 Lambda functions accessing the database and I want to change the password. It will be disaster — how long it will take?

That is why I prefer to use a “centralized” approach, and Secrets Manager helps me here.

I use the simplest approach. I copy the code from Secrets Manager’s snippet and use it in the simplest possible Lambda, but I change one thing. I don’t want to have a hardcoded location (or name) of the Secret, so I use Environment Variable for it. Why? Imagine this function is in CI/CD for multiple environments and there is a specific approach where all resources have environment in name. It is easier, and even should be recognized as the best practice, to have those kind of data as a variable.

This is my very simple sample code.

I use an example from AWS, but I have modified it a little. Anyway, no magic here. One important element to remember: The policy attached to the Lambda function must be able to reach the Secrets Manager. You need to add (and play a little, to be secure) something similar to the code below to your IAM policy.

{
  "Effect": "Allow",
  "Action": [
    "secretsmanager:*"
  ],
  "Resource": "*"
}

This is very simple, not really secured, and should be adjusted.

No matter how I put my code in AWS, I can use just GUI or CloudFormation, or SAM model. This is not the topic of this article.

When Lambda is deployed, I create the test case for it, in order to check if all is set.

The test case is very simple. I use just simple “hello world” test case, with an empty request. If all is ok, I should see the proper result. Well, it has worked:

Time for API Gateway

Lambda is here, so it is time to create a very simple API for this exercise. I use just GUI for it (now you know I didn’t use the SAM for deployment ;) ), as it is a very simple API with only one endpoint.

Shortly, I create a REST API Gateway, add getmysecret endpoint and connect it with the Lambda function. There are many tutorials how to do it.

When done, I can call my API and see how it works:

$ curl [https://oiferofd.execute-api.eu-west-1.amazonaws.com/test/getmysecret](https://ldyktznn84.execute-api.eu-west-1.amazonaws.com/test/getmysecret)
{"message": "{\n  \"firstName\": \"Pawel\",\n  \"lastName\": \"Piwosz\",\n  \"age\": 45\n}\n"}

Yes, it works :)

Clean the mess

Now it is time to clean up the sandbox. How to remove other resources, again, there is a lot of tutorials. For the secret, I simply run

aws secretsmanager delete-secret --secret-id mySecret

Please remember: Secrets are not “just removed” after delete command. They become unavailable, but can be restored in 7 days. After that time, they are really gone.

One more thing to remember: Secret can store many versions of data.

And the last thing, not touched in this tutorial (but mentioned at the very beginning). Secrets can be used to create automatic rotation of credentials. That is really great and very handy.