AWS Secrets Rotation

Doppler rotates secrets that reside within AWS infrastructure by utilizing AWS Lambda as a proxy. There's no need to expose your services to the internet or grant Doppler access to your services.

Requirements

📷

Requires an upgraded subscription

This feature is exclusive to our Enterprise Plan. Book a demo to see it in action.

• Understand the Doppler rotation methodology
• AWS Account
  • Ability to create IAM roles and Policies
  • Ability to create Lambda functions
  • Ability to create AWS Gateway Endpoints

Overview

To rotate secrets for any system within AWS, Doppler invokes a Lambda function that you create and own within your cloud account using a minimally scoped IAM role. Each Lambda function is open source, signed, and hosted in S3. Each Lambda invocation payload is signed and verified using our public key, which is also hosted in S3.

Primitives

Each AWS rotated secret, regardless of the type, utilizes the same AWS primitives.

For each primitive, there may be customization required depending on your architecture - e.g. VPC peering. As well, the service you are rotating may need its own configuration, which will be covered in its own dedicated doc.

Supported Services

• MySQL (RDS or self-hosted)
• Postgres (RDS or self-hosted)

Rotated Secret Creation

1. Navigate to the secrets config that the rotated secret will reside in
2. Go to the Advanced Secrets tab
3. Select New Rotated Secret

4. Select the service you'd like to rotate and then select AWS

Lambda Setup

Executor Role

The Lambda executor role is the authorization context Lambda will execute in; this is not the role Doppler will assume into to invoke the Lambda. We describe the minimum grants below, but there's a chance your role will require additional grants based on what service it is rotating and how you've architected your infrastructure.

1. In your AWS account, navigate to IAM Roles
2. Create Role
3. Use AWS service for Trusted entity type
4. Use Lambda for Use Case
5. Under permissions, select AmazonS3ReadOnlyAccess and AWSLambdaVPCAccessExecutionRole
6. Name your role and optionally provide a description
7. Create

Lambda Function

Each service that is rotated uses a Lambda function powered by an open source rotator. You can use the same Lambda function to rotate n instances of the same service - i.e. you can use a single MySQL Lambda rotator to rotate as many MySQL secrets on as many MySQL instances as you'd like.

1. In your AWS account - most likely in the region of the app or service you're rotating - navigate to Lambda
2. Create a new function
3. Name your function, ensuring the name begins with Doppler_. This allows Doppler to use more finely scope access.
4. Set Node.js 16.x as the runtime
5. Expand the permissions section to expose the executor role configuration. Choose the role you created in the section above .
6. Expand Advanced settings. Ensure Enable Code signing is checked and configure the settings as follows
   6.1 Code signing configuration: Create new configuration
   6.2 Description: Doppler code signing profile
   6.3 Signing profile ARN: arn:aws:signer:us-east-1:299900769157:/signing-profiles/DopplerSecretAgents/7zfqycFf2B
   6.4 Signature validation policy for code: Enforce
   6.5 Code location: Select the appropriate S3 URL from the table

7. Enable VPC: You will most likely want your function to live in the same VPC as the resource being rotated. If not, you must ensure it can peer into the applicable VPC to reach the service being rotated.
8. Create your function
9. Scroll to the Runtime settings section and click edit. Update the handler with the corresponding value from the table below

🚧

Increasing Lambda Timeout

The default timeout for a Lambda function is 3 seconds. For larger instance sizes, that will probably work fine, but if your database is particularly busy or running on a smaller instance size, you may need to increase the Lambda function timeout.

You'll see an error like this during the setup process in the Doppler dashboard if you're running into this:
The Lambda agent timed out during execution. This can be caused by a network connection issue. Check your network configuration and verify that your Lambda can reach the database host.

IAM Role for Doppler to assume into

Policy

1. In your AWS account, navigate to IAM Policies
2. Click Create Policy
3. Click the JSON tab
4. Enter the following policy below after adding your Lambda's ARN in the resource field.
   1. Note: lambda:UpdateFunctionCode is optional but necessary if you'd like Doppler to be able to perform automatic updates. Otherwise, you'll need to manually upgrade to new versions.
5. Optionally add any tags
6. Name your policy and click create policy

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "lambda:GetFunction",
        "lambda:InvokeFunction",
        "lambda:UpdateFunctionCode"
      ],
      "Effect": "Allow",
      "Resource": "YOUR_LAMBDA_ARN"
    },
     {
            "Action": [
                "s3:GetObject"
            ],
            "Effect": "Allow",
            "Resource": "arn:aws:s3:::doppler-keys-*/*"
        }
  ]
}

Role

1. In your AWS account, navigate to IAM Roles
2. Click Create Role
3. Select AWS account for Trusted entity type
4. Check Another AWS account
5. For Account ID enter 299900769157
6. Check Require external ID
   1. Enter your workplace slug for the External ID value. You can obtain your workplace slug by visiting the Doppler dashboard. In the URL, grab the value after /workplace/
7. Attach the policy you created above.

S3 Gateway Endpoint (Optional)

If your VPC does not have outbound access enabled, a S3 Gateway Endpoint is required to reach Doppler's public key. An AWS Gateway Endpoint enables your Lambda to communicate with our S3 bucket in order to retrieve our public key for payload signing verification.

1. In your AWS account, navigate to VPC
2. Click Endpoints and Create Endpoint
3. Name your endpoint
4. Leave Service Category as AWS Services
5. In the Services section search for S3
6. Select the endpoint with a type of Gateway. It will include the region in its name, e.g. com.amazonaws.us-east-1.s3
7. Select the VPC that the Lambda rotator resides in
8. Select the appropriate route table from your VPC
9. A policy is not required

Service Configuration

After the Lambda function is setup, you can move on to the next screen, where you'll configure service-specific settings. Select the appropriate page below to carry on

• AWS MySQL
• AWS Postgres

Automatic Lambda function updates

If the role provided to Doppler is given lambda:UpdateFunctionCode, we will automatically apply updates when available. If the lambda:UpdateFunctionCode is omitted, updates will need to be performed manually.

Manual Lambda function updates

If you do not grant lambda:UpdateFunctionCode to the Doppler IAM role, you will be responsible for manually updating your Lambda function.

1. In the AWS console, navigate to the Lambda function
2. In the Code Source section, click the Upload from drop-down and select Amazon S3 Location
3. Enter the appropriate S3 URL for your AWS region from the here