Guides
GuidesAPISupportStatusCommunitySign Up

GCP Secret Manager

reading time 3 mins

This guide will show you how to set up automatic syncing of Doppler secrets to GCP Secret Manager.

Prerequisites

IAM Service Account

📘

Ensure gcloud is configured to use the correct project, e.g. gcloud config set project yodaspeak before proceeding.

We need to set up a Service Account so Doppler has the required permissions to sync secrets to GCP Secret Manager.

  • Adjust the SECRET_PREFIX as desired - this limits Doppler's access to only GCP Secrets starting with this prefix.
  • The prefix may only contain alphanumeric characters, dashes, and underscores - a dash is recommended for the last character.

This is best done using the gcloud CLI:

# To narrow permission scope use this prefix for Doppler accessible secrets, adjust as desired
# This should match the prefix you enter in Doppler when setting up your integration
SECRET_PREFIX="doppler-";

# Get current project
PROJECT_ID="$(gcloud config get-value project --quiet)";

# Create a new Service Account
gcloud iam service-accounts create doppler-secret-manager \
  --description="Service account for Doppler to sync secrets to Secret Manager" \
  --display-name="Doppler Secret Manager";

# Attach SecretManagerAdmin policy to the new service account
gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member="serviceAccount:doppler-secret-manager@$PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/secretmanager.admin" \
  --condition="expression=resource.name.extract(\"secrets/{rest}\").startsWith(\"$SECRET_PREFIX\"),title=\"$SECRET_PREFIX*\"";

Then we create a new key for the service account to generate the required credentials for Doppler:

# Generate a key for your new service account
gcloud iam service-accounts keys create iam-key.json \
  --iam-account="doppler-secret-manager@$PROJECT_ID.iam.gserviceaccount.com";

# Print (and then remove) the JSON credentials
cat iam-key.json && rm iam-key.json;

Copy the JSON credentials output in your shell as you'll need it for the next step.

Authorization

Navigate to the project you would like to integrate, click Integrations from the Projects menu, then select GCP Secret Manager to begin the authorization process.

1646

Enter the prefix used above (e.g., "doppler-"), paste the JSON from the shell into the credentials text field, then click Connect.

Configuration

Now chose the config to sync, the Region(s), and the enter a secret Name.

For region, Automatic replication is recommended, but you can instead specify which regions secrets should be replicated to. Learn more in the GCP Secret Manager replication docs.

Enter a Name (alphanumeric characters, dashes, and underscores) which will be concatenated to the prefix to display the Secret Manager Name (the GCP secret where Doppler will sync your secrets).

Click "Set Up Integration," and you're all set! Click the DESTINATION link in the table to see your secrets in the GCP console.

🚧

Doppler sync and secret versions

Every time a secret is changed in Doppler, this will create a new version of the secret in GCP Secret Manager, so ensure sure your code always retrieves the latest version using /versions/latest.

Import Secrets

The GCP Secret Manager integration doesn't currently support importing secrets. However, it's possible to import secrets programmatically using a script like this:

#!/bin/bash

# Check if jq is installed
if ! command -v jq &> /dev/null; then
    echo "jq could not be found, please install jq to process JSON data."
    exit 1
fi

# Check if gcloud is installed
if ! command -v gcloud &> /dev/null; then
    echo "gcloud could not be found, please install the Google Cloud CLI."
    exit 1
fi

# Check if doppler is installed
if ! command -v doppler &> /dev/null; then
    echo "doppler could not be found, please install the doppler CLI."
    exit 1
fi

# Configuration variables
gcp_project="${GCP_PROJECT}"
gcp_prefix="${GCP_PREFIX}"
doppler_project="${DOPPLER_PROJECT}"
doppler_config="${DOPPLER_CONFIG}"

date
echo "========================================"
echo "Initiating import from GCP to Doppler..."
echo "========================================"
echo "-------------- SOURCE ------------------"
echo "        GCP Project: $gcp_project"
echo "  GCP Secret Prefix: $gcp_prefix"
echo "------------ DESTINATION ---------------"
echo "  Doppler Project: $doppler_project"
echo "   Doppler Config: $doppler_config"
echo "----------------------------------------"
echo

# We're going to store all secrets in this variable in .env format. Note that we're not
# really doing any special character escaping with values, so be mindful of that if you
# have secrets that contain special characters or double-quotes.
secrets_to_import=""

# Iterate through all secrets in the specified GCP project that have the specified prefix in their name.
# Note that this filter is just looking for the prefix ANYWHERE in the name, it's not specifically limited
# to actually being a prefix.
for secret in $(gcloud secrets list --project $gcp_project --filter $gcp_prefix --format=json | jq -r '.[] | .name'); do
  # Extract the secret name from the last portion of the fully specified secret name
  secret_name=$(echo $secret | rev | cut -d'/' -f1 | rev)

  # Remove the prefix from the secret name
  secret_name=${secret_name//"$gcp_prefix"/}

  echo "fetching ${secret_name}..."
  # Fetch the secret value
  secret_value=$(gcloud secrets versions access --project $gcp_project "${secret}/versions/latest")

  # Append the secret to the result
  secrets_to_import="${secrets_to_import}\\n${secret_name}=\"${secret_value}\""
done

# Upload the secrets to the specified Doppler project and config
doppler secrets upload -p $doppler_project -c $doppler_config --silent <(echo -e "$secrets_to_import")

echo
date
echo "Import complete."

The script expects for arguments to be passed in as environment variables (you could also modify the script to accept them as arguments): the GCP project (GCP_PROJECT), the GCP secret prefix (GCP_PREFIX), the Doppler project (DOPPLER_PROJECT) and the Doppler config (DOPPLER_CONFIG). It functions by listing all secrets in the specified GCP project that contain the specified prefix in the name. It will then iterate through them, remove the prefix from the name and import them into the specified Doppler project and config.

Note that the script doesn't perform any special character escaping and simply wraps the secret values in double-quotes, so be mindful of this when using the script.

👍

Amazing Work!

You've successfully set up the Doppler GCP Secret Manager integration! Every time you update your secrets in Doppler, we will automatically sync them to GCP Secret Manager, creating a new version of that secret.

Updated 6 days ago