Skip to content
BranchesTags

Repository files navigation

Go Report Card

treasury

Treasury is a very simple tool for managing secrets. It uses Amazon S3 or SSM (Systems Manager Parameter Store) service to store secrets. By default, Treasury uses SSM as a backend. The secrets are encrypted before saving them on disks in Amazon data centers and decrypted when being read. Treasury uses Server-Side Encryption with AWS KMS-Managed Keys (SSE-KMS).

Architecture

Architecture overwiev

Command Line interface (CLI)

Treasury is controlled via a very easy to use command-line interface (CLI). Treasury is only a single command-line application: treasury. This application takes a subcommand such as "read", "write", "import" or "export".

The Treasury CLI is a well-behaved command line application. In erroneous cases, a non-zero exit status will be returned. It also responds to -h and --help as you'd most likely expect.

To view a list of the available commands at any time, just run treasury with no arguments. To get help for any specific subcommand, run the subcommand with the -h argument.

Configuration

AWS Credentials

Before using the Treasury CLI, ensure that you've configured AWS credentials. The best way to configure credentials on your machine is to use the ~/.aws/credentials file, which might look like:

[default]
aws_access_key_id = AKID1234567890
aws_secret_access_key = MY-SECRET-KEY

Alternatively, you can set the following environment variables:

AWS_ACCESS_KEY_ID=AKID1234567890
AWS_SECRET_ACCESS_KEY=MY-SECRET-KEY

You can also use non-default awscli profile:

AWS_PROFILE=development treasury read development/webapp/cockpit_api_pass`

And non-default awscli profile without default region:

AWS_PROFILE=development ./treasury --region eu-west-1 read test/webapp/cockpit_pass`

SSM store configuration

Treasury uses SSM store by default. No additional configuration is required.

S3 store configuration

In order to use S3 as a store, set TREASURY_S3 environment variable with S3 bucket name.

For example:

export TREASURY_S3=ah-dev-treasury-development

In order to use SSM as a store, unset previously configured TREASURY_S3 environment variable.

AWS Region configuration

In addition to the IAM credentials you'll need to specify the AWS region. You can specify the region either with an environment variable (example below), or directly in AWS configuration file $HOME/.aws/config. More about configuration here.

export AWS_DEFAULT_REGION=eu-west-1

Installation

on OSX:

brew tap airhelp/taps git@github.com:AirHelp/homebrew-taps.git
brew update
brew install treasury

CLI Usage

Write secret

The secret value is never passed as a command line argument, so it does not leak into the shell history nor into the process list. Treasury asks for it and hides the input:

> treasury write development/webapp/cockpit_api_pass
Please paste your secret:
Success! Data written to: development/webapp/cockpit_api_pass (19 characters)

In scripts and CI, where there is no terminal, the secret is read from the standard input:

> echo "${SECRET}" | treasury write development/webapp/cockpit_api_pass
Success! Data written to: development/webapp/cockpit_api_pass (19 characters)

The single trailing newline that echo adds is stripped, so there is no need for echo -n (which is not portable between shells anyway). Any other whitespace is treated as a part of the secret. If the secret has to end with a newline, use printf '%s\n' "${SECRET}" or write it from a file with --file.

Do not put the secret itself in the pipe - echo thisIsASecret | treasury write KEY leaks into the shell history exactly like the old treasury write KEY thisIsASecret did. Pipe a variable or another command (echo "${SECRET}" | ...), and type secrets by hand only into the interactive prompt.

Note: if secret value is equal to existing one, write is skipped. --force flag can be used to overwrite.

Write file content

> treasury write development/webapp/credentials_yaml <filename> --file
Success! Data written to: development/webapp/credentials_yaml

Note: if secret value is equal to existing one, write is skipped. --force flag can be used to overwrite. Stored file content in key is gzipped and converted to base64. You can decode it with bash script easily:

> treasury read  development/webapp/credentials_yaml | base64 -D | gzip -d > creadentails.yaml

or you can handle it with your application. Size limit depends on backend that you are using - for AWS SSM size limit is 4096 bytes (after gzip and base64).

Read secret

> treasury read development/webapp/cockpit_api_pass
superSecretPassword

List secrets

List the secrets set for a path

> treasury list development/application/
development/application/app_api_pass
development/application/test

Delete secret

Delete the secret from treasury

> treasury delete development/application/secret_key
Key development/application/secret_key has been successfully deleted

Import secrets

Assuming properties file ./secrets.env with content:

ke1=secret1
key2=secret2

To import these values into a previously configured store:

> treasury import development/application/ ./secrets.env
Import successful

Export secrets

Assuming stored secrets pairs on treasury store

development/webapp/key1 => superSecretPassword1
development/webapp/key2 => superSecretPassword2

To see exported values:

> treasury export development/webapp/
export key1=superSecretPassword1
export key2=superSecretPassword2

To export them into shell environment variables:

eval $(treasury export development/webapp/)

Run a command with secrets

Runs a command with secrets exported as environment variables, without ever writing them to disk.

The secrets are described in an environment file, .env.treasury by default. A complete example:

# Comments and empty lines are ignored.

# Exports all secrets from the path, each one named after the last part of the
# key, so development/mobile-app-gateway/API_TOKEN becomes API_TOKEN
{{ export "development/mobile-app-gateway/" }}

# Exports a single secret under a name of your choice
USER_PROFILES_API_PASSWORD={{ read "development/user-profiles/API_PASSWORD_MOBILE_GATEWAY" }}
AUTH_API_PASSWORD={{ read "development/auth/INTERNAL_API_BASIC_AUTH_PASSWORD" }}
AUTH_ACCESS_TOKEN_SECRET={{ read "development/auth/ACCESS_TOKEN_SECRET" }}

# Plain values are taken as they are, no secret store involved
RAILS_ENV=development
API_TOKEN=test

Every line is a Go template, and read and export are the directives you know from the template command. read inserts the value of a single secret, export turns a whole path into entries, one per secret. Entries are applied in the order of appearance, so a later entry overrides an earlier one with the same name.

> treasury run bundle exec rake db:migrate
> treasury run --env-file .env.staging -- rails server

The AWS profile is taken from the AWS_PROFILE environment variable, use --profile to pick another one:

> treasury run --profile development bundle exec rails console

Treasury steps out of the way of the command it runs. Signals it receives, a SIGTERM from a supervisor for example, are passed to the command, and its exit code becomes the exit code of treasury: 127 when the command does not exist, 126 when it cannot be executed and 128 + signal number when it is killed.

Template usage

Render the template on disk at /tmp/template.tpl to /tmp/result:

treasury template --src /tmp/template.tpl --dst /tmp/result

Treasury parses file in the Go Template format. The input text for a template is UTF-8-encoded text in any format. "Actions"--data evaluations or control structures--are delimited by "{{" and "}}"; all text outside actions is copied to the output unchanged.

Template usage with string append to secret value

treasury template --src /tmp/template.tpl --dst /tmp/result --append key1:v2

This command ends up with output file where the value of variable key1 has a string "v2" appended.

Template usage with variables interpolation

Example template:

APPNAME={{ .AppName }}
API_PASSWORD={{ .ApiPassword }}

treasury template --src /tmp/template.tpl --dst /tmp/result.env --env AppName=test,ApiPassword=qwerty12345

Supported actions:

read

DEPRECATED (please use readFromEnv) Returns single value for given key

{{ read "ENVIRONMENT/APPLICATION/SECRET_NAME" }}

Example:

COCKPIT_API_PASSWORD={{ read "production/cockpit/cockpit_api_password" }}

readFromEnv

Returns single value for given key in specified environment

{{ readFromEnv "ENVIRONMENT" "APPLICATION/SECRET_NAME" }}

# example using interpolation:

{{ readFromEnv .Environment "app/API_PASSWORD" }}

export

DEPRECATED (please use exportFromEnv) Returns all values for a given path in key=value format

{{ export "development/treasury/" }}

will generate:

key1=secret1
key2=secret2
key3=secret3
key4=secret4

exportFromEnv

Returns all values from given environment and given key in key=value format

{{ exportFromEnv "development" "treasury" }}

# or using interpolation

{{ exportFromEnv .Environment "treasury" }}

will generate:

key1=secret1
key2=secret2
key3=secret3
key4=secret4

exportMap

Returns all values for a given path in Go map structure

{{ range $key, $value := exportMap "development/treasury/" }} {{ $key }}: {{ $value }}{{ end }}

will generate:

key1: secret1
key2: secret2
key3: secret3
key4: secret4

Setting up the infrastructure

IAM Policy for S3 store

  • Read and Write policy for test/test/* and test/cockpit/* keys
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1491319766000",
            "Effect": "Allow",
            "Action": [
                "kms:Encrypt",
                "kms:Decrypt",
                "kms:ListAliases",
                "kms:ListKeys",
                "kms:GenerateDataKey*"
            ],
            "Resource": [
                "arn:aws:kms:eu-west-1:064764542321:key/14b4a163-6c9d-4edb-a4bf-5adc4cd50ad8"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::TREASURY_S3_BUCKET_NAME"
            ]
        },
        {
            "Sid": "Stmt1491319793000",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject*",
                "s3:GetObject*",
                "s3:DeleteObject*"
            ],
            "Resource": [
                "arn:aws:s3:::TREASURY_S3_BUCKET_NAME/test/test/*",
                "arn:aws:s3:::TREASURY_S3_BUCKET_NAME/test/cockpit/*"
            ]
        }
    ]
}
  • Read only policy for test/* keys
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1491319766000",
            "Effect": "Allow",
            "Action": [
                "kms:Decrypt",
                "kms:ListAliases",
                "kms:ListKeys",
            ],
            "Resource": [
                "arn:aws:kms:eu-west-1:064764542321:key/14b4a163-6c9d-4edb-a4bf-5adc4cd50ad8"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::TREASURY_S3_BUCKET_NAME"
            ]
        },
        {
            "Sid": "Stmt1491319793000",
            "Effect": "Allow",
            "Action": [
                "s3:GetObject*"
            ],
            "Resource": [
                "arn:aws:s3:::TREASURY_S3_BUCKET_NAME/test/*"
            ]
        }
    ]
}
  • Example AWS S3 Bucket Policy

The following bucket policy denies upload object (s3:PutObject) permission to everyone if the request does not include the x-amz-server-side-encryption header requesting server-side encryption with SSE-KMS.

{
  "Version": "2012-10-17",
  "Id": "PutObjPolicy",
  "Statement": [
    {
      "Sid": "DenyIncorrectEncryptionHeader",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::TREASURY_S3_BUCKET_NAME/*",
      "Condition": {
        "StringNotEquals": {
          "s3:x-amz-server-side-encryption": "aws:kms"
        }
      }
    },
    {
      "Sid": "DenyUnEncryptedObjectUploads",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::TREASURY_S3_BUCKET_NAME/*",
      "Condition": {
        "Null": {
          "s3:x-amz-server-side-encryption": true
        }
      }
    }
  ]
}

IAM Policy for SSM Store

  • Read only policy for development/cockpit/* keys
{
  "Version": "2012-10-17",
  "Statement": [
      {
          "Effect": "Allow",
          "Action": [
              "ssm:DescribeParameters",
              "ssm:GetParameterHistory",
              "ssm:ListTagsForResource"
          ],
          "Resource": "*"
      },
      {
          "Effect": "Allow",
          "Action": [
              "ssm:GetParameter*"
          ],
          "Resource": [
              "arn:aws:ssm:eu-west-1:064764542321:parameter/development/cockpit/*"
          ]
      },
      {
          "Effect": "Allow",
          "Action": [
              "kms:Decrypt"
          ],
          "Resource": [
              "arn:aws:kms:eu-west-1:064764542321:key/14b4a163-6c9d-4edb-a4bf-5adc4cd50ad8"
          ]
      }
  ]
}
  • Read and Write policy for /development/application/* keys
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ssm:DescribeParameters",
                "ssm:GetParameterHistory",
                "ssm:ListTagsForResource"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "ssm:GetParameter*",
                "ssm:PutParameter",
                "ssm:DeleteParameter"
            ],
            "Resource": [
                "arn:aws:ssm:eu-west-1:064764542321:parameter/development/application/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "kms:Encrypt",
                "kms:Decrypt",
                "kms:GenerateDataKey*"
            ],
            "Resource": [
                "arn:aws:kms:eu-west-1:064764542321:key/14b4a163-6c9d-4edb-a4bf-5adc4cd50ad8"
            ]
        }
    ]
}

Treasury as a user vault

You can now use the treasure as a user vault with minimal policy change. Including the following statement in the default policy allows users to manage their secrets.

  statement {
    actions = [
      "ssm:*",
    ]

    resources = [
      "arn:aws:ssm::${var.aws_account_id}:parameter/user/$${aws:username}",
      "arn:aws:ssm::${var.aws_account_id}:parameter/user/$${aws:username}/*",
    ]
  }
  • Write user/marcin.janas/phone
$ treasury write user/firstname.lastname/phone
Please paste your secret:
Success! Data written to: user/firstname.lastname/phone (13 characters)
  • Read user/firstname.lastname/phone
$ treasury read user/firstname.lastname/phone
+48987654321

Go Client

Example for S3 as a store:

import "github.com/AirHelp/treasury/client"

// use default client options
treasury, err := client.New(&client.Options{
    Backend: "s3",
    S3BucketName: TREASURY_S3_BUCKET_NAME,
})
secret, err := treasury.Read(key)
if err != nil {
  return err
}

fmt.Println(secret.Value)

TREASURY_S3_BUCKET_NAME is a S3 bucket name dedicated for treasury use.

Example for SSM as a store:

import "github.com/AirHelp/treasury/client"

// use default client options
treasury, err := client.New(&client.Options{
    Backend: "ssm",
})
secret, err := treasury.Read(key)
if err != nil {
  return err
}

fmt.Println(secret.Value)

** TO DO: add terraform resource **

Development

Build for development

make build

Tests

Go tests

make test

Bats tests

AWS_PROFILE=development make testall

If bats missing, install it:

brew install bats

About

A tool for managing secrets.

Topics

aws-iamaws-s3clidevopsdevops-teamgogolangiammanaged-by-terraforms3secrets

Resources

Readme
Apache-2.0 license
Activity
Custom properties

Stars

12 stars

Watchers

42 watching

Forks

3 forks
Report repository

Releases

Packages

Used by

Contributors

Languages