Blog
July 6, 2017 Marie H.

HashiCorp Vault: Stop Putting Secrets Everywhere

HashiCorp Vault: Stop Putting Secrets Everywhere

Photo by <a href="https://unsplash.com/@americanaez225?utm_source=cloudista&utm_medium=referral" target="_blank" rel="noopener">Arturo Añez</a> on <a href="https://unsplash.com/?utm_source=cloudista&utm_medium=referral" target="_blank" rel="noopener">Unsplash</a>

If you've ever found a database password in a Git commit, a hardcoded API key in a .env file, or passed credentials through an environment variable that ended up in a log somewhere — this post is for you. We've all done it. The obvious solution is LET'S AUTOMATE IT, but in this case the answer is actually: use HashiCorp Vault. It's a secrets management tool that centralizes, encrypts, and controls access to credentials. It took me a weekend to get comfortable with and I immediately started wondering why I waited this long.

The Problem with "Just Use Env Vars"

Environment variables aren't encrypted, they show up in /proc, they get logged, and there's no audit trail. "Who changed that DB password?" becomes an unanswerable question. Vault solves all of this: secrets are encrypted at rest, every read is logged, and access is controlled by policies.

Installing Vault (Dev Mode)

Dev mode is great for learning. Don't use it in production — it stores everything in memory and starts pre-unsealed with a root token. But for getting a feel for the tool, it's perfect.

$ wget https://releases.hashicorp.com/vault/0.8.0/vault_0.8.0_linux_amd64.zip
$ unzip vault_0.8.0_linux_amd64.zip
$ sudo mv vault /usr/local/bin/
$ vault --version
Vault v0.8.0

Start the dev server:

$ vault server -dev

==> Vault server configuration:

         Backend: inmem
         Listener 1: tcp (addr: "127.0.0.1:8200", tls: "disabled")
          Log Level: info
              Mlock: supported: true, enabled: false
            Version: Vault v0.8.0

==> WARNING: Dev mode is enabled!

In this mode, Vault is completely in-memory and unsealed.
Vault is configured to only have a single unseal key. The root
token has already been set to "s.xxxxxxxxxxxxxxxxxxxxxxxx". You do
not need to run 'vault init' as it has been done automatically.
Root Token: s.xxxxxxxxxxxxxxxxxxxxxxxx

Export the address and token in another terminal:

$ export VAULT_ADDR='http://127.0.0.1:8200'
$ export VAULT_TOKEN='s.xxxxxxxxxxxxxxxxxxxxxxxx'

Init and Unseal (for Real Deployments)

In production you initialize Vault once, which generates unseal keys and an initial root token. Vault uses Shamir's Secret Sharing — by default it produces 5 key shares and requires 3 to unseal.

$ vault init
Unseal Key 1: abc123...
Unseal Key 2: def456...
Unseal Key 3: ghi789...
Unseal Key 4: jkl012...
Unseal Key 5: mno345...
Initial Root Token: s.xxxxxxxxxxxxxxxxxxxxxxxx

$ vault unseal  # run 3 times with 3 different keys
Key (will be hidden):
Sealed: true
Key Shares: 5
Key Threshold: 3
Unseal Progress: 1

Store those unseal keys somewhere safe and separate. Not in the same place. Not in the same region. Not "I'll do it later."

The KV Secrets Engine

The Key/Value secrets engine is the most straightforward way to store static secrets. Enable it and start writing:

$ vault secrets enable -path=secret kv

$ vault write secret/myapp/database \
    host=mydb.internal \
    username=appuser \
    password=supersecretpassword

Success! Data written to: secret/myapp/database

$ vault read secret/myapp/database
Key                 Value
---                 -----
refresh_interval    768h0m0s
host                mydb.internal
password            supersecretpassword
username            appuser

You can also read it as JSON, which is what you'll want to do programmatically:

$ vault read -format=json secret/myapp/database
{
  "request_id": "a1b2c3d4-...",
  "data": {
    "host": "mydb.internal",
    "password": "supersecretpassword",
    "username": "appuser"
  }
}

Reading Secrets in Python with hvac

The hvac library is the official Python client for Vault. Install it:

$ pip install hvac

Reading the database credentials from above looks like this:

import hvac
import os

client = hvac.Client(
    url='http://127.0.0.1:8200',
    token=os.environ['VAULT_TOKEN']
)

# Read the secret
secret = client.read('secret/myapp/database')
db_host = secret['data']['host']
db_user = secret['data']['username']
db_pass = secret['data']['password']

# Connect to your DB with real credentials
import psycopg2
conn = psycopg2.connect(
    host=db_host,
    user=db_user,
    password=db_pass,
    dbname='myapp'
)

No credentials in code, no credentials in environment variables that escape into logs. The app authenticates to Vault using a token (more on token management below), Vault returns the secret.

Policies

The root token can do everything, which means you shouldn't use it for your app. Create a policy that only allows reading specific paths:

# myapp-policy.hcl
path "secret/myapp/*" {
  capabilities = ["read", "list"]
}
$ vault policy write myapp myapp-policy.hcl
Policy 'myapp' written.

Create a token tied to this policy:

$ vault token create -policy=myapp
Key            Value
---            -----
token          s.apptoken1234567
token_duration 768h0m0s
token_policies [myapp]

That token can read secret/myapp/* and nothing else. This is the token you give to your application. Not the root token.

Wrapping Up

This scratches the surface — Vault also does dynamic secrets (generate a short-lived DB user per request), PKI, AWS IAM credential generation, and a lot more. But even the basics here — KV secrets, policies, audit logging — are a massive improvement over the env var free-for-all most projects start with. Start with your most sensitive credentials, get comfortable with the CLI, then layer in the more advanced features as you need them.