Blog
June 17, 2020 Marie H.

Envelope Encryption with AWS KMS in Go

Envelope Encryption with AWS KMS in Go

Envelope Encryption with AWS KMS in Go

Envelope encryption is the pattern behind every serious encryption-at-rest system. AWS KMS uses it. Google Cloud KMS uses it. IBM's own key management services use it. If you're storing encrypted data, you should understand this pattern and why it's designed the way it is.

The Problem with Encrypting Directly with KMS

The naive approach: send your data to KMS, get ciphertext back, store it. KMS handles everything. Simple.

This breaks in practice for several reasons. First, KMS has a payload size limit — AWS KMS caps it at 4KB per Encrypt call. You can't encrypt a database record, a file, or a stream directly. Second, every encrypt and decrypt operation is a network call to KMS with associated latency (typically 5-30ms) and cost ($0.03 per 10,000 API calls, which adds up at volume). Third, if KMS is unavailable, your application can't decrypt anything.

The right pattern: use KMS to protect a small symmetric key, and use that key locally to encrypt your actual data. This is envelope encryption.

How It Works

  1. Ask KMS to generate a data key. You get back two things: the plaintext data key and the same key encrypted with your Customer Master Key (CMK). KMS never sees your data.
  2. Use the plaintext data key to encrypt your data locally with AES-256-GCM.
  3. Store the encrypted data key alongside the ciphertext. Discard the plaintext data key from memory.
  4. To decrypt: send the encrypted data key to KMS to get the plaintext data key back, then decrypt your data locally.

The CMK never leaves AWS. KMS uses it only to wrap and unwrap data keys. The actual data encryption happens on your machines.

The Implementation

I'll build a EncryptFile / DecryptFile utility that demonstrates the full pattern. Using aws-sdk-go v1, which is what we're running.

package envelope

import (
    "context"
    "crypto/aes"
    "crypto/cipher"
    "crypto/rand"
    "encoding/binary"
    "fmt"
    "io"
    "os"

    "github.com/aws/aws-sdk-go/aws"
    "github.com/aws/aws-sdk-go/aws/session"
    "github.com/aws/aws-sdk-go/service/kms"
)

// File format:
// [4 bytes: encrypted key length][encrypted data key][nonce][ciphertext+tag]

const (
    nonceSize   = 12 // AES-GCM standard nonce size
    keySpec     = "AES_256"
)

type Encryptor struct {
    kmsClient *kms.KMS
    cmkID     string
}

func NewEncryptor(cmkID string) (*Encryptor, error) {
    sess, err := session.NewSession()
    if err != nil {
        return nil, fmt.Errorf("creating AWS session: %w", err)
    }
    return &Encryptor{
        kmsClient: kms.New(sess),
        cmkID:     cmkID,
    }, nil
}

Now the encrypt function:

func (e *Encryptor) EncryptFile(ctx context.Context, src, dst string) error {
    // Step 1: Generate a data key from KMS
    result, err := e.kmsClient.GenerateDataKeyWithContext(ctx, &kms.GenerateDataKeyInput{
        KeyId:   aws.String(e.cmkID),
        KeySpec: aws.String(keySpec),
    })
    if err != nil {
        return fmt.Errorf("generating data key: %w", err)
    }

    plaintextKey := result.Plaintext
    encryptedKey := result.CiphertextBlob

    // Zero the plaintext key when we're done with it
    defer func() {
        for i := range plaintextKey {
            plaintextKey[i] = 0
        }
    }()

    // Step 2: Read the plaintext file
    plaintext, err := os.ReadFile(src)
    if err != nil {
        return fmt.Errorf("reading source file: %w", err)
    }

    // Step 3: Encrypt locally with AES-256-GCM
    block, err := aes.NewCipher(plaintextKey)
    if err != nil {
        return fmt.Errorf("creating cipher: %w", err)
    }

    gcm, err := cipher.NewGCM(block)
    if err != nil {
        return fmt.Errorf("creating GCM: %w", err)
    }

    nonce := make([]byte, nonceSize)
    if _, err := rand.Read(nonce); err != nil {
        return fmt.Errorf("generating nonce: %w", err)
    }

    ciphertext := gcm.Seal(nil, nonce, plaintext, nil)

    // Step 4: Write the output file
    // Format: [4-byte key len][encrypted key][12-byte nonce][ciphertext]
    out, err := os.Create(dst)
    if err != nil {
        return fmt.Errorf("creating output file: %w", err)
    }
    defer out.Close()

    keyLen := make([]byte, 4)
    binary.BigEndian.PutUint32(keyLen, uint32(len(encryptedKey)))

    for _, chunk := range [][]byte{keyLen, encryptedKey, nonce, ciphertext} {
        if _, err := out.Write(chunk); err != nil {
            return fmt.Errorf("writing output: %w", err)
        }
    }

    return nil
}

And the decrypt path:

func (e *Encryptor) DecryptFile(ctx context.Context, src, dst string) error {
    // Step 1: Read and parse the encrypted file
    data, err := os.ReadFile(src)
    if err != nil {
        return fmt.Errorf("reading encrypted file: %w", err)
    }

    if len(data) < 4 {
        return fmt.Errorf("file too short to be valid")
    }

    keyLen := int(binary.BigEndian.Uint32(data[:4]))
    offset := 4

    if len(data) < offset+keyLen+nonceSize {
        return fmt.Errorf("file too short: expected key len %d", keyLen)
    }

    encryptedKey := data[offset : offset+keyLen]
    offset += keyLen
    nonce := data[offset : offset+nonceSize]
    offset += nonceSize
    ciphertext := data[offset:]

    // Step 2: Decrypt the data key with KMS
    result, err := e.kmsClient.DecryptWithContext(ctx, &kms.DecryptInput{
        CiphertextBlob: encryptedKey,
        KeyId:          aws.String(e.cmkID), // optional but explicit is better
    })
    if err != nil {
        return fmt.Errorf("decrypting data key: %w", err)
    }

    plaintextKey := result.Plaintext
    defer func() {
        for i := range plaintextKey {
            plaintextKey[i] = 0
        }
    }()

    // Step 3: Decrypt the data locally
    block, err := aes.NewCipher(plaintextKey)
    if err != nil {
        return fmt.Errorf("creating cipher: %w", err)
    }

    gcm, err := cipher.NewGCM(block)
    if err != nil {
        return fmt.Errorf("creating GCM: %w", err)
    }

    plaintext, err := gcm.Open(nil, nonce, ciphertext, nil)
    if err != nil {
        return fmt.Errorf("decrypting data: %w", err)
    }

    return os.WriteFile(dst, plaintext, 0600)
}

Note the explicit zeroing of plaintextKey after use. Go's GC doesn't guarantee when memory gets reclaimed or whether it might be swapped. Zeroing the key material immediately after use is the right habit, especially in a FIPS context.

Data Key Caching

If you're encrypting many small records, a GenerateDataKey call per record is expensive — both in latency and cost. AWS's Encryption SDK solves this with a data key cache. Here's a simpler version:

type cachedKey struct {
    plaintext  []byte
    ciphertext []byte
    expiresAt  time.Time
    uses       int
}

type KeyCache struct {
    mu         sync.Mutex
    kmsClient  *kms.KMS
    cmkID      string
    current    *cachedKey
    maxUses    int
    maxAge     time.Duration
}

func NewKeyCache(cmkID string, maxUses int, maxAge time.Duration) (*KeyCache, error) {
    sess, err := session.NewSession()
    if err != nil {
        return nil, err
    }
    return &KeyCache{
        kmsClient: kms.New(sess),
        cmkID:     cmkID,
        maxUses:   maxUses,
        maxAge:    maxAge,
    }, nil
}

func (c *KeyCache) GetDataKey(ctx context.Context) (plaintext, ciphertext []byte, err error) {
    c.mu.Lock()
    defer c.mu.Unlock()

    if c.current != nil && c.current.uses < c.maxUses && time.Now().Before(c.current.expiresAt) {
        c.current.uses++
        return c.current.plaintext, c.current.ciphertext, nil
    }

    // Need a fresh key
    result, err := c.kmsClient.GenerateDataKeyWithContext(ctx, &kms.GenerateDataKeyInput{
        KeyId:   aws.String(c.cmkID),
        KeySpec: aws.String(keySpec),
    })
    if err != nil {
        return nil, nil, fmt.Errorf("generating data key: %w", err)
    }

    // Zero out old key
    if c.current != nil {
        for i := range c.current.plaintext {
            c.current.plaintext[i] = 0
        }
    }

    c.current = &cachedKey{
        plaintext:  result.Plaintext,
        ciphertext: result.CiphertextBlob,
        expiresAt:  time.Now().Add(c.maxAge),
        uses:       1,
    }

    return c.current.plaintext, c.current.ciphertext, nil
}

The cache limits a single data key to maxUses encryptions and maxAge lifetime. I use 1000 uses / 5 minutes as defaults for bulk record encryption — adjust based on your security requirements and threat model.

Critical point: a cached plaintext key is sitting in your process memory. Don't increase maxAge past what your security policy allows. If your service gets memory-dumped, a longer-lived cached key is more exposure.

Decryption Caching

For decryption you can also cache: if you see the same encrypted data key again, you don't need to call KMS. A simple LRU cache keyed on the base64 of the encrypted key blob works:

// Using golang.org/x/tools or any LRU library
// Key: base64(encryptedDataKey) -> plaintext key
decryptCache *lru.Cache // thread-safe, bounded, TTL-based

Be careful with decryption caches. The security tradeoff is different: you're caching plaintext key material keyed off something an attacker might control. Bound the cache tightly — small size, short TTL.

IAM Policy

The IAM policy for the service role needs kms:GenerateDataKey for encryption and kms:Decrypt for decryption. Principle of least privilege: if a service only decrypts, don't give it GenerateDataKey.

{
    "Version": "2012-10-17",
    "Statement": [{
        "Effect": "Allow",
        "Action": ["kms:GenerateDataKey", "kms:Decrypt"],
        "Resource": "arn:aws:kms:us-east-1:123456789012:key/your-cmk-id"
    }]
}

Updated March 2026: The aws-sdk-go v1 (shown here) reached maintenance mode in 2023 — it still works but no new features are being added. The v2 SDK (github.com/aws/aws-sdk-go-v2) has a different API style: it uses context-first function signatures and the client configuration uses config.LoadDefaultConfig(ctx) instead of session.NewSession(). The KMS client call changes from kmsClient.GenerateDataKeyWithContext(ctx, input) to kmsClient.GenerateDataKey(ctx, input) — context is the first argument in all v2 operations, not a WithContext variant. The aws.String() helper still exists in v2. If you're starting a new project, use v2; the concepts and envelope encryption pattern remain identical.

Envelope encryption is the right default for data at rest. It gives you key rotation (rotate the CMK, re-wrap the data keys), key granularity (different CMKs for different data classifications), audit logging via KMS CloudTrail events, and the ability to revoke access by disabling the CMK. Build it in from the start — retrofitting encryption is painful.